Jul 13 18:32:23.594926 ip-10-0-129-191 systemd[1]: Starting Kubernetes Kubelet... Jul 13 18:32:23.982533 ip-10-0-129-191 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 18:32:23.982533 ip-10-0-129-191 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 13 18:32:23.982533 ip-10-0-129-191 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 18:32:23.982533 ip-10-0-129-191 kubenswrapper[2582]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 13 18:32:23.982533 ip-10-0-129-191 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 18:32:23.983517 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.983427 2582 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 13 18:32:23.989565 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989549 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 18:32:23.989565 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989565 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 18:32:23.989643 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989569 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 18:32:23.989643 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989572 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 18:32:23.989643 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989575 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 18:32:23.989643 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989578 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 18:32:23.989643 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989581 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 18:32:23.989643 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989584 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 18:32:23.989643 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989586 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 18:32:23.989643 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989589 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 18:32:23.989643 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989591 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 18:32:23.989643 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989594 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 18:32:23.989643 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989610 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 18:32:23.989643 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989615 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 18:32:23.989643 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989618 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 18:32:23.989643 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989621 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 18:32:23.989643 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989624 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 18:32:23.989643 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989626 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 18:32:23.989643 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989629 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 18:32:23.989643 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989632 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 18:32:23.989643 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989634 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 18:32:23.989643 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989637 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 18:32:23.990121 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989640 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 18:32:23.990121 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989642 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 18:32:23.990121 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989645 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 18:32:23.990121 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989647 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 18:32:23.990121 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989650 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 18:32:23.990121 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989653 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 18:32:23.990121 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989656 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 18:32:23.990121 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989659 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 18:32:23.990121 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989662 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 18:32:23.990121 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989664 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 18:32:23.990121 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989667 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 18:32:23.990121 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989670 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 18:32:23.990121 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989672 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 18:32:23.990121 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989675 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 18:32:23.990121 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989677 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 18:32:23.990121 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989680 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 18:32:23.990121 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989682 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 18:32:23.990121 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989685 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 18:32:23.990121 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989687 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 18:32:23.990121 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989690 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 18:32:23.990641 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989693 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 18:32:23.990641 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989695 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 18:32:23.990641 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989697 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 18:32:23.990641 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989701 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 18:32:23.990641 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989705 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 18:32:23.990641 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989708 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 18:32:23.990641 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989712 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 18:32:23.990641 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989714 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 18:32:23.990641 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989718 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 18:32:23.990641 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989720 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 18:32:23.990641 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989723 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 18:32:23.990641 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989725 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 18:32:23.990641 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989728 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 18:32:23.990641 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989731 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 18:32:23.990641 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989734 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 18:32:23.990641 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989737 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 18:32:23.990641 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989740 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 18:32:23.990641 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989743 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 18:32:23.990641 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989746 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 18:32:23.991109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989748 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 18:32:23.991109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989751 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 18:32:23.991109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989754 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 18:32:23.991109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989757 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 18:32:23.991109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989759 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 18:32:23.991109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989762 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 18:32:23.991109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989766 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 18:32:23.991109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989768 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 18:32:23.991109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989771 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 18:32:23.991109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989773 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 18:32:23.991109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989776 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 18:32:23.991109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989779 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 18:32:23.991109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989781 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 18:32:23.991109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989784 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 18:32:23.991109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989786 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 18:32:23.991109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989788 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 18:32:23.991109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989791 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 18:32:23.991109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989793 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 18:32:23.991109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989795 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 18:32:23.991109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989798 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 18:32:23.991570 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989800 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 18:32:23.991570 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989803 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 18:32:23.991570 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989805 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 18:32:23.991570 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989807 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 18:32:23.991570 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989810 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 13 18:32:23.991570 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989812 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 18:32:23.991570 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989816 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 18:32:23.991570 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989819 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 18:32:23.991570 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989821 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 18:32:23.991570 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989824 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 18:32:23.991570 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989826 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 18:32:23.991570 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989829 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 18:32:23.991570 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989831 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 18:32:23.991570 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989834 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 18:32:23.991570 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989836 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 18:32:23.991570 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989839 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 18:32:23.991570 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989841 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 18:32:23.991570 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.989843 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 18:32:23.991570 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990550 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 18:32:23.991570 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990557 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 18:32:23.992060 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990560 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 18:32:23.992060 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990563 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 18:32:23.992060 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990565 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 18:32:23.992060 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990568 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 18:32:23.992060 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990571 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 18:32:23.992060 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990574 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 18:32:23.992060 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990576 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 18:32:23.992060 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990579 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 18:32:23.992060 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990581 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 18:32:23.992060 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990584 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 18:32:23.992060 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990586 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 18:32:23.992060 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990588 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 18:32:23.992060 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990591 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 18:32:23.992060 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990593 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 18:32:23.992060 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990595 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 18:32:23.992060 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990598 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 18:32:23.992060 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990622 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 18:32:23.992060 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990626 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 18:32:23.992060 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990632 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 18:32:23.992060 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990636 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 18:32:23.992545 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990639 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 18:32:23.992545 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990642 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 18:32:23.992545 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990645 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 18:32:23.992545 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990648 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 18:32:23.992545 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990650 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 18:32:23.992545 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990653 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 18:32:23.992545 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990655 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 18:32:23.992545 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990658 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 18:32:23.992545 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990661 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 18:32:23.992545 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990664 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 18:32:23.992545 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990666 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 18:32:23.992545 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990669 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 18:32:23.992545 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990671 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 18:32:23.992545 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990673 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 18:32:23.992545 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990676 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 18:32:23.992545 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990678 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 18:32:23.992545 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990682 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 18:32:23.992545 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990685 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 18:32:23.992545 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990687 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 18:32:23.992545 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990690 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 18:32:23.993039 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990692 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 18:32:23.993039 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990695 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 18:32:23.993039 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990697 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 18:32:23.993039 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990699 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 18:32:23.993039 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990702 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 18:32:23.993039 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990705 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 18:32:23.993039 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990707 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 18:32:23.993039 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990709 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 18:32:23.993039 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990712 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 18:32:23.993039 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990714 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 18:32:23.993039 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990720 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 18:32:23.993039 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990724 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 18:32:23.993039 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990727 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 13 18:32:23.993039 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990730 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 18:32:23.993039 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990733 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 18:32:23.993039 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990735 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 18:32:23.993039 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990739 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 18:32:23.993039 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990743 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 18:32:23.993508 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990747 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 18:32:23.993508 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990750 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 18:32:23.993508 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990753 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 18:32:23.993508 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990756 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 18:32:23.993508 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990759 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 18:32:23.993508 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990762 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 18:32:23.993508 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990765 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 18:32:23.993508 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990768 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 18:32:23.993508 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990770 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 18:32:23.993508 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990773 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 18:32:23.993508 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990775 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 18:32:23.993508 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990778 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 18:32:23.993508 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990781 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 18:32:23.993508 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990784 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 18:32:23.993508 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990787 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 18:32:23.993508 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990789 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 18:32:23.993508 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990791 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 18:32:23.993508 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990794 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 18:32:23.993508 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990796 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 18:32:23.993508 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990799 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 18:32:23.994006 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990801 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 18:32:23.994006 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990804 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 18:32:23.994006 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990806 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 18:32:23.994006 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990809 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 18:32:23.994006 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990812 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 18:32:23.994006 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990815 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 18:32:23.994006 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990818 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 18:32:23.994006 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990820 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 18:32:23.994006 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990823 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 18:32:23.994006 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990825 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 18:32:23.994006 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990828 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 18:32:23.994006 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990830 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 18:32:23.994006 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990833 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 18:32:23.994006 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990835 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 18:32:23.994006 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990838 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 18:32:23.994006 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990840 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 18:32:23.994006 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990842 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 18:32:23.994006 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990845 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 18:32:23.994006 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.990847 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 18:32:23.994006 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991262 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 18:32:23.994507 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991268 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 18:32:23.994507 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991271 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 18:32:23.994507 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991274 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 18:32:23.994507 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991278 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 18:32:23.994507 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991281 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 18:32:23.994507 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991284 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 18:32:23.994507 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991287 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 18:32:23.994507 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991289 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 18:32:23.994507 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991292 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 18:32:23.994507 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991295 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 18:32:23.994507 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991299 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 18:32:23.994507 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991302 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 18:32:23.994507 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991305 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 18:32:23.994507 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991308 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 18:32:23.994507 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991310 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 18:32:23.994507 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991313 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 18:32:23.994507 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991316 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 18:32:23.994507 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991318 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 18:32:23.994507 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991321 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 18:32:23.994973 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991323 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 18:32:23.994973 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991326 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 18:32:23.994973 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991329 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 18:32:23.994973 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991331 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 18:32:23.994973 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991334 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 18:32:23.994973 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991336 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 18:32:23.994973 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991339 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 18:32:23.994973 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991341 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 18:32:23.994973 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991344 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 18:32:23.994973 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991346 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 18:32:23.994973 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991349 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 18:32:23.994973 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991351 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 18:32:23.994973 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991354 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 18:32:23.994973 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991356 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 18:32:23.994973 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991358 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 18:32:23.994973 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991361 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 18:32:23.994973 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991363 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 18:32:23.994973 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991366 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 18:32:23.994973 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991369 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 18:32:23.994973 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991372 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 13 18:32:23.995488 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991374 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 18:32:23.995488 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991377 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 18:32:23.995488 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991380 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 18:32:23.995488 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991382 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 18:32:23.995488 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991384 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 18:32:23.995488 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991387 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 18:32:23.995488 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991389 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 18:32:23.995488 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991391 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 18:32:23.995488 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991394 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 18:32:23.995488 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991400 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 18:32:23.995488 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991402 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 18:32:23.995488 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991405 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 18:32:23.995488 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991407 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 18:32:23.995488 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991409 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 18:32:23.995488 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991413 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 18:32:23.995488 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991416 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 18:32:23.995488 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991419 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 18:32:23.995488 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991422 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 18:32:23.995488 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991424 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 18:32:23.995993 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991428 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 18:32:23.995993 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991431 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 18:32:23.995993 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991434 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 18:32:23.995993 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991436 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 18:32:23.995993 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991439 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 18:32:23.995993 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991441 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 18:32:23.995993 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991443 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 18:32:23.995993 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991446 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 18:32:23.995993 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991449 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 18:32:23.995993 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991452 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 18:32:23.995993 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991455 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 18:32:23.995993 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991458 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 18:32:23.995993 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991460 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 18:32:23.995993 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991463 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 18:32:23.995993 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991465 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 18:32:23.995993 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991468 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 18:32:23.995993 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991470 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 18:32:23.995993 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991472 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 18:32:23.995993 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991475 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 18:32:23.996440 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991477 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 18:32:23.996440 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991480 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 18:32:23.996440 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991482 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 18:32:23.996440 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991487 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 18:32:23.996440 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991489 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 18:32:23.996440 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991492 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 18:32:23.996440 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991494 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 18:32:23.996440 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991497 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 18:32:23.996440 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991499 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 18:32:23.996440 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991502 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 18:32:23.996440 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991505 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 18:32:23.996440 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991508 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 18:32:23.996440 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991510 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 18:32:23.996440 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991513 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 18:32:23.996440 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991515 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 18:32:23.996440 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991518 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 18:32:23.996440 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991520 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 18:32:23.996440 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991522 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 18:32:23.996440 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991525 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 18:32:23.996440 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991527 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 18:32:23.996921 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.991530 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 18:32:23.996921 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991623 2582 flags.go:64] FLAG: --address="0.0.0.0" Jul 13 18:32:23.996921 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991646 2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 13 18:32:23.996921 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991653 2582 flags.go:64] FLAG: --anonymous-auth="true" Jul 13 18:32:23.996921 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991658 2582 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 13 18:32:23.996921 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991663 2582 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 13 18:32:23.996921 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991666 2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 13 18:32:23.996921 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991670 2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 13 18:32:23.996921 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991674 2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 13 18:32:23.996921 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991677 2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 13 18:32:23.996921 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991680 2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 13 18:32:23.996921 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991683 2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 13 18:32:23.996921 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991687 2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 13 18:32:23.996921 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991690 2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 13 18:32:23.996921 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991693 2582 flags.go:64] FLAG: --cgroup-root="" Jul 13 18:32:23.996921 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991697 2582 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 13 18:32:23.996921 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991700 2582 flags.go:64] FLAG: --client-ca-file="" Jul 13 18:32:23.996921 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991702 2582 flags.go:64] FLAG: --cloud-provider="external" Jul 13 18:32:23.996921 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991705 2582 flags.go:64] FLAG: --cluster-dns="[]" Jul 13 18:32:23.996921 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991709 2582 flags.go:64] FLAG: --cluster-domain="" Jul 13 18:32:23.996921 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991712 2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 13 18:32:23.996921 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991715 2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 13 18:32:23.996921 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991718 2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 13 18:32:23.996921 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991722 2582 flags.go:64] FLAG: --container-log-max-files="5" Jul 13 18:32:23.997515 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991725 2582 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 13 18:32:23.997515 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991728 2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 13 18:32:23.997515 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991731 2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 13 18:32:23.997515 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991734 2582 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 13 18:32:23.997515 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991737 2582 flags.go:64] FLAG: --contention-profiling="false" Jul 13 18:32:23.997515 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991740 2582 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 13 18:32:23.997515 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991743 2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 13 18:32:23.997515 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991748 2582 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 13 18:32:23.997515 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991750 2582 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 13 18:32:23.997515 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991755 2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 13 18:32:23.997515 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991757 2582 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 13 18:32:23.997515 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991760 2582 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 13 18:32:23.997515 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991763 2582 flags.go:64] FLAG: --enable-load-reader="false" Jul 13 18:32:23.997515 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991766 2582 flags.go:64] FLAG: --enable-server="true" Jul 13 18:32:23.997515 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991769 2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 13 18:32:23.997515 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991774 2582 flags.go:64] FLAG: --event-burst="100" Jul 13 18:32:23.997515 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991777 2582 flags.go:64] FLAG: --event-qps="50" Jul 13 18:32:23.997515 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991780 2582 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 13 18:32:23.997515 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991783 2582 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 13 18:32:23.997515 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991786 2582 flags.go:64] FLAG: --eviction-hard="" Jul 13 18:32:23.997515 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991790 2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 13 18:32:23.997515 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991793 2582 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 13 18:32:23.997515 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991796 2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 13 18:32:23.997515 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991800 2582 flags.go:64] FLAG: --eviction-soft="" Jul 13 18:32:23.997515 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991803 2582 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 13 18:32:23.998115 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991806 2582 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 13 18:32:23.998115 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991808 2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 13 18:32:23.998115 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991811 2582 flags.go:64] FLAG: --experimental-mounter-path="" Jul 13 18:32:23.998115 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991814 2582 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 13 18:32:23.998115 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991817 2582 flags.go:64] FLAG: --fail-swap-on="true" Jul 13 18:32:23.998115 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991819 2582 flags.go:64] FLAG: --feature-gates="" Jul 13 18:32:23.998115 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991824 2582 flags.go:64] FLAG: --file-check-frequency="20s" Jul 13 18:32:23.998115 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991827 2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 13 18:32:23.998115 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991830 2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 13 18:32:23.998115 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991832 2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 13 18:32:23.998115 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991835 2582 flags.go:64] FLAG: --healthz-port="10248" Jul 13 18:32:23.998115 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991838 2582 flags.go:64] FLAG: --help="false" Jul 13 18:32:23.998115 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991841 2582 flags.go:64] FLAG: --hostname-override="ip-10-0-129-191.ec2.internal" Jul 13 18:32:23.998115 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991844 2582 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 13 18:32:23.998115 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991848 2582 flags.go:64] FLAG: --http-check-frequency="20s" Jul 13 18:32:23.998115 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991850 2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 13 18:32:23.998115 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991854 2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 13 18:32:23.998115 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991858 2582 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 13 18:32:23.998115 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991861 2582 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 13 18:32:23.998115 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991863 2582 flags.go:64] FLAG: --image-service-endpoint="" Jul 13 18:32:23.998115 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991866 2582 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 13 18:32:23.998115 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991869 2582 flags.go:64] FLAG: --kube-api-burst="100" Jul 13 18:32:23.998115 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991872 2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 13 18:32:23.998711 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991875 2582 flags.go:64] FLAG: --kube-api-qps="50" Jul 13 18:32:23.998711 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991878 2582 flags.go:64] FLAG: --kube-reserved="" Jul 13 18:32:23.998711 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991881 2582 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 13 18:32:23.998711 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991884 2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 13 18:32:23.998711 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991886 2582 flags.go:64] FLAG: --kubelet-cgroups="" Jul 13 18:32:23.998711 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991889 2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 13 18:32:23.998711 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991892 2582 flags.go:64] FLAG: --lock-file="" Jul 13 18:32:23.998711 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991895 2582 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 13 18:32:23.998711 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991898 2582 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 13 18:32:23.998711 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991901 2582 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 13 18:32:23.998711 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991907 2582 flags.go:64] FLAG: --log-json-split-stream="false" Jul 13 18:32:23.998711 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991910 2582 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 13 18:32:23.998711 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991912 2582 flags.go:64] FLAG: --log-text-split-stream="false" Jul 13 18:32:23.998711 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991915 2582 flags.go:64] FLAG: --logging-format="text" Jul 13 18:32:23.998711 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991918 2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 13 18:32:23.998711 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991921 2582 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 13 18:32:23.998711 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991924 2582 flags.go:64] FLAG: --manifest-url="" Jul 13 18:32:23.998711 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991926 2582 flags.go:64] FLAG: --manifest-url-header="" Jul 13 18:32:23.998711 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991931 2582 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 13 18:32:23.998711 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991934 2582 flags.go:64] FLAG: --max-open-files="1000000" Jul 13 18:32:23.998711 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991938 2582 flags.go:64] FLAG: --max-pods="110" Jul 13 18:32:23.998711 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991941 2582 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 13 18:32:23.998711 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991944 2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 13 18:32:23.998711 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991948 2582 flags.go:64] FLAG: --memory-manager-policy="None" Jul 13 18:32:23.998711 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991951 2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 13 18:32:23.999342 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991954 2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 13 18:32:23.999342 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991957 2582 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 13 18:32:23.999342 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991959 2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 13 18:32:23.999342 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991966 2582 flags.go:64] FLAG: --node-status-max-images="50" Jul 13 18:32:23.999342 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991969 2582 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 13 18:32:23.999342 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991972 2582 flags.go:64] FLAG: --oom-score-adj="-999" Jul 13 18:32:23.999342 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991975 2582 flags.go:64] FLAG: --pod-cidr="" Jul 13 18:32:23.999342 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991979 2582 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 13 18:32:23.999342 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991984 2582 flags.go:64] FLAG: --pod-manifest-path="" Jul 13 18:32:23.999342 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991987 2582 flags.go:64] FLAG: --pod-max-pids="-1" Jul 13 18:32:23.999342 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991990 2582 flags.go:64] FLAG: --pods-per-core="0" Jul 13 18:32:23.999342 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991993 2582 flags.go:64] FLAG: --port="10250" Jul 13 18:32:23.999342 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991996 2582 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 13 18:32:23.999342 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.991999 2582 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-022268c778bd108fd" Jul 13 18:32:23.999342 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992002 2582 flags.go:64] FLAG: --qos-reserved="" Jul 13 18:32:23.999342 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992006 2582 flags.go:64] FLAG: --read-only-port="10255" Jul 13 18:32:23.999342 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992009 2582 flags.go:64] FLAG: --register-node="true" Jul 13 18:32:23.999342 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992012 2582 flags.go:64] FLAG: --register-with-taints="" Jul 13 18:32:23.999342 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992015 2582 flags.go:64] FLAG: --registry-burst="10" Jul 13 18:32:23.999342 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992018 2582 flags.go:64] FLAG: --registry-qps="5" Jul 13 18:32:23.999342 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992021 2582 flags.go:64] FLAG: --reserved-cpus="" Jul 13 18:32:23.999342 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992024 2582 flags.go:64] FLAG: --reserved-memory="" Jul 13 18:32:23.999342 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992027 2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 13 18:32:23.999342 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992030 2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 13 18:32:23.999342 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992033 2582 flags.go:64] FLAG: --rotate-certificates="false" Jul 13 18:32:24.000074 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992035 2582 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 13 18:32:24.000074 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992038 2582 flags.go:64] FLAG: --runonce="false" Jul 13 18:32:24.000074 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992041 2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 13 18:32:24.000074 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992044 2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 13 18:32:24.000074 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992047 2582 flags.go:64] FLAG: --seccomp-default="false" Jul 13 18:32:24.000074 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992051 2582 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 13 18:32:24.000074 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992054 2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 13 18:32:24.000074 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992057 2582 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 13 18:32:24.000074 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992060 2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 13 18:32:24.000074 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992063 2582 flags.go:64] FLAG: --storage-driver-password="root" Jul 13 18:32:24.000074 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992066 2582 flags.go:64] FLAG: --storage-driver-secure="false" Jul 13 18:32:24.000074 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992069 2582 flags.go:64] FLAG: --storage-driver-table="stats" Jul 13 18:32:24.000074 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992071 2582 flags.go:64] FLAG: --storage-driver-user="root" Jul 13 18:32:24.000074 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992074 2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 13 18:32:24.000074 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992077 2582 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 13 18:32:24.000074 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992081 2582 flags.go:64] FLAG: --system-cgroups="" Jul 13 18:32:24.000074 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992084 2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 13 18:32:24.000074 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992089 2582 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 13 18:32:24.000074 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992092 2582 flags.go:64] FLAG: --tls-cert-file="" Jul 13 18:32:24.000074 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992094 2582 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 13 18:32:24.000074 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992098 2582 flags.go:64] FLAG: --tls-min-version="" Jul 13 18:32:24.000074 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992101 2582 flags.go:64] FLAG: --tls-private-key-file="" Jul 13 18:32:24.000074 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992104 2582 flags.go:64] FLAG: --topology-manager-policy="none" Jul 13 18:32:24.000074 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992107 2582 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 13 18:32:24.000074 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992110 2582 flags.go:64] FLAG: --topology-manager-scope="container" Jul 13 18:32:24.001148 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992112 2582 flags.go:64] FLAG: --v="2" Jul 13 18:32:24.001148 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992116 2582 flags.go:64] FLAG: --version="false" Jul 13 18:32:24.001148 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992120 2582 flags.go:64] FLAG: --vmodule="" Jul 13 18:32:24.001148 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992124 2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 13 18:32:24.001148 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.992127 2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 13 18:32:24.001148 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992246 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 18:32:24.001148 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992250 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 18:32:24.001148 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992254 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 18:32:24.001148 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992256 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 18:32:24.001148 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992259 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 18:32:24.001148 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992262 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 18:32:24.001148 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992264 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 18:32:24.001148 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992268 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 18:32:24.001148 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992271 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 18:32:24.001148 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992273 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 18:32:24.001148 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992276 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 18:32:24.001148 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992279 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 18:32:24.001148 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992281 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 18:32:24.001148 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992283 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 18:32:24.001148 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992286 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 18:32:24.001148 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992288 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 18:32:24.002109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992291 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 18:32:24.002109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992293 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 18:32:24.002109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992296 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 18:32:24.002109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992299 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 18:32:24.002109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992301 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 18:32:24.002109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992303 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 18:32:24.002109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992306 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 18:32:24.002109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992308 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 18:32:24.002109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992311 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 13 18:32:24.002109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992314 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 18:32:24.002109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992316 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 18:32:24.002109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992319 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 18:32:24.002109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992321 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 18:32:24.002109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992324 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 18:32:24.002109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992326 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 18:32:24.002109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992329 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 18:32:24.002109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992331 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 18:32:24.002109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992334 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 18:32:24.002109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992336 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 18:32:24.002109 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992339 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 18:32:24.002887 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992341 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 18:32:24.002887 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992344 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 18:32:24.002887 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992346 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 18:32:24.002887 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992350 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 18:32:24.002887 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992353 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 18:32:24.002887 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992355 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 18:32:24.002887 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992357 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 18:32:24.002887 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992360 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 18:32:24.002887 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992364 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 18:32:24.002887 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992367 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 18:32:24.002887 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992370 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 18:32:24.002887 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992372 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 18:32:24.002887 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992375 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 18:32:24.002887 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992377 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 18:32:24.002887 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992380 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 18:32:24.002887 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992393 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 18:32:24.002887 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992396 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 18:32:24.002887 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992398 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 18:32:24.002887 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992401 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 18:32:24.003359 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992403 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 18:32:24.003359 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992407 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 18:32:24.003359 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992410 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 18:32:24.003359 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992413 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 18:32:24.003359 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992416 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 18:32:24.003359 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992419 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 18:32:24.003359 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992422 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 18:32:24.003359 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992425 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 18:32:24.003359 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992427 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 18:32:24.003359 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992430 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 18:32:24.003359 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992433 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 18:32:24.003359 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992436 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 18:32:24.003359 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992438 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 18:32:24.003359 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992441 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 18:32:24.003359 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992443 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 18:32:24.003359 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992446 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 18:32:24.003359 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992450 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 18:32:24.003359 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992452 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 18:32:24.003359 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992455 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 18:32:24.003359 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992457 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 18:32:24.004297 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992460 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 18:32:24.004297 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992462 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 18:32:24.004297 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992464 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 18:32:24.004297 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992467 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 18:32:24.004297 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992469 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 18:32:24.004297 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992472 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 18:32:24.004297 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992474 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 18:32:24.004297 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992477 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 18:32:24.004297 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992479 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 18:32:24.004297 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992486 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 18:32:24.004297 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992489 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 18:32:24.004297 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992491 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 18:32:24.004297 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992493 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 18:32:24.004297 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992496 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 18:32:24.004297 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992498 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 18:32:24.004297 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992500 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 18:32:24.004297 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992503 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 18:32:24.004297 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992505 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 18:32:24.004297 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992507 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 18:32:24.004297 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992510 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 18:32:24.005142 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992514 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 18:32:24.005142 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992517 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 18:32:24.005142 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992519 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 18:32:24.005142 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:23.992522 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 18:32:24.005142 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:23.993087 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 18:32:24.005142 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.002691 2582 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 13 18:32:24.005142 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.002718 2582 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 13 18:32:24.005142 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.003461 2582 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 13 18:32:24.005142 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.003475 2582 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 13 18:32:24.005142 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003591 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 18:32:24.005142 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003618 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 18:32:24.005142 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003624 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 18:32:24.005142 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003628 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 18:32:24.005142 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003632 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 18:32:24.005506 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003636 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 18:32:24.005506 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003640 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 18:32:24.005506 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003643 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 18:32:24.005506 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003647 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 18:32:24.005506 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003651 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 18:32:24.005506 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003655 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 18:32:24.005506 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003658 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 18:32:24.005506 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003662 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 18:32:24.005506 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003666 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 18:32:24.005506 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003669 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 18:32:24.005506 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003672 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 18:32:24.005506 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003676 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 18:32:24.005506 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003679 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 13 18:32:24.005506 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003684 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 18:32:24.005506 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003687 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 18:32:24.005506 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003691 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 18:32:24.005506 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003694 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 18:32:24.005506 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003698 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 18:32:24.005506 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003702 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 18:32:24.006275 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003705 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 18:32:24.006275 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003708 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 18:32:24.006275 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003712 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 18:32:24.006275 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003715 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 18:32:24.006275 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003719 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 18:32:24.006275 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003733 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 18:32:24.006275 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003737 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 18:32:24.006275 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003742 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 18:32:24.006275 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003746 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 18:32:24.006275 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003750 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 18:32:24.006275 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003754 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 18:32:24.006275 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003758 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 18:32:24.006275 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003762 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 18:32:24.006275 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003766 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 18:32:24.006275 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003770 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 18:32:24.006275 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003774 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 18:32:24.006275 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003778 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 18:32:24.006275 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003782 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 18:32:24.006275 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003789 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 18:32:24.007077 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003795 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 18:32:24.007077 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003800 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 18:32:24.007077 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003806 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 18:32:24.007077 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003812 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 18:32:24.007077 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003817 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 18:32:24.007077 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003821 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 18:32:24.007077 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003826 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 18:32:24.007077 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003831 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 18:32:24.007077 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003835 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 18:32:24.007077 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003841 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 18:32:24.007077 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003848 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 18:32:24.007077 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003852 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 18:32:24.007077 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003857 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 18:32:24.007077 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003861 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 18:32:24.007077 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003866 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 18:32:24.007077 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003871 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 18:32:24.007077 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003875 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 18:32:24.007077 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003880 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 18:32:24.007077 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003886 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 18:32:24.007620 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003898 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 18:32:24.007620 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003903 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 18:32:24.007620 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003907 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 18:32:24.007620 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003912 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 18:32:24.007620 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003916 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 18:32:24.007620 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003920 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 18:32:24.007620 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003924 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 18:32:24.007620 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003928 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 18:32:24.007620 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003932 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 18:32:24.007620 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003936 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 18:32:24.007620 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003941 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 18:32:24.007620 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003945 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 18:32:24.007620 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003949 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 18:32:24.007620 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003953 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 18:32:24.007620 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003957 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 18:32:24.007620 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003961 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 18:32:24.007620 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003965 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 18:32:24.007620 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003969 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 18:32:24.007620 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003973 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 18:32:24.007620 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003977 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 18:32:24.008229 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003981 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 18:32:24.008229 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003985 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 18:32:24.008229 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003990 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 18:32:24.008229 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003994 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 18:32:24.008229 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.003998 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 18:32:24.008229 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004002 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 18:32:24.008229 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004007 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 18:32:24.008229 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004010 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 18:32:24.008229 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004014 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 18:32:24.008229 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004018 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 18:32:24.008229 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004022 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 18:32:24.008229 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004027 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 18:32:24.008229 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004031 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 18:32:24.008229 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004043 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 18:32:24.008229 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004048 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 18:32:24.008229 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004052 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 18:32:24.008229 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004056 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 18:32:24.008687 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.004065 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 18:32:24.008687 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004295 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 18:32:24.008687 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004304 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 18:32:24.008687 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004309 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 18:32:24.008687 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004313 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 18:32:24.008687 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004317 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 18:32:24.008687 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004321 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 18:32:24.008687 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004326 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 18:32:24.008687 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004331 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 18:32:24.008687 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004335 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 18:32:24.008687 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004339 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 18:32:24.008687 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004343 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 18:32:24.008687 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004347 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 18:32:24.008687 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004351 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 18:32:24.008687 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004355 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 18:32:24.009059 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004359 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 18:32:24.009059 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004363 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 18:32:24.009059 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004367 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 18:32:24.009059 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004371 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 18:32:24.009059 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004375 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 18:32:24.009059 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004380 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 18:32:24.009059 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004384 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 18:32:24.009059 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004387 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 18:32:24.009059 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004392 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 18:32:24.009059 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004396 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 18:32:24.009059 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004400 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 18:32:24.009059 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004404 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 18:32:24.009059 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004408 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 18:32:24.009059 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004412 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 18:32:24.009059 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004426 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 18:32:24.009059 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004431 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 18:32:24.009059 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004435 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 18:32:24.009059 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004439 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 18:32:24.009059 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004443 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 18:32:24.009582 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004447 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 18:32:24.009582 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004451 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 18:32:24.009582 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004455 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 18:32:24.009582 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004460 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 18:32:24.009582 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004464 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 18:32:24.009582 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004468 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 18:32:24.009582 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004472 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 18:32:24.009582 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004478 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 18:32:24.009582 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004484 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 18:32:24.009582 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004489 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 18:32:24.009582 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004494 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 18:32:24.009582 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004499 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 18:32:24.009582 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004503 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 18:32:24.009582 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004507 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 18:32:24.009582 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004511 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 18:32:24.009582 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004515 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 18:32:24.009582 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004519 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 18:32:24.009582 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004523 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 18:32:24.009582 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004527 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 18:32:24.009582 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004531 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 18:32:24.010142 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004535 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 18:32:24.010142 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004539 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 18:32:24.010142 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004543 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 18:32:24.010142 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004547 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 18:32:24.010142 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004553 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 18:32:24.010142 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004558 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 18:32:24.010142 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004562 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 18:32:24.010142 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004566 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 18:32:24.010142 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004578 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 18:32:24.010142 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004583 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 18:32:24.010142 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004586 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 18:32:24.010142 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004591 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 18:32:24.010142 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004594 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 18:32:24.010142 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004620 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 18:32:24.010142 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004625 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 18:32:24.010142 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004629 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 18:32:24.010142 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004633 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 18:32:24.010142 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004638 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 13 18:32:24.010142 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004642 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 18:32:24.010721 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004646 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 18:32:24.010721 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004650 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 18:32:24.010721 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004654 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 18:32:24.010721 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004658 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 18:32:24.010721 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004662 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 18:32:24.010721 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004666 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 18:32:24.010721 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004670 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 18:32:24.010721 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004674 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 18:32:24.010721 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004678 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 18:32:24.010721 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004682 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 18:32:24.010721 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004686 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 18:32:24.010721 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004690 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 18:32:24.010721 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004694 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 18:32:24.010721 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004697 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 18:32:24.010721 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004702 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 18:32:24.010721 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004706 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 18:32:24.010721 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004710 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 18:32:24.010721 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004714 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 18:32:24.010721 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004718 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 18:32:24.010721 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004723 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 18:32:24.011430 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004727 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 18:32:24.011430 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004731 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 18:32:24.011430 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004743 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 18:32:24.011430 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004748 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 18:32:24.011430 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004752 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 18:32:24.011430 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004758 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 18:32:24.011430 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:24.004763 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 18:32:24.011430 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.004771 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 18:32:24.011430 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.004972 2582 server.go:962] "Client rotation is on, will bootstrap in background" Jul 13 18:32:24.011430 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.007763 2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 13 18:32:24.011430 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.009066 2582 server.go:1019] "Starting client certificate rotation" Jul 13 18:32:24.011430 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.009170 2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 13 18:32:24.011430 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.009218 2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 13 18:32:24.014414 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.014386 2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 13 18:32:24.018024 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.018007 2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 13 18:32:24.029064 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.029043 2582 log.go:25] "Validated CRI v1 runtime API" Jul 13 18:32:24.031324 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.031308 2582 log.go:25] "Validated CRI v1 image API" Jul 13 18:32:24.032138 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.032124 2582 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 13 18:32:24.034435 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.034409 2582 fs.go:135] Filesystem UUIDs: map[04f8effa-96e0-4f6a-8cc9-34f3a2b405a7:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 9c30f357-8645-4efb-b2a8-ee858f4c78d7:/dev/nvme0n1p4] Jul 13 18:32:24.034484 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.034436 2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 13 18:32:24.039693 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.039569 2582 manager.go:216] Machine: {Timestamp:2026-07-13 18:32:24.039092918 +0000 UTC m=+0.322521522 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101293 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f1b758545ca87526f942b6af9ccd4 SystemUUID:ec2f1b75-8545-ca87-526f-942b6af9ccd4 BootID:11a560fb-d882-475f-b985-a81fca10b447 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:52:7d:d7:c3:f5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:52:7d:d7:c3:f5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2a:d3:6c:d1:9c:88 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 13 18:32:24.039693 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.039688 2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 13 18:32:24.039802 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.039770 2582 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 13 18:32:24.043088 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.043049 2582 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 13 18:32:24.043249 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.043091 2582 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-191.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 13 18:32:24.043458 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.043443 2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 13 18:32:24.043790 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.043778 2582 topology_manager.go:138] "Creating topology manager with none policy" Jul 13 18:32:24.043823 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.043793 2582 container_manager_linux.go:309] "Creating device plugin manager" Jul 13 18:32:24.043823 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.043800 2582 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 13 18:32:24.044475 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.044464 2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 13 18:32:24.045692 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.045682 2582 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 13 18:32:24.049502 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.049490 2582 state_mem.go:36] "Initialized new in-memory state store" Jul 13 18:32:24.049645 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.049635 2582 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 13 18:32:24.049677 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.049670 2582 kubelet.go:486] "Attempting to sync node with API server" Jul 13 18:32:24.049703 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.049681 2582 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 13 18:32:24.049703 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.049692 2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 13 18:32:24.049763 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.049714 2582 kubelet.go:398] "Adding apiserver pod source" Jul 13 18:32:24.049795 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.049765 2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 13 18:32:24.050268 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.050249 2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 13 18:32:24.050321 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.050286 2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 13 18:32:24.053836 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.053761 2582 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 13 18:32:24.055775 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.055760 2582 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 13 18:32:24.055841 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.055792 2582 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 13 18:32:24.056438 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.056426 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 13 18:32:24.056476 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.056448 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 13 18:32:24.056476 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.056454 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 13 18:32:24.056476 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.056459 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 13 18:32:24.056476 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.056467 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 13 18:32:24.056476 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.056472 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 13 18:32:24.056621 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.056480 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 13 18:32:24.056621 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.056485 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 13 18:32:24.056621 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.056492 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 13 18:32:24.056621 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.056497 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 13 18:32:24.056621 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.056506 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 13 18:32:24.056621 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.056511 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 13 18:32:24.057960 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.057950 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 13 18:32:24.057996 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.057960 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 13 18:32:24.061245 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.061226 2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-191.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 13 18:32:24.061374 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.061362 2582 server.go:1268] "Started kubelet" Jul 13 18:32:24.061466 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.061439 2582 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 13 18:32:24.061700 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.061663 2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 13 18:32:24.061789 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.061731 2582 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 13 18:32:24.061789 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:24.061725 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-191.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 13 18:32:24.062132 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:24.062091 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 13 18:32:24.062451 ip-10-0-129-191 systemd[1]: Started Kubernetes Kubelet. Jul 13 18:32:24.062851 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.062832 2582 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 13 18:32:24.064038 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.064023 2582 server.go:310] "Adding debug handlers to kubelet server" Jul 13 18:32:24.067066 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:24.067029 2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 13 18:32:24.067809 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.067789 2582 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 13 18:32:24.067809 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.067797 2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 13 18:32:24.067961 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:24.067896 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-191.ec2.internal\" not found" Jul 13 18:32:24.068399 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.068383 2582 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 13 18:32:24.068399 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.068400 2582 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 13 18:32:24.068530 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.068477 2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 13 18:32:24.068530 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.068511 2582 reconstruct.go:97] "Volume reconstruction finished" Jul 13 18:32:24.068530 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.068532 2582 reconciler.go:29] "Reconciler: start to sync state" Jul 13 18:32:24.068683 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:24.067671 2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-191.ec2.internal.18c1ed078a9a0cd8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-191.ec2.internal,UID:ip-10-0-129-191.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-191.ec2.internal,},FirstTimestamp:2026-07-13 18:32:24.061332696 +0000 UTC m=+0.344761289,LastTimestamp:2026-07-13 18:32:24.061332696 +0000 UTC m=+0.344761289,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-191.ec2.internal,}" Jul 13 18:32:24.069696 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.069680 2582 factory.go:55] Registering systemd factory Jul 13 18:32:24.069696 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.069697 2582 factory.go:223] Registration of the systemd container factory successfully Jul 13 18:32:24.069922 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.069907 2582 factory.go:153] Registering CRI-O factory Jul 13 18:32:24.069922 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.069923 2582 factory.go:223] Registration of the crio container factory successfully Jul 13 18:32:24.070023 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.069970 2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 13 18:32:24.070023 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.069994 2582 factory.go:103] Registering Raw factory Jul 13 18:32:24.070023 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.070011 2582 manager.go:1196] Started watching for new ooms in manager Jul 13 18:32:24.070664 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.070651 2582 manager.go:349] Starting recovery of all containers Jul 13 18:32:24.076950 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:24.076921 2582 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-191.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 13 18:32:24.078894 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:24.078723 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 13 18:32:24.083618 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.082983 2582 manager.go:354] Recovery completed Jul 13 18:32:24.088966 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.088951 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 18:32:24.090920 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.090904 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-191.ec2.internal" event="NodeHasSufficientMemory" Jul 13 18:32:24.090966 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.090938 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-191.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 18:32:24.090966 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.090952 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-191.ec2.internal" event="NodeHasSufficientPID" Jul 13 18:32:24.091517 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.091505 2582 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 13 18:32:24.091553 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.091517 2582 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 13 18:32:24.091553 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.091532 2582 state_mem.go:36] "Initialized new in-memory state store" Jul 13 18:32:24.091630 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.091568 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 18:32:24.092203 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.092187 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-191.ec2.internal" event="NodeHasSufficientMemory" Jul 13 18:32:24.092287 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.092214 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-191.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 18:32:24.092287 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.092224 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-191.ec2.internal" event="NodeHasSufficientPID" Jul 13 18:32:24.093659 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:24.093555 2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-191.ec2.internal.18c1ed078c5d8cb6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-191.ec2.internal,UID:ip-10-0-129-191.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-191.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-191.ec2.internal,},FirstTimestamp:2026-07-13 18:32:24.090922166 +0000 UTC m=+0.374350759,LastTimestamp:2026-07-13 18:32:24.090922166 +0000 UTC m=+0.374350759,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-191.ec2.internal,}" Jul 13 18:32:24.093784 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.093772 2582 policy_none.go:49] "None policy: Start" Jul 13 18:32:24.093838 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.093792 2582 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 13 18:32:24.093838 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.093806 2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 13 18:32:24.095817 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.095804 2582 policy_none.go:47] "Start" Jul 13 18:32:24.104668 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:24.104582 2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-191.ec2.internal.18c1ed078c5de874 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-191.ec2.internal,UID:ip-10-0-129-191.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-191.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-191.ec2.internal,},FirstTimestamp:2026-07-13 18:32:24.090945652 +0000 UTC m=+0.374374245,LastTimestamp:2026-07-13 18:32:24.090945652 +0000 UTC m=+0.374374245,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-191.ec2.internal,}" Jul 13 18:32:24.113651 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:24.113560 2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-191.ec2.internal.18c1ed078c5e1058 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-191.ec2.internal,UID:ip-10-0-129-191.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-129-191.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-129-191.ec2.internal,},FirstTimestamp:2026-07-13 18:32:24.090955864 +0000 UTC m=+0.374384457,LastTimestamp:2026-07-13 18:32:24.090955864 +0000 UTC m=+0.374384457,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-191.ec2.internal,}" Jul 13 18:32:24.122671 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:24.122561 2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-191.ec2.internal.18c1ed078c5d8cb6\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-191.ec2.internal.18c1ed078c5d8cb6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-191.ec2.internal,UID:ip-10-0-129-191.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-191.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-191.ec2.internal,},FirstTimestamp:2026-07-13 18:32:24.090922166 +0000 UTC m=+0.374350759,LastTimestamp:2026-07-13 18:32:24.092201057 +0000 UTC m=+0.375629650,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-191.ec2.internal,}" Jul 13 18:32:24.133824 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:24.133753 2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-191.ec2.internal.18c1ed078c5de874\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-191.ec2.internal.18c1ed078c5de874 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-191.ec2.internal,UID:ip-10-0-129-191.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-191.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-191.ec2.internal,},FirstTimestamp:2026-07-13 18:32:24.090945652 +0000 UTC m=+0.374374245,LastTimestamp:2026-07-13 18:32:24.092219363 +0000 UTC m=+0.375647955,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-191.ec2.internal,}" Jul 13 18:32:24.139269 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.139257 2582 manager.go:337] "Starting Device Plugin manager" Jul 13 18:32:24.139340 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:24.139291 2582 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 13 18:32:24.139340 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.139300 2582 server.go:87] "Starting device plugin registration server" Jul 13 18:32:24.139635 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.139623 2582 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 13 18:32:24.139700 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.139637 2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 13 18:32:24.139779 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.139762 2582 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 13 18:32:24.139879 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.139867 2582 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 13 18:32:24.139929 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.139880 2582 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 13 18:32:24.141046 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:24.141022 2582 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 13 18:32:24.141099 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:24.141082 2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-191.ec2.internal\" not found" Jul 13 18:32:24.144893 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:24.144833 2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-191.ec2.internal.18c1ed078c5e1058\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-191.ec2.internal.18c1ed078c5e1058 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-191.ec2.internal,UID:ip-10-0-129-191.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-129-191.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-129-191.ec2.internal,},FirstTimestamp:2026-07-13 18:32:24.090955864 +0000 UTC m=+0.374384457,LastTimestamp:2026-07-13 18:32:24.092228441 +0000 UTC m=+0.375657034,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-191.ec2.internal,}" Jul 13 18:32:24.157109 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:24.157050 2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-191.ec2.internal.18c1ed078f634967 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-191.ec2.internal,UID:ip-10-0-129-191.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-129-191.ec2.internal,},FirstTimestamp:2026-07-13 18:32:24.141629799 +0000 UTC m=+0.425058379,LastTimestamp:2026-07-13 18:32:24.141629799 +0000 UTC m=+0.425058379,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-191.ec2.internal,}" Jul 13 18:32:24.205880 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.205854 2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tlzkz" Jul 13 18:32:24.213364 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.213342 2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tlzkz" Jul 13 18:32:24.240555 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.240491 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 18:32:24.241505 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.241487 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-191.ec2.internal" event="NodeHasSufficientMemory" Jul 13 18:32:24.241626 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.241526 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-191.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 18:32:24.241626 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.241540 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-191.ec2.internal" event="NodeHasSufficientPID" Jul 13 18:32:24.241736 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.241630 2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-191.ec2.internal" Jul 13 18:32:24.254497 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.254473 2582 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-191.ec2.internal" Jul 13 18:32:24.254497 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:24.254496 2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-191.ec2.internal\": node \"ip-10-0-129-191.ec2.internal\" not found" Jul 13 18:32:24.270756 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:24.270729 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-191.ec2.internal\" not found" Jul 13 18:32:24.272505 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.272482 2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 13 18:32:24.273864 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.273841 2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 13 18:32:24.273942 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.273886 2582 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 13 18:32:24.274327 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.274308 2582 kubelet.go:2448] "Starting kubelet main sync loop" Jul 13 18:32:24.274442 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:24.274349 2582 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 13 18:32:24.276436 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.276413 2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 18:32:24.371559 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:24.371527 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-191.ec2.internal\" not found" Jul 13 18:32:24.374671 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.374641 2582 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-191.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-191.ec2.internal"] Jul 13 18:32:24.374797 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.374718 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 18:32:24.375685 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.375670 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-191.ec2.internal" event="NodeHasSufficientMemory" Jul 13 18:32:24.375756 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.375717 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-191.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 18:32:24.375756 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.375729 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-191.ec2.internal" event="NodeHasSufficientPID" Jul 13 18:32:24.377911 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.377899 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 18:32:24.378037 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.378022 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-191.ec2.internal" Jul 13 18:32:24.378079 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.378053 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 18:32:24.378657 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.378640 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-191.ec2.internal" event="NodeHasSufficientMemory" Jul 13 18:32:24.378758 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.378668 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-191.ec2.internal" event="NodeHasSufficientMemory" Jul 13 18:32:24.378758 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.378685 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-191.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 18:32:24.378758 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.378692 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-191.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 18:32:24.378758 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.378696 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-191.ec2.internal" event="NodeHasSufficientPID" Jul 13 18:32:24.378758 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.378703 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-191.ec2.internal" event="NodeHasSufficientPID" Jul 13 18:32:24.380899 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.380880 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-191.ec2.internal" Jul 13 18:32:24.380985 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.380908 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 18:32:24.381615 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.381583 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-191.ec2.internal" event="NodeHasSufficientMemory" Jul 13 18:32:24.381676 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.381637 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-191.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 18:32:24.381676 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.381653 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-191.ec2.internal" event="NodeHasSufficientPID" Jul 13 18:32:24.408422 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:24.408398 2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-191.ec2.internal\" not found" node="ip-10-0-129-191.ec2.internal" Jul 13 18:32:24.413083 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:24.413064 2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-191.ec2.internal\" not found" node="ip-10-0-129-191.ec2.internal" Jul 13 18:32:24.471528 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.471496 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ac6f50c32a4e57fee7b6cab885240e5e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-191.ec2.internal\" (UID: \"ac6f50c32a4e57fee7b6cab885240e5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-191.ec2.internal" Jul 13 18:32:24.471528 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.471527 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac6f50c32a4e57fee7b6cab885240e5e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-191.ec2.internal\" (UID: \"ac6f50c32a4e57fee7b6cab885240e5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-191.ec2.internal" Jul 13 18:32:24.471754 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.471556 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/717de0a249da5500e718fed7ba33a501-config\") pod \"kube-apiserver-proxy-ip-10-0-129-191.ec2.internal\" (UID: \"717de0a249da5500e718fed7ba33a501\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-191.ec2.internal" Jul 13 18:32:24.471754 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:24.471622 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-191.ec2.internal\" not found" Jul 13 18:32:24.572569 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:24.572501 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-191.ec2.internal\" not found" Jul 13 18:32:24.572569 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.572563 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ac6f50c32a4e57fee7b6cab885240e5e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-191.ec2.internal\" (UID: \"ac6f50c32a4e57fee7b6cab885240e5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-191.ec2.internal" Jul 13 18:32:24.572711 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.572588 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac6f50c32a4e57fee7b6cab885240e5e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-191.ec2.internal\" (UID: \"ac6f50c32a4e57fee7b6cab885240e5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-191.ec2.internal" Jul 13 18:32:24.572711 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.572626 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/717de0a249da5500e718fed7ba33a501-config\") pod \"kube-apiserver-proxy-ip-10-0-129-191.ec2.internal\" (UID: \"717de0a249da5500e718fed7ba33a501\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-191.ec2.internal" Jul 13 18:32:24.572711 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.572654 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac6f50c32a4e57fee7b6cab885240e5e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-191.ec2.internal\" (UID: \"ac6f50c32a4e57fee7b6cab885240e5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-191.ec2.internal" Jul 13 18:32:24.572711 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.572655 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ac6f50c32a4e57fee7b6cab885240e5e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-191.ec2.internal\" (UID: \"ac6f50c32a4e57fee7b6cab885240e5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-191.ec2.internal" Jul 13 18:32:24.572711 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.572684 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/717de0a249da5500e718fed7ba33a501-config\") pod \"kube-apiserver-proxy-ip-10-0-129-191.ec2.internal\" (UID: \"717de0a249da5500e718fed7ba33a501\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-191.ec2.internal" Jul 13 18:32:24.672945 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:24.672913 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-191.ec2.internal\" not found" Jul 13 18:32:24.710135 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.710100 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-191.ec2.internal" Jul 13 18:32:24.715845 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.715831 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-191.ec2.internal" Jul 13 18:32:24.773460 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:24.773425 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-191.ec2.internal\" not found" Jul 13 18:32:24.873868 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:24.873788 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-191.ec2.internal\" not found" Jul 13 18:32:24.974225 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:24.974191 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-191.ec2.internal\" not found" Jul 13 18:32:24.978551 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.978534 2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 18:32:24.979673 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:24.979658 2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 18:32:25.008562 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.008535 2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 13 18:32:25.009045 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.008698 2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 13 18:32:25.009045 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.008735 2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 13 18:32:25.009045 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.008735 2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 13 18:32:25.050892 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.050860 2582 apiserver.go:52] "Watching apiserver" Jul 13 18:32:25.068351 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.068323 2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 13 18:32:25.068351 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.068340 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-191.ec2.internal" Jul 13 18:32:25.068351 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.068349 2582 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 13 18:32:25.069532 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.069508 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-r24p4","openshift-network-operator/iptables-alerter-v6knc","openshift-ovn-kubernetes/ovnkube-node-8l7w6","kube-system/global-pull-secret-syncer-pg9xz","kube-system/konnectivity-agent-7x978","openshift-image-registry/node-ca-5qgnw","openshift-multus/multus-additional-cni-plugins-f7lnn","openshift-multus/network-metrics-daemon-hvwvw","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4p7qj","openshift-cluster-node-tuning-operator/tuned-v4m5p","openshift-dns/node-resolver-nt9r9","openshift-multus/multus-w59t6"] Jul 13 18:32:25.072833 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.072801 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:32:25.072942 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:25.072883 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r24p4" podUID="241bb069-48c9-418e-9ef8-e2d8ffae0114" Jul 13 18:32:25.074592 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.074574 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnwrl\" (UniqueName: \"kubernetes.io/projected/241bb069-48c9-418e-9ef8-e2d8ffae0114-kube-api-access-tnwrl\") pod \"network-check-target-r24p4\" (UID: \"241bb069-48c9-418e-9ef8-e2d8ffae0114\") " pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:32:25.075014 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.074997 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-v6knc" Jul 13 18:32:25.077425 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.077404 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 13 18:32:25.077526 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.077421 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 13 18:32:25.077526 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.077450 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 13 18:32:25.077526 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.077518 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rcxzj\"" Jul 13 18:32:25.077765 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.077751 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.080278 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.080261 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:32:25.080351 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:25.080309 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pg9xz" podUID="cfcad9fb-77bc-460d-9ee0-fbfb5573ba07" Jul 13 18:32:25.080572 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.080556 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 13 18:32:25.080651 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.080564 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 13 18:32:25.081638 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.081614 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 13 18:32:25.081638 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.081634 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 13 18:32:25.081789 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.081664 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-scm8l\"" Jul 13 18:32:25.081789 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.081692 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 13 18:32:25.081789 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.081638 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 13 18:32:25.081789 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.081707 2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 13 18:32:25.082811 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.082794 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 13 18:32:25.082903 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.082859 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-191.ec2.internal" Jul 13 18:32:25.082903 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.082876 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7x978" Jul 13 18:32:25.085543 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.085524 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-f7lnn" Jul 13 18:32:25.085815 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.085709 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 13 18:32:25.085815 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.085782 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-fb4z4\"" Jul 13 18:32:25.086014 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.086001 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 13 18:32:25.088342 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.088322 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 13 18:32:25.088456 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.088440 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:32:25.088526 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.088512 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 13 18:32:25.088582 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:25.088520 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hvwvw" podUID="63f8372e-78ca-4580-8d1e-a57593c38dac" Jul 13 18:32:25.088582 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.088441 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 13 18:32:25.088582 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.088568 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 13 18:32:25.088751 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.088446 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 13 18:32:25.088751 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.088736 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-cw29c\"" Jul 13 18:32:25.090819 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.090802 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4p7qj" Jul 13 18:32:25.092808 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.092794 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 13 18:32:25.093119 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.093102 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 13 18:32:25.093254 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.093240 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 13 18:32:25.093390 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.093377 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 13 18:32:25.093500 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.093486 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.093909 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.093895 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-sb9x8\"" Jul 13 18:32:25.095880 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.095863 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 13 18:32:25.096008 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.095992 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 13 18:32:25.096067 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.096021 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9pvbg\"" Jul 13 18:32:25.096208 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.096193 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nt9r9" Jul 13 18:32:25.098347 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.098331 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 13 18:32:25.098787 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.098773 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 13 18:32:25.098787 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.098785 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-gcfsk\"" Jul 13 18:32:25.099129 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.099116 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.101390 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.101372 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 13 18:32:25.101486 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.101457 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-mkd2t\"" Jul 13 18:32:25.101588 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.101571 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-129-191.ec2.internal"] Jul 13 18:32:25.101660 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.101621 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-191.ec2.internal"] Jul 13 18:32:25.101727 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.101659 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5qgnw" Jul 13 18:32:25.103830 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.103808 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 13 18:32:25.103946 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.103810 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 13 18:32:25.103946 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.103919 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 13 18:32:25.104054 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.103974 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qdhnz\"" Jul 13 18:32:25.106908 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.106890 2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-mp67j" Jul 13 18:32:25.116362 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.116336 2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-mp67j" Jul 13 18:32:25.169160 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.169129 2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 13 18:32:25.175078 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.175051 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-os-release\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.175219 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.175084 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c402e9df-0605-412f-89d2-7aeaa8cb1277-cnibin\") pod \"multus-additional-cni-plugins-f7lnn\" (UID: \"c402e9df-0605-412f-89d2-7aeaa8cb1277\") " pod="openshift-multus/multus-additional-cni-plugins-f7lnn" Jul 13 18:32:25.175219 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.175104 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6xq6\" (UniqueName: \"kubernetes.io/projected/c402e9df-0605-412f-89d2-7aeaa8cb1277-kube-api-access-x6xq6\") pod \"multus-additional-cni-plugins-f7lnn\" (UID: \"c402e9df-0605-412f-89d2-7aeaa8cb1277\") " pod="openshift-multus/multus-additional-cni-plugins-f7lnn" Jul 13 18:32:25.175219 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.175129 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e85e7a89-c195-4f67-9843-c0f017468d83-etc-modprobe-d\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.175219 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.175167 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-host-kubelet\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.175219 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.175192 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-run-systemd\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.175219 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.175215 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b14ef695-c589-4fca-bd12-2e951f04e466-ovnkube-config\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.175451 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.175239 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b14ef695-c589-4fca-bd12-2e951f04e466-ovnkube-script-lib\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.175451 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.175265 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a25a1123-769c-46d1-90a7-ceaf4dc36537-agent-certs\") pod \"konnectivity-agent-7x978\" (UID: \"a25a1123-769c-46d1-90a7-ceaf4dc36537\") " pod="kube-system/konnectivity-agent-7x978" Jul 13 18:32:25.175451 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.175308 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e85e7a89-c195-4f67-9843-c0f017468d83-run\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.175451 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.175342 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-log-socket\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.175451 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.175373 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b14ef695-c589-4fca-bd12-2e951f04e466-env-overrides\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.175451 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.175413 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-host-run-netns\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.175451 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.175437 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f8804320-8ca2-4737-b2cc-b7e585541b30-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4p7qj\" (UID: \"f8804320-8ca2-4737-b2cc-b7e585541b30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4p7qj" Jul 13 18:32:25.175691 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.175452 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f8804320-8ca2-4737-b2cc-b7e585541b30-registration-dir\") pod \"aws-ebs-csi-driver-node-4p7qj\" (UID: \"f8804320-8ca2-4737-b2cc-b7e585541b30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4p7qj" Jul 13 18:32:25.175691 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.175471 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e85e7a89-c195-4f67-9843-c0f017468d83-etc-kubernetes\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.175691 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.175494 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5fee7e65-59ed-4225-b607-67c9a5218be4-serviceca\") pod \"node-ca-5qgnw\" (UID: \"5fee7e65-59ed-4225-b607-67c9a5218be4\") " pod="openshift-image-registry/node-ca-5qgnw" Jul 13 18:32:25.175691 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.175511 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36baa7e8-9898-491d-a961-c364c6e6af25-tmp-dir\") pod \"node-resolver-nt9r9\" (UID: \"36baa7e8-9898-491d-a961-c364c6e6af25\") " pod="openshift-dns/node-resolver-nt9r9" Jul 13 18:32:25.175691 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.175525 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/549c7399-8092-48f5-9596-048204fd4a82-host-slash\") pod \"iptables-alerter-v6knc\" (UID: \"549c7399-8092-48f5-9596-048204fd4a82\") " pod="openshift-network-operator/iptables-alerter-v6knc" Jul 13 18:32:25.175691 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.175542 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2qfm\" (UniqueName: \"kubernetes.io/projected/f8804320-8ca2-4737-b2cc-b7e585541b30-kube-api-access-b2qfm\") pod \"aws-ebs-csi-driver-node-4p7qj\" (UID: \"f8804320-8ca2-4737-b2cc-b7e585541b30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4p7qj" Jul 13 18:32:25.175691 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.175568 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-etc-openvswitch\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.175691 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.175596 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.175691 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.175642 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjtkb\" (UniqueName: \"kubernetes.io/projected/518044c4-7583-4f6b-b75c-710afb3ca571-kube-api-access-cjtkb\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.175691 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.175668 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/36baa7e8-9898-491d-a961-c364c6e6af25-hosts-file\") pod \"node-resolver-nt9r9\" (UID: \"36baa7e8-9898-491d-a961-c364c6e6af25\") " pod="openshift-dns/node-resolver-nt9r9" Jul 13 18:32:25.175691 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.175686 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-hostroot\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.176129 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.175728 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-multus-conf-dir\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.176129 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.175751 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c402e9df-0605-412f-89d2-7aeaa8cb1277-os-release\") pod \"multus-additional-cni-plugins-f7lnn\" (UID: \"c402e9df-0605-412f-89d2-7aeaa8cb1277\") " pod="openshift-multus/multus-additional-cni-plugins-f7lnn" Jul 13 18:32:25.176129 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.175769 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e85e7a89-c195-4f67-9843-c0f017468d83-lib-modules\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.176129 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.175795 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-run-ovn\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.176129 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.175814 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b14ef695-c589-4fca-bd12-2e951f04e466-ovn-node-metrics-cert\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.176129 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.175829 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63f8372e-78ca-4580-8d1e-a57593c38dac-metrics-certs\") pod \"network-metrics-daemon-hvwvw\" (UID: \"63f8372e-78ca-4580-8d1e-a57593c38dac\") " pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:32:25.176129 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.175855 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-host-var-lib-kubelet\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.176129 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.175878 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmzr7\" (UniqueName: \"kubernetes.io/projected/5fee7e65-59ed-4225-b607-67c9a5218be4-kube-api-access-qmzr7\") pod \"node-ca-5qgnw\" (UID: \"5fee7e65-59ed-4225-b607-67c9a5218be4\") " pod="openshift-image-registry/node-ca-5qgnw" Jul 13 18:32:25.176129 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.175894 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cfcad9fb-77bc-460d-9ee0-fbfb5573ba07-kubelet-config\") pod \"global-pull-secret-syncer-pg9xz\" (UID: \"cfcad9fb-77bc-460d-9ee0-fbfb5573ba07\") " pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:32:25.176129 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.175910 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-etc-kubernetes\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.176129 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.175925 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e85e7a89-c195-4f67-9843-c0f017468d83-etc-sysconfig\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.176129 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.175965 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-host-run-ovn-kubernetes\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.176129 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.176012 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnwrl\" (UniqueName: \"kubernetes.io/projected/241bb069-48c9-418e-9ef8-e2d8ffae0114-kube-api-access-tnwrl\") pod \"network-check-target-r24p4\" (UID: \"241bb069-48c9-418e-9ef8-e2d8ffae0114\") " pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:32:25.176129 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.176048 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c402e9df-0605-412f-89d2-7aeaa8cb1277-cni-binary-copy\") pod \"multus-additional-cni-plugins-f7lnn\" (UID: \"c402e9df-0605-412f-89d2-7aeaa8cb1277\") " pod="openshift-multus/multus-additional-cni-plugins-f7lnn" Jul 13 18:32:25.176129 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.176076 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jtkn\" (UniqueName: \"kubernetes.io/projected/e85e7a89-c195-4f67-9843-c0f017468d83-kube-api-access-5jtkn\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.176129 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.176099 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e85e7a89-c195-4f67-9843-c0f017468d83-sys\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.176949 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.176128 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-host-run-k8s-cni-cncf-io\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.176949 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.176153 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/518044c4-7583-4f6b-b75c-710afb3ca571-multus-daemon-config\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.176949 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.176178 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e85e7a89-c195-4f67-9843-c0f017468d83-etc-sysctl-conf\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.176949 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.176212 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5fee7e65-59ed-4225-b607-67c9a5218be4-host\") pod \"node-ca-5qgnw\" (UID: \"5fee7e65-59ed-4225-b607-67c9a5218be4\") " pod="openshift-image-registry/node-ca-5qgnw" Jul 13 18:32:25.176949 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.176264 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-var-lib-openvswitch\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.176949 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.176292 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-host-cni-netd\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.176949 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.176314 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-multus-cni-dir\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.176949 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.176338 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-multus-socket-dir-parent\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.176949 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.176363 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/549c7399-8092-48f5-9596-048204fd4a82-iptables-alerter-script\") pod \"iptables-alerter-v6knc\" (UID: \"549c7399-8092-48f5-9596-048204fd4a82\") " pod="openshift-network-operator/iptables-alerter-v6knc" Jul 13 18:32:25.176949 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.176392 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c402e9df-0605-412f-89d2-7aeaa8cb1277-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-f7lnn\" (UID: \"c402e9df-0605-412f-89d2-7aeaa8cb1277\") " pod="openshift-multus/multus-additional-cni-plugins-f7lnn" Jul 13 18:32:25.176949 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.176426 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f8804320-8ca2-4737-b2cc-b7e585541b30-sys-fs\") pod \"aws-ebs-csi-driver-node-4p7qj\" (UID: \"f8804320-8ca2-4737-b2cc-b7e585541b30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4p7qj" Jul 13 18:32:25.176949 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.176448 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e85e7a89-c195-4f67-9843-c0f017468d83-var-lib-kubelet\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.176949 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.176465 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-systemd-units\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.176949 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.176496 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-host-cni-bin\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.176949 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.176525 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vh5kw\" (UniqueName: \"kubernetes.io/projected/36baa7e8-9898-491d-a961-c364c6e6af25-kube-api-access-vh5kw\") pod \"node-resolver-nt9r9\" (UID: \"36baa7e8-9898-491d-a961-c364c6e6af25\") " pod="openshift-dns/node-resolver-nt9r9" Jul 13 18:32:25.176949 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.176549 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c402e9df-0605-412f-89d2-7aeaa8cb1277-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-f7lnn\" (UID: \"c402e9df-0605-412f-89d2-7aeaa8cb1277\") " pod="openshift-multus/multus-additional-cni-plugins-f7lnn" Jul 13 18:32:25.177726 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.176572 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e85e7a89-c195-4f67-9843-c0f017468d83-tmp\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.177726 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.176597 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g72r\" (UniqueName: \"kubernetes.io/projected/b14ef695-c589-4fca-bd12-2e951f04e466-kube-api-access-7g72r\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.177726 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.176844 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-cnibin\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.177726 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.176874 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-host-var-lib-cni-multus\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.177726 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.176902 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c402e9df-0605-412f-89d2-7aeaa8cb1277-tuning-conf-dir\") pod \"multus-additional-cni-plugins-f7lnn\" (UID: \"c402e9df-0605-412f-89d2-7aeaa8cb1277\") " pod="openshift-multus/multus-additional-cni-plugins-f7lnn" Jul 13 18:32:25.177726 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.176927 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-host-slash\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.177726 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.176951 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-host-run-netns\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.177726 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.176978 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-node-log\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.177726 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.177006 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cfcad9fb-77bc-460d-9ee0-fbfb5573ba07-dbus\") pod \"global-pull-secret-syncer-pg9xz\" (UID: \"cfcad9fb-77bc-460d-9ee0-fbfb5573ba07\") " pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:32:25.177726 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.177036 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-host-run-multus-certs\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.177726 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.177067 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jd2bt\" (UniqueName: \"kubernetes.io/projected/63f8372e-78ca-4580-8d1e-a57593c38dac-kube-api-access-jd2bt\") pod \"network-metrics-daemon-hvwvw\" (UID: \"63f8372e-78ca-4580-8d1e-a57593c38dac\") " pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:32:25.177726 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.177096 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvlm4\" (UniqueName: \"kubernetes.io/projected/549c7399-8092-48f5-9596-048204fd4a82-kube-api-access-qvlm4\") pod \"iptables-alerter-v6knc\" (UID: \"549c7399-8092-48f5-9596-048204fd4a82\") " pod="openshift-network-operator/iptables-alerter-v6knc" Jul 13 18:32:25.177726 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.177122 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c402e9df-0605-412f-89d2-7aeaa8cb1277-system-cni-dir\") pod \"multus-additional-cni-plugins-f7lnn\" (UID: \"c402e9df-0605-412f-89d2-7aeaa8cb1277\") " pod="openshift-multus/multus-additional-cni-plugins-f7lnn" Jul 13 18:32:25.177726 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.177156 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cfcad9fb-77bc-460d-9ee0-fbfb5573ba07-original-pull-secret\") pod \"global-pull-secret-syncer-pg9xz\" (UID: \"cfcad9fb-77bc-460d-9ee0-fbfb5573ba07\") " pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:32:25.177726 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.177183 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f8804320-8ca2-4737-b2cc-b7e585541b30-device-dir\") pod \"aws-ebs-csi-driver-node-4p7qj\" (UID: \"f8804320-8ca2-4737-b2cc-b7e585541b30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4p7qj" Jul 13 18:32:25.177726 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.177208 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e85e7a89-c195-4f67-9843-c0f017468d83-etc-sysctl-d\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.178243 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.177232 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e85e7a89-c195-4f67-9843-c0f017468d83-etc-systemd\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.178243 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.177253 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a25a1123-769c-46d1-90a7-ceaf4dc36537-konnectivity-ca\") pod \"konnectivity-agent-7x978\" (UID: \"a25a1123-769c-46d1-90a7-ceaf4dc36537\") " pod="kube-system/konnectivity-agent-7x978" Jul 13 18:32:25.178243 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.177271 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-system-cni-dir\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.178243 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.177297 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/518044c4-7583-4f6b-b75c-710afb3ca571-cni-binary-copy\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.178243 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.177320 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e85e7a89-c195-4f67-9843-c0f017468d83-host\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.178243 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.177368 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e85e7a89-c195-4f67-9843-c0f017468d83-etc-tuned\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.178243 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.177432 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-run-openvswitch\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.178243 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.177460 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-host-var-lib-cni-bin\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.178243 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.177485 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f8804320-8ca2-4737-b2cc-b7e585541b30-socket-dir\") pod \"aws-ebs-csi-driver-node-4p7qj\" (UID: \"f8804320-8ca2-4737-b2cc-b7e585541b30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4p7qj" Jul 13 18:32:25.178243 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.177512 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f8804320-8ca2-4737-b2cc-b7e585541b30-etc-selinux\") pod \"aws-ebs-csi-driver-node-4p7qj\" (UID: \"f8804320-8ca2-4737-b2cc-b7e585541b30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4p7qj" Jul 13 18:32:25.183321 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:25.183299 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 18:32:25.183456 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:25.183329 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 18:32:25.183456 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:25.183341 2582 projected.go:196] Error preparing data for projected volume kube-api-access-tnwrl for pod openshift-network-diagnostics/network-check-target-r24p4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:32:25.183456 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:25.183411 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/241bb069-48c9-418e-9ef8-e2d8ffae0114-kube-api-access-tnwrl podName:241bb069-48c9-418e-9ef8-e2d8ffae0114 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:25.683388475 +0000 UTC m=+1.966817075 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tnwrl" (UniqueName: "kubernetes.io/projected/241bb069-48c9-418e-9ef8-e2d8ffae0114-kube-api-access-tnwrl") pod "network-check-target-r24p4" (UID: "241bb069-48c9-418e-9ef8-e2d8ffae0114") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:32:25.217200 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.217139 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-12 18:27:24 +0000 UTC" deadline="2028-03-16 20:36:31.06610562 +0000 UTC" Jul 13 18:32:25.217343 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.217190 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14690h4m5.848919828s" Jul 13 18:32:25.249141 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:25.249106 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac6f50c32a4e57fee7b6cab885240e5e.slice/crio-8e3216f831389f277de94ee4070a4515aba7ce847de4fb782d3a75487d07cfaa WatchSource:0}: Error finding container 8e3216f831389f277de94ee4070a4515aba7ce847de4fb782d3a75487d07cfaa: Status 404 returned error can't find the container with id 8e3216f831389f277de94ee4070a4515aba7ce847de4fb782d3a75487d07cfaa Jul 13 18:32:25.251098 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:25.251069 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod717de0a249da5500e718fed7ba33a501.slice/crio-9ed224f26d3bbff78f4ac45db8256e3b82bf677bd0b3f805c20567560c0f0c9e WatchSource:0}: Error finding container 9ed224f26d3bbff78f4ac45db8256e3b82bf677bd0b3f805c20567560c0f0c9e: Status 404 returned error can't find the container with id 9ed224f26d3bbff78f4ac45db8256e3b82bf677bd0b3f805c20567560c0f0c9e Jul 13 18:32:25.254506 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.254489 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 18:32:25.277312 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.277267 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-191.ec2.internal" event={"ID":"717de0a249da5500e718fed7ba33a501","Type":"ContainerStarted","Data":"9ed224f26d3bbff78f4ac45db8256e3b82bf677bd0b3f805c20567560c0f0c9e"} Jul 13 18:32:25.277842 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.277803 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cfcad9fb-77bc-460d-9ee0-fbfb5573ba07-original-pull-secret\") pod \"global-pull-secret-syncer-pg9xz\" (UID: \"cfcad9fb-77bc-460d-9ee0-fbfb5573ba07\") " pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:32:25.277981 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.277855 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f8804320-8ca2-4737-b2cc-b7e585541b30-device-dir\") pod \"aws-ebs-csi-driver-node-4p7qj\" (UID: \"f8804320-8ca2-4737-b2cc-b7e585541b30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4p7qj" Jul 13 18:32:25.277981 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.277882 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e85e7a89-c195-4f67-9843-c0f017468d83-etc-sysctl-d\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.277981 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.277901 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e85e7a89-c195-4f67-9843-c0f017468d83-etc-systemd\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.277981 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.277925 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a25a1123-769c-46d1-90a7-ceaf4dc36537-konnectivity-ca\") pod \"konnectivity-agent-7x978\" (UID: \"a25a1123-769c-46d1-90a7-ceaf4dc36537\") " pod="kube-system/konnectivity-agent-7x978" Jul 13 18:32:25.277981 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.277962 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-system-cni-dir\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.277981 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:25.277964 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 18:32:25.278303 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:25.278040 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfcad9fb-77bc-460d-9ee0-fbfb5573ba07-original-pull-secret podName:cfcad9fb-77bc-460d-9ee0-fbfb5573ba07 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:25.778023207 +0000 UTC m=+2.061451808 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cfcad9fb-77bc-460d-9ee0-fbfb5573ba07-original-pull-secret") pod "global-pull-secret-syncer-pg9xz" (UID: "cfcad9fb-77bc-460d-9ee0-fbfb5573ba07") : object "kube-system"/"original-pull-secret" not registered Jul 13 18:32:25.278303 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.277966 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f8804320-8ca2-4737-b2cc-b7e585541b30-device-dir\") pod \"aws-ebs-csi-driver-node-4p7qj\" (UID: \"f8804320-8ca2-4737-b2cc-b7e585541b30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4p7qj" Jul 13 18:32:25.278303 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.278032 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e85e7a89-c195-4f67-9843-c0f017468d83-etc-systemd\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.278303 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.278069 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e85e7a89-c195-4f67-9843-c0f017468d83-etc-sysctl-d\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.278303 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.278122 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/518044c4-7583-4f6b-b75c-710afb3ca571-cni-binary-copy\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.278303 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.278155 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e85e7a89-c195-4f67-9843-c0f017468d83-host\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.278303 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.278177 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-system-cni-dir\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.278303 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.278183 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e85e7a89-c195-4f67-9843-c0f017468d83-etc-tuned\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.278303 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.278238 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-run-openvswitch\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.278303 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.278269 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-host-var-lib-cni-bin\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.278303 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.278281 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e85e7a89-c195-4f67-9843-c0f017468d83-host\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.278303 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.278302 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f8804320-8ca2-4737-b2cc-b7e585541b30-socket-dir\") pod \"aws-ebs-csi-driver-node-4p7qj\" (UID: \"f8804320-8ca2-4737-b2cc-b7e585541b30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4p7qj" Jul 13 18:32:25.278303 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.278327 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f8804320-8ca2-4737-b2cc-b7e585541b30-etc-selinux\") pod \"aws-ebs-csi-driver-node-4p7qj\" (UID: \"f8804320-8ca2-4737-b2cc-b7e585541b30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4p7qj" Jul 13 18:32:25.278303 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.278320 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-run-openvswitch\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.278303 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.278356 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-os-release\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.279129 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.278392 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c402e9df-0605-412f-89d2-7aeaa8cb1277-cnibin\") pod \"multus-additional-cni-plugins-f7lnn\" (UID: \"c402e9df-0605-412f-89d2-7aeaa8cb1277\") " pod="openshift-multus/multus-additional-cni-plugins-f7lnn" Jul 13 18:32:25.279129 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.278420 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-os-release\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.279129 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.278424 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6xq6\" (UniqueName: \"kubernetes.io/projected/c402e9df-0605-412f-89d2-7aeaa8cb1277-kube-api-access-x6xq6\") pod \"multus-additional-cni-plugins-f7lnn\" (UID: \"c402e9df-0605-412f-89d2-7aeaa8cb1277\") " pod="openshift-multus/multus-additional-cni-plugins-f7lnn" Jul 13 18:32:25.279129 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.278429 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-191.ec2.internal" event={"ID":"ac6f50c32a4e57fee7b6cab885240e5e","Type":"ContainerStarted","Data":"8e3216f831389f277de94ee4070a4515aba7ce847de4fb782d3a75487d07cfaa"} Jul 13 18:32:25.279129 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.278457 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e85e7a89-c195-4f67-9843-c0f017468d83-etc-modprobe-d\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.279129 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.278466 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c402e9df-0605-412f-89d2-7aeaa8cb1277-cnibin\") pod \"multus-additional-cni-plugins-f7lnn\" (UID: \"c402e9df-0605-412f-89d2-7aeaa8cb1277\") " pod="openshift-multus/multus-additional-cni-plugins-f7lnn" Jul 13 18:32:25.279129 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.278468 2582 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 13 18:32:25.279129 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.278487 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-host-kubelet\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.279129 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.278526 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-host-kubelet\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.279129 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.278534 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-run-systemd\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.279129 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.278566 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-run-systemd\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.279129 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.278327 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-host-var-lib-cni-bin\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.279129 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.278614 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a25a1123-769c-46d1-90a7-ceaf4dc36537-konnectivity-ca\") pod \"konnectivity-agent-7x978\" (UID: \"a25a1123-769c-46d1-90a7-ceaf4dc36537\") " pod="kube-system/konnectivity-agent-7x978" Jul 13 18:32:25.279129 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.278670 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f8804320-8ca2-4737-b2cc-b7e585541b30-socket-dir\") pod \"aws-ebs-csi-driver-node-4p7qj\" (UID: \"f8804320-8ca2-4737-b2cc-b7e585541b30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4p7qj" Jul 13 18:32:25.279129 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.278691 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f8804320-8ca2-4737-b2cc-b7e585541b30-etc-selinux\") pod \"aws-ebs-csi-driver-node-4p7qj\" (UID: \"f8804320-8ca2-4737-b2cc-b7e585541b30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4p7qj" Jul 13 18:32:25.279129 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.278701 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b14ef695-c589-4fca-bd12-2e951f04e466-ovnkube-config\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.279129 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.278765 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b14ef695-c589-4fca-bd12-2e951f04e466-ovnkube-script-lib\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.279129 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.278792 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a25a1123-769c-46d1-90a7-ceaf4dc36537-agent-certs\") pod \"konnectivity-agent-7x978\" (UID: \"a25a1123-769c-46d1-90a7-ceaf4dc36537\") " pod="kube-system/konnectivity-agent-7x978" Jul 13 18:32:25.279787 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.278826 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/518044c4-7583-4f6b-b75c-710afb3ca571-cni-binary-copy\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.279787 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.278909 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e85e7a89-c195-4f67-9843-c0f017468d83-run\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.279787 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279013 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e85e7a89-c195-4f67-9843-c0f017468d83-etc-modprobe-d\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.279787 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279047 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-log-socket\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.279787 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279064 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b14ef695-c589-4fca-bd12-2e951f04e466-env-overrides\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.279787 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279083 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-host-run-netns\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.279787 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279109 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f8804320-8ca2-4737-b2cc-b7e585541b30-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4p7qj\" (UID: \"f8804320-8ca2-4737-b2cc-b7e585541b30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4p7qj" Jul 13 18:32:25.279787 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279140 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f8804320-8ca2-4737-b2cc-b7e585541b30-registration-dir\") pod \"aws-ebs-csi-driver-node-4p7qj\" (UID: \"f8804320-8ca2-4737-b2cc-b7e585541b30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4p7qj" Jul 13 18:32:25.279787 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279173 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e85e7a89-c195-4f67-9843-c0f017468d83-etc-kubernetes\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.279787 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279197 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5fee7e65-59ed-4225-b607-67c9a5218be4-serviceca\") pod \"node-ca-5qgnw\" (UID: \"5fee7e65-59ed-4225-b607-67c9a5218be4\") " pod="openshift-image-registry/node-ca-5qgnw" Jul 13 18:32:25.279787 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279223 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36baa7e8-9898-491d-a961-c364c6e6af25-tmp-dir\") pod \"node-resolver-nt9r9\" (UID: \"36baa7e8-9898-491d-a961-c364c6e6af25\") " pod="openshift-dns/node-resolver-nt9r9" Jul 13 18:32:25.279787 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279249 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/549c7399-8092-48f5-9596-048204fd4a82-host-slash\") pod \"iptables-alerter-v6knc\" (UID: \"549c7399-8092-48f5-9596-048204fd4a82\") " pod="openshift-network-operator/iptables-alerter-v6knc" Jul 13 18:32:25.279787 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279276 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2qfm\" (UniqueName: \"kubernetes.io/projected/f8804320-8ca2-4737-b2cc-b7e585541b30-kube-api-access-b2qfm\") pod \"aws-ebs-csi-driver-node-4p7qj\" (UID: \"f8804320-8ca2-4737-b2cc-b7e585541b30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4p7qj" Jul 13 18:32:25.279787 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279313 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-etc-openvswitch\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.279787 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279342 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.279787 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279368 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjtkb\" (UniqueName: \"kubernetes.io/projected/518044c4-7583-4f6b-b75c-710afb3ca571-kube-api-access-cjtkb\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.279787 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279392 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/36baa7e8-9898-491d-a961-c364c6e6af25-hosts-file\") pod \"node-resolver-nt9r9\" (UID: \"36baa7e8-9898-491d-a961-c364c6e6af25\") " pod="openshift-dns/node-resolver-nt9r9" Jul 13 18:32:25.280642 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279406 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b14ef695-c589-4fca-bd12-2e951f04e466-ovnkube-script-lib\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.280642 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279416 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-hostroot\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.280642 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279440 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-multus-conf-dir\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.280642 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279472 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c402e9df-0605-412f-89d2-7aeaa8cb1277-os-release\") pod \"multus-additional-cni-plugins-f7lnn\" (UID: \"c402e9df-0605-412f-89d2-7aeaa8cb1277\") " pod="openshift-multus/multus-additional-cni-plugins-f7lnn" Jul 13 18:32:25.280642 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279497 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e85e7a89-c195-4f67-9843-c0f017468d83-lib-modules\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.280642 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279501 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b14ef695-c589-4fca-bd12-2e951f04e466-env-overrides\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.280642 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279521 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-run-ovn\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.280642 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279553 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b14ef695-c589-4fca-bd12-2e951f04e466-ovn-node-metrics-cert\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.280642 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279570 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f8804320-8ca2-4737-b2cc-b7e585541b30-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4p7qj\" (UID: \"f8804320-8ca2-4737-b2cc-b7e585541b30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4p7qj" Jul 13 18:32:25.280642 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279580 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63f8372e-78ca-4580-8d1e-a57593c38dac-metrics-certs\") pod \"network-metrics-daemon-hvwvw\" (UID: \"63f8372e-78ca-4580-8d1e-a57593c38dac\") " pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:32:25.280642 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279606 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-host-var-lib-kubelet\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.280642 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279617 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-host-run-netns\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.280642 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279633 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmzr7\" (UniqueName: \"kubernetes.io/projected/5fee7e65-59ed-4225-b607-67c9a5218be4-kube-api-access-qmzr7\") pod \"node-ca-5qgnw\" (UID: \"5fee7e65-59ed-4225-b607-67c9a5218be4\") " pod="openshift-image-registry/node-ca-5qgnw" Jul 13 18:32:25.280642 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279659 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cfcad9fb-77bc-460d-9ee0-fbfb5573ba07-kubelet-config\") pod \"global-pull-secret-syncer-pg9xz\" (UID: \"cfcad9fb-77bc-460d-9ee0-fbfb5573ba07\") " pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:32:25.280642 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279687 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-log-socket\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.280642 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279710 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-etc-kubernetes\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.280642 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279739 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e85e7a89-c195-4f67-9843-c0f017468d83-etc-sysconfig\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.280642 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279767 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-host-run-ovn-kubernetes\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.281471 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279804 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e85e7a89-c195-4f67-9843-c0f017468d83-etc-kubernetes\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.281471 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279835 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-host-run-ovn-kubernetes\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.281471 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279850 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f8804320-8ca2-4737-b2cc-b7e585541b30-registration-dir\") pod \"aws-ebs-csi-driver-node-4p7qj\" (UID: \"f8804320-8ca2-4737-b2cc-b7e585541b30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4p7qj" Jul 13 18:32:25.281471 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279888 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/549c7399-8092-48f5-9596-048204fd4a82-host-slash\") pod \"iptables-alerter-v6knc\" (UID: \"549c7399-8092-48f5-9596-048204fd4a82\") " pod="openshift-network-operator/iptables-alerter-v6knc" Jul 13 18:32:25.281471 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279882 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c402e9df-0605-412f-89d2-7aeaa8cb1277-cni-binary-copy\") pod \"multus-additional-cni-plugins-f7lnn\" (UID: \"c402e9df-0605-412f-89d2-7aeaa8cb1277\") " pod="openshift-multus/multus-additional-cni-plugins-f7lnn" Jul 13 18:32:25.281471 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279931 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jtkn\" (UniqueName: \"kubernetes.io/projected/e85e7a89-c195-4f67-9843-c0f017468d83-kube-api-access-5jtkn\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.281471 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.279894 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36baa7e8-9898-491d-a961-c364c6e6af25-tmp-dir\") pod \"node-resolver-nt9r9\" (UID: \"36baa7e8-9898-491d-a961-c364c6e6af25\") " pod="openshift-dns/node-resolver-nt9r9" Jul 13 18:32:25.281471 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.280008 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e85e7a89-c195-4f67-9843-c0f017468d83-etc-sysconfig\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.281471 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.280026 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c402e9df-0605-412f-89d2-7aeaa8cb1277-os-release\") pod \"multus-additional-cni-plugins-f7lnn\" (UID: \"c402e9df-0605-412f-89d2-7aeaa8cb1277\") " pod="openshift-multus/multus-additional-cni-plugins-f7lnn" Jul 13 18:32:25.281471 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.280045 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-multus-conf-dir\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.281471 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.280065 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e85e7a89-c195-4f67-9843-c0f017468d83-lib-modules\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.281471 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.280082 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-run-ovn\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.281471 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.280110 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-host-var-lib-kubelet\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.281471 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.280166 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b14ef695-c589-4fca-bd12-2e951f04e466-ovnkube-config\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.281471 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.280231 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e85e7a89-c195-4f67-9843-c0f017468d83-run\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.281471 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.280283 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5fee7e65-59ed-4225-b607-67c9a5218be4-serviceca\") pod \"node-ca-5qgnw\" (UID: \"5fee7e65-59ed-4225-b607-67c9a5218be4\") " pod="openshift-image-registry/node-ca-5qgnw" Jul 13 18:32:25.281471 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.280282 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/36baa7e8-9898-491d-a961-c364c6e6af25-hosts-file\") pod \"node-resolver-nt9r9\" (UID: \"36baa7e8-9898-491d-a961-c364c6e6af25\") " pod="openshift-dns/node-resolver-nt9r9" Jul 13 18:32:25.281471 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.280311 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cfcad9fb-77bc-460d-9ee0-fbfb5573ba07-kubelet-config\") pod \"global-pull-secret-syncer-pg9xz\" (UID: \"cfcad9fb-77bc-460d-9ee0-fbfb5573ba07\") " pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:32:25.282117 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.280353 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-etc-kubernetes\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.282117 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.280361 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-etc-openvswitch\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.282117 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.280537 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.282117 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.280577 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e85e7a89-c195-4f67-9843-c0f017468d83-sys\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.282117 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:25.280661 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:32:25.282117 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.280707 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-host-run-k8s-cni-cncf-io\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.282117 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.280732 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-hostroot\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.282117 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.280746 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-host-run-k8s-cni-cncf-io\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.282117 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.280818 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e85e7a89-c195-4f67-9843-c0f017468d83-sys\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.282117 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.280810 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/518044c4-7583-4f6b-b75c-710afb3ca571-multus-daemon-config\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.282117 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:25.280882 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63f8372e-78ca-4580-8d1e-a57593c38dac-metrics-certs podName:63f8372e-78ca-4580-8d1e-a57593c38dac nodeName:}" failed. No retries permitted until 2026-07-13 18:32:25.780863753 +0000 UTC m=+2.064292340 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63f8372e-78ca-4580-8d1e-a57593c38dac-metrics-certs") pod "network-metrics-daemon-hvwvw" (UID: "63f8372e-78ca-4580-8d1e-a57593c38dac") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:32:25.282117 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.280930 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e85e7a89-c195-4f67-9843-c0f017468d83-etc-sysctl-conf\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.282117 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.281017 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5fee7e65-59ed-4225-b607-67c9a5218be4-host\") pod \"node-ca-5qgnw\" (UID: \"5fee7e65-59ed-4225-b607-67c9a5218be4\") " pod="openshift-image-registry/node-ca-5qgnw" Jul 13 18:32:25.282117 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.281058 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e85e7a89-c195-4f67-9843-c0f017468d83-etc-sysctl-conf\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.282117 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.281058 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-var-lib-openvswitch\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.282117 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.281109 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-var-lib-openvswitch\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.282117 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.281160 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-host-cni-netd\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.282923 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.281183 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5fee7e65-59ed-4225-b607-67c9a5218be4-host\") pod \"node-ca-5qgnw\" (UID: \"5fee7e65-59ed-4225-b607-67c9a5218be4\") " pod="openshift-image-registry/node-ca-5qgnw" Jul 13 18:32:25.282923 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.281229 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-host-cni-netd\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.282923 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.281266 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-multus-cni-dir\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.282923 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.281307 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-multus-socket-dir-parent\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.282923 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.281325 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-multus-cni-dir\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.282923 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.281342 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/518044c4-7583-4f6b-b75c-710afb3ca571-multus-daemon-config\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.282923 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.281346 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/549c7399-8092-48f5-9596-048204fd4a82-iptables-alerter-script\") pod \"iptables-alerter-v6knc\" (UID: \"549c7399-8092-48f5-9596-048204fd4a82\") " pod="openshift-network-operator/iptables-alerter-v6knc" Jul 13 18:32:25.282923 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.281382 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-multus-socket-dir-parent\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.282923 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.281413 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c402e9df-0605-412f-89d2-7aeaa8cb1277-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-f7lnn\" (UID: \"c402e9df-0605-412f-89d2-7aeaa8cb1277\") " pod="openshift-multus/multus-additional-cni-plugins-f7lnn" Jul 13 18:32:25.282923 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.281433 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f8804320-8ca2-4737-b2cc-b7e585541b30-sys-fs\") pod \"aws-ebs-csi-driver-node-4p7qj\" (UID: \"f8804320-8ca2-4737-b2cc-b7e585541b30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4p7qj" Jul 13 18:32:25.282923 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.281491 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f8804320-8ca2-4737-b2cc-b7e585541b30-sys-fs\") pod \"aws-ebs-csi-driver-node-4p7qj\" (UID: \"f8804320-8ca2-4737-b2cc-b7e585541b30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4p7qj" Jul 13 18:32:25.282923 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.281557 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e85e7a89-c195-4f67-9843-c0f017468d83-var-lib-kubelet\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.282923 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.281588 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-systemd-units\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.282923 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.281614 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-host-cni-bin\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.282923 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.281659 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vh5kw\" (UniqueName: \"kubernetes.io/projected/36baa7e8-9898-491d-a961-c364c6e6af25-kube-api-access-vh5kw\") pod \"node-resolver-nt9r9\" (UID: \"36baa7e8-9898-491d-a961-c364c6e6af25\") " pod="openshift-dns/node-resolver-nt9r9" Jul 13 18:32:25.282923 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.281684 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c402e9df-0605-412f-89d2-7aeaa8cb1277-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-f7lnn\" (UID: \"c402e9df-0605-412f-89d2-7aeaa8cb1277\") " pod="openshift-multus/multus-additional-cni-plugins-f7lnn" Jul 13 18:32:25.282923 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.281707 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e85e7a89-c195-4f67-9843-c0f017468d83-tmp\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.283501 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.281733 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g72r\" (UniqueName: \"kubernetes.io/projected/b14ef695-c589-4fca-bd12-2e951f04e466-kube-api-access-7g72r\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.283501 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.281761 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-cnibin\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.283501 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.281787 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-host-var-lib-cni-multus\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.283501 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.281811 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c402e9df-0605-412f-89d2-7aeaa8cb1277-tuning-conf-dir\") pod \"multus-additional-cni-plugins-f7lnn\" (UID: \"c402e9df-0605-412f-89d2-7aeaa8cb1277\") " pod="openshift-multus/multus-additional-cni-plugins-f7lnn" Jul 13 18:32:25.283501 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.281840 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-host-slash\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.283501 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.281867 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-host-run-netns\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.283501 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.281894 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-node-log\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.283501 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.281921 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cfcad9fb-77bc-460d-9ee0-fbfb5573ba07-dbus\") pod \"global-pull-secret-syncer-pg9xz\" (UID: \"cfcad9fb-77bc-460d-9ee0-fbfb5573ba07\") " pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:32:25.283501 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.281948 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-host-run-multus-certs\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.283501 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.281937 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c402e9df-0605-412f-89d2-7aeaa8cb1277-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-f7lnn\" (UID: \"c402e9df-0605-412f-89d2-7aeaa8cb1277\") " pod="openshift-multus/multus-additional-cni-plugins-f7lnn" Jul 13 18:32:25.283501 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.281977 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jd2bt\" (UniqueName: \"kubernetes.io/projected/63f8372e-78ca-4580-8d1e-a57593c38dac-kube-api-access-jd2bt\") pod \"network-metrics-daemon-hvwvw\" (UID: \"63f8372e-78ca-4580-8d1e-a57593c38dac\") " pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:32:25.283501 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.282008 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvlm4\" (UniqueName: \"kubernetes.io/projected/549c7399-8092-48f5-9596-048204fd4a82-kube-api-access-qvlm4\") pod \"iptables-alerter-v6knc\" (UID: \"549c7399-8092-48f5-9596-048204fd4a82\") " pod="openshift-network-operator/iptables-alerter-v6knc" Jul 13 18:32:25.283501 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.282009 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/549c7399-8092-48f5-9596-048204fd4a82-iptables-alerter-script\") pod \"iptables-alerter-v6knc\" (UID: \"549c7399-8092-48f5-9596-048204fd4a82\") " pod="openshift-network-operator/iptables-alerter-v6knc" Jul 13 18:32:25.283501 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.282001 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c402e9df-0605-412f-89d2-7aeaa8cb1277-cni-binary-copy\") pod \"multus-additional-cni-plugins-f7lnn\" (UID: \"c402e9df-0605-412f-89d2-7aeaa8cb1277\") " pod="openshift-multus/multus-additional-cni-plugins-f7lnn" Jul 13 18:32:25.283501 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.282041 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-cnibin\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.283501 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.282051 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c402e9df-0605-412f-89d2-7aeaa8cb1277-system-cni-dir\") pod \"multus-additional-cni-plugins-f7lnn\" (UID: \"c402e9df-0605-412f-89d2-7aeaa8cb1277\") " pod="openshift-multus/multus-additional-cni-plugins-f7lnn" Jul 13 18:32:25.283501 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.282124 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c402e9df-0605-412f-89d2-7aeaa8cb1277-system-cni-dir\") pod \"multus-additional-cni-plugins-f7lnn\" (UID: \"c402e9df-0605-412f-89d2-7aeaa8cb1277\") " pod="openshift-multus/multus-additional-cni-plugins-f7lnn" Jul 13 18:32:25.284067 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.282169 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-systemd-units\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.284067 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.282214 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e85e7a89-c195-4f67-9843-c0f017468d83-var-lib-kubelet\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.284067 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.282223 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-host-cni-bin\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.284067 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.282251 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-host-run-netns\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.284067 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.282286 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-host-var-lib-cni-multus\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.284067 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.282358 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-node-log\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.284067 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.282403 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c402e9df-0605-412f-89d2-7aeaa8cb1277-tuning-conf-dir\") pod \"multus-additional-cni-plugins-f7lnn\" (UID: \"c402e9df-0605-412f-89d2-7aeaa8cb1277\") " pod="openshift-multus/multus-additional-cni-plugins-f7lnn" Jul 13 18:32:25.284067 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.282468 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b14ef695-c589-4fca-bd12-2e951f04e466-host-slash\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.284067 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.282556 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e85e7a89-c195-4f67-9843-c0f017468d83-etc-tuned\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.284067 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.282556 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/518044c4-7583-4f6b-b75c-710afb3ca571-host-run-multus-certs\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.284067 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.282627 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cfcad9fb-77bc-460d-9ee0-fbfb5573ba07-dbus\") pod \"global-pull-secret-syncer-pg9xz\" (UID: \"cfcad9fb-77bc-460d-9ee0-fbfb5573ba07\") " pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:32:25.284067 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.282930 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b14ef695-c589-4fca-bd12-2e951f04e466-ovn-node-metrics-cert\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.284067 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.282978 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a25a1123-769c-46d1-90a7-ceaf4dc36537-agent-certs\") pod \"konnectivity-agent-7x978\" (UID: \"a25a1123-769c-46d1-90a7-ceaf4dc36537\") " pod="kube-system/konnectivity-agent-7x978" Jul 13 18:32:25.284067 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.283220 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c402e9df-0605-412f-89d2-7aeaa8cb1277-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-f7lnn\" (UID: \"c402e9df-0605-412f-89d2-7aeaa8cb1277\") " pod="openshift-multus/multus-additional-cni-plugins-f7lnn" Jul 13 18:32:25.284067 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.284054 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e85e7a89-c195-4f67-9843-c0f017468d83-tmp\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.284989 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.284970 2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 18:32:25.288331 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.288302 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6xq6\" (UniqueName: \"kubernetes.io/projected/c402e9df-0605-412f-89d2-7aeaa8cb1277-kube-api-access-x6xq6\") pod \"multus-additional-cni-plugins-f7lnn\" (UID: \"c402e9df-0605-412f-89d2-7aeaa8cb1277\") " pod="openshift-multus/multus-additional-cni-plugins-f7lnn" Jul 13 18:32:25.289151 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.289101 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jtkn\" (UniqueName: \"kubernetes.io/projected/e85e7a89-c195-4f67-9843-c0f017468d83-kube-api-access-5jtkn\") pod \"tuned-v4m5p\" (UID: \"e85e7a89-c195-4f67-9843-c0f017468d83\") " pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.289346 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.289325 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmzr7\" (UniqueName: \"kubernetes.io/projected/5fee7e65-59ed-4225-b607-67c9a5218be4-kube-api-access-qmzr7\") pod \"node-ca-5qgnw\" (UID: \"5fee7e65-59ed-4225-b607-67c9a5218be4\") " pod="openshift-image-registry/node-ca-5qgnw" Jul 13 18:32:25.289444 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.289415 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2qfm\" (UniqueName: \"kubernetes.io/projected/f8804320-8ca2-4737-b2cc-b7e585541b30-kube-api-access-b2qfm\") pod \"aws-ebs-csi-driver-node-4p7qj\" (UID: \"f8804320-8ca2-4737-b2cc-b7e585541b30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4p7qj" Jul 13 18:32:25.289652 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.289636 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjtkb\" (UniqueName: \"kubernetes.io/projected/518044c4-7583-4f6b-b75c-710afb3ca571-kube-api-access-cjtkb\") pod \"multus-w59t6\" (UID: \"518044c4-7583-4f6b-b75c-710afb3ca571\") " pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.292829 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.292807 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvlm4\" (UniqueName: \"kubernetes.io/projected/549c7399-8092-48f5-9596-048204fd4a82-kube-api-access-qvlm4\") pod \"iptables-alerter-v6knc\" (UID: \"549c7399-8092-48f5-9596-048204fd4a82\") " pod="openshift-network-operator/iptables-alerter-v6knc" Jul 13 18:32:25.294506 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.294484 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vh5kw\" (UniqueName: \"kubernetes.io/projected/36baa7e8-9898-491d-a961-c364c6e6af25-kube-api-access-vh5kw\") pod \"node-resolver-nt9r9\" (UID: \"36baa7e8-9898-491d-a961-c364c6e6af25\") " pod="openshift-dns/node-resolver-nt9r9" Jul 13 18:32:25.294975 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.294958 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g72r\" (UniqueName: \"kubernetes.io/projected/b14ef695-c589-4fca-bd12-2e951f04e466-kube-api-access-7g72r\") pod \"ovnkube-node-8l7w6\" (UID: \"b14ef695-c589-4fca-bd12-2e951f04e466\") " pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.295249 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.295232 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jd2bt\" (UniqueName: \"kubernetes.io/projected/63f8372e-78ca-4580-8d1e-a57593c38dac-kube-api-access-jd2bt\") pod \"network-metrics-daemon-hvwvw\" (UID: \"63f8372e-78ca-4580-8d1e-a57593c38dac\") " pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:32:25.404171 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.404075 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-v6knc" Jul 13 18:32:25.415386 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:25.415358 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod549c7399_8092_48f5_9596_048204fd4a82.slice/crio-4acec4de53f06f35c756a95d107c0b7e224d5d9637cbdddaf64314f21bfa4f37 WatchSource:0}: Error finding container 4acec4de53f06f35c756a95d107c0b7e224d5d9637cbdddaf64314f21bfa4f37: Status 404 returned error can't find the container with id 4acec4de53f06f35c756a95d107c0b7e224d5d9637cbdddaf64314f21bfa4f37 Jul 13 18:32:25.420847 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.420827 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:25.429750 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:25.429720 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb14ef695_c589_4fca_bd12_2e951f04e466.slice/crio-e11f89c0c80dfab4e351dbdfaecb1335743f54eed70177a1763d69a294c294b7 WatchSource:0}: Error finding container e11f89c0c80dfab4e351dbdfaecb1335743f54eed70177a1763d69a294c294b7: Status 404 returned error can't find the container with id e11f89c0c80dfab4e351dbdfaecb1335743f54eed70177a1763d69a294c294b7 Jul 13 18:32:25.436611 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.436593 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7x978" Jul 13 18:32:25.445127 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.445099 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-f7lnn" Jul 13 18:32:25.445587 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:25.445565 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda25a1123_769c_46d1_90a7_ceaf4dc36537.slice/crio-10425af9f160e0e1bed856615c93cd3dcde0d79d92c32de56f1284f93a64bc44 WatchSource:0}: Error finding container 10425af9f160e0e1bed856615c93cd3dcde0d79d92c32de56f1284f93a64bc44: Status 404 returned error can't find the container with id 10425af9f160e0e1bed856615c93cd3dcde0d79d92c32de56f1284f93a64bc44 Jul 13 18:32:25.456177 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:25.456153 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc402e9df_0605_412f_89d2_7aeaa8cb1277.slice/crio-72f21e2d0ae287170f38ca3c4008339b424694ecc56bd2a8c7599509932356ab WatchSource:0}: Error finding container 72f21e2d0ae287170f38ca3c4008339b424694ecc56bd2a8c7599509932356ab: Status 404 returned error can't find the container with id 72f21e2d0ae287170f38ca3c4008339b424694ecc56bd2a8c7599509932356ab Jul 13 18:32:25.460353 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.460336 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4p7qj" Jul 13 18:32:25.469961 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:25.469936 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8804320_8ca2_4737_b2cc_b7e585541b30.slice/crio-380905e95a0e48b946b8f6a5191491573be6f7c16d50540017354f019f984a90 WatchSource:0}: Error finding container 380905e95a0e48b946b8f6a5191491573be6f7c16d50540017354f019f984a90: Status 404 returned error can't find the container with id 380905e95a0e48b946b8f6a5191491573be6f7c16d50540017354f019f984a90 Jul 13 18:32:25.475338 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.475310 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" Jul 13 18:32:25.481643 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.481603 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nt9r9" Jul 13 18:32:25.484055 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:25.484032 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode85e7a89_c195_4f67_9843_c0f017468d83.slice/crio-b43ff4bfb13e23d33879b3b8e587dc238250a8204252340f08544bd7c5d6909d WatchSource:0}: Error finding container b43ff4bfb13e23d33879b3b8e587dc238250a8204252340f08544bd7c5d6909d: Status 404 returned error can't find the container with id b43ff4bfb13e23d33879b3b8e587dc238250a8204252340f08544bd7c5d6909d Jul 13 18:32:25.490505 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:25.490483 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36baa7e8_9898_491d_a961_c364c6e6af25.slice/crio-9264cc67b09d4862b26c0b4ddf2fa3d94d0735f8e50625e45233f200991c7416 WatchSource:0}: Error finding container 9264cc67b09d4862b26c0b4ddf2fa3d94d0735f8e50625e45233f200991c7416: Status 404 returned error can't find the container with id 9264cc67b09d4862b26c0b4ddf2fa3d94d0735f8e50625e45233f200991c7416 Jul 13 18:32:25.498990 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.498971 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-w59t6" Jul 13 18:32:25.503852 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.503834 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5qgnw" Jul 13 18:32:25.508676 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:25.508653 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod518044c4_7583_4f6b_b75c_710afb3ca571.slice/crio-4a8c7c0715d8214fc03e31f73578485fa538f1e8ee292ec0169e24ec64fe3068 WatchSource:0}: Error finding container 4a8c7c0715d8214fc03e31f73578485fa538f1e8ee292ec0169e24ec64fe3068: Status 404 returned error can't find the container with id 4a8c7c0715d8214fc03e31f73578485fa538f1e8ee292ec0169e24ec64fe3068 Jul 13 18:32:25.514019 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:32:25.513997 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fee7e65_59ed_4225_b607_67c9a5218be4.slice/crio-97270a8e2164dc97198e1df5ed8940946c1288e89ae7425a350146eae93c06c9 WatchSource:0}: Error finding container 97270a8e2164dc97198e1df5ed8940946c1288e89ae7425a350146eae93c06c9: Status 404 returned error can't find the container with id 97270a8e2164dc97198e1df5ed8940946c1288e89ae7425a350146eae93c06c9 Jul 13 18:32:25.685138 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.685042 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnwrl\" (UniqueName: \"kubernetes.io/projected/241bb069-48c9-418e-9ef8-e2d8ffae0114-kube-api-access-tnwrl\") pod \"network-check-target-r24p4\" (UID: \"241bb069-48c9-418e-9ef8-e2d8ffae0114\") " pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:32:25.685292 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:25.685211 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 18:32:25.685292 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:25.685229 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 18:32:25.685292 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:25.685239 2582 projected.go:196] Error preparing data for projected volume kube-api-access-tnwrl for pod openshift-network-diagnostics/network-check-target-r24p4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:32:25.685292 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:25.685291 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/241bb069-48c9-418e-9ef8-e2d8ffae0114-kube-api-access-tnwrl podName:241bb069-48c9-418e-9ef8-e2d8ffae0114 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:26.685276595 +0000 UTC m=+2.968705176 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-tnwrl" (UniqueName: "kubernetes.io/projected/241bb069-48c9-418e-9ef8-e2d8ffae0114-kube-api-access-tnwrl") pod "network-check-target-r24p4" (UID: "241bb069-48c9-418e-9ef8-e2d8ffae0114") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:32:25.785787 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.785754 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63f8372e-78ca-4580-8d1e-a57593c38dac-metrics-certs\") pod \"network-metrics-daemon-hvwvw\" (UID: \"63f8372e-78ca-4580-8d1e-a57593c38dac\") " pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:32:25.785956 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:25.785813 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cfcad9fb-77bc-460d-9ee0-fbfb5573ba07-original-pull-secret\") pod \"global-pull-secret-syncer-pg9xz\" (UID: \"cfcad9fb-77bc-460d-9ee0-fbfb5573ba07\") " pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:32:25.785956 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:25.785917 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 18:32:25.785956 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:25.785922 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:32:25.786065 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:25.785971 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfcad9fb-77bc-460d-9ee0-fbfb5573ba07-original-pull-secret podName:cfcad9fb-77bc-460d-9ee0-fbfb5573ba07 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:26.785957588 +0000 UTC m=+3.069386169 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cfcad9fb-77bc-460d-9ee0-fbfb5573ba07-original-pull-secret") pod "global-pull-secret-syncer-pg9xz" (UID: "cfcad9fb-77bc-460d-9ee0-fbfb5573ba07") : object "kube-system"/"original-pull-secret" not registered Jul 13 18:32:25.786065 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:25.785989 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63f8372e-78ca-4580-8d1e-a57593c38dac-metrics-certs podName:63f8372e-78ca-4580-8d1e-a57593c38dac nodeName:}" failed. No retries permitted until 2026-07-13 18:32:26.785980462 +0000 UTC m=+3.069409045 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63f8372e-78ca-4580-8d1e-a57593c38dac-metrics-certs") pod "network-metrics-daemon-hvwvw" (UID: "63f8372e-78ca-4580-8d1e-a57593c38dac") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:32:26.119818 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:26.119712 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 18:27:25 +0000 UTC" deadline="2028-03-02 12:23:18.54358552 +0000 UTC" Jul 13 18:32:26.119818 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:26.119756 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14345h50m52.423834844s" Jul 13 18:32:26.274708 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:26.274664 2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 18:32:26.275043 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:26.274998 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:32:26.275155 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:26.275130 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r24p4" podUID="241bb069-48c9-418e-9ef8-e2d8ffae0114" Jul 13 18:32:26.308136 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:26.308099 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5qgnw" event={"ID":"5fee7e65-59ed-4225-b607-67c9a5218be4","Type":"ContainerStarted","Data":"97270a8e2164dc97198e1df5ed8940946c1288e89ae7425a350146eae93c06c9"} Jul 13 18:32:26.323256 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:26.323084 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nt9r9" event={"ID":"36baa7e8-9898-491d-a961-c364c6e6af25","Type":"ContainerStarted","Data":"9264cc67b09d4862b26c0b4ddf2fa3d94d0735f8e50625e45233f200991c7416"} Jul 13 18:32:26.326439 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:26.326364 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" event={"ID":"e85e7a89-c195-4f67-9843-c0f017468d83","Type":"ContainerStarted","Data":"b43ff4bfb13e23d33879b3b8e587dc238250a8204252340f08544bd7c5d6909d"} Jul 13 18:32:26.332232 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:26.332151 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4p7qj" event={"ID":"f8804320-8ca2-4737-b2cc-b7e585541b30","Type":"ContainerStarted","Data":"380905e95a0e48b946b8f6a5191491573be6f7c16d50540017354f019f984a90"} Jul 13 18:32:26.335694 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:26.335600 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f7lnn" event={"ID":"c402e9df-0605-412f-89d2-7aeaa8cb1277","Type":"ContainerStarted","Data":"72f21e2d0ae287170f38ca3c4008339b424694ecc56bd2a8c7599509932356ab"} Jul 13 18:32:26.340567 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:26.340408 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" event={"ID":"b14ef695-c589-4fca-bd12-2e951f04e466","Type":"ContainerStarted","Data":"e11f89c0c80dfab4e351dbdfaecb1335743f54eed70177a1763d69a294c294b7"} Jul 13 18:32:26.361563 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:26.361517 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-w59t6" event={"ID":"518044c4-7583-4f6b-b75c-710afb3ca571","Type":"ContainerStarted","Data":"4a8c7c0715d8214fc03e31f73578485fa538f1e8ee292ec0169e24ec64fe3068"} Jul 13 18:32:26.368987 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:26.368939 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7x978" event={"ID":"a25a1123-769c-46d1-90a7-ceaf4dc36537","Type":"ContainerStarted","Data":"10425af9f160e0e1bed856615c93cd3dcde0d79d92c32de56f1284f93a64bc44"} Jul 13 18:32:26.380071 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:26.379986 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-v6knc" event={"ID":"549c7399-8092-48f5-9596-048204fd4a82","Type":"ContainerStarted","Data":"4acec4de53f06f35c756a95d107c0b7e224d5d9637cbdddaf64314f21bfa4f37"} Jul 13 18:32:26.694473 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:26.694379 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnwrl\" (UniqueName: \"kubernetes.io/projected/241bb069-48c9-418e-9ef8-e2d8ffae0114-kube-api-access-tnwrl\") pod \"network-check-target-r24p4\" (UID: \"241bb069-48c9-418e-9ef8-e2d8ffae0114\") " pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:32:26.694648 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:26.694613 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 18:32:26.694648 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:26.694635 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 18:32:26.694826 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:26.694666 2582 projected.go:196] Error preparing data for projected volume kube-api-access-tnwrl for pod openshift-network-diagnostics/network-check-target-r24p4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:32:26.694826 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:26.694737 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/241bb069-48c9-418e-9ef8-e2d8ffae0114-kube-api-access-tnwrl podName:241bb069-48c9-418e-9ef8-e2d8ffae0114 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:28.694715522 +0000 UTC m=+4.978144107 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-tnwrl" (UniqueName: "kubernetes.io/projected/241bb069-48c9-418e-9ef8-e2d8ffae0114-kube-api-access-tnwrl") pod "network-check-target-r24p4" (UID: "241bb069-48c9-418e-9ef8-e2d8ffae0114") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:32:26.794810 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:26.794769 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cfcad9fb-77bc-460d-9ee0-fbfb5573ba07-original-pull-secret\") pod \"global-pull-secret-syncer-pg9xz\" (UID: \"cfcad9fb-77bc-460d-9ee0-fbfb5573ba07\") " pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:32:26.795005 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:26.794848 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63f8372e-78ca-4580-8d1e-a57593c38dac-metrics-certs\") pod \"network-metrics-daemon-hvwvw\" (UID: \"63f8372e-78ca-4580-8d1e-a57593c38dac\") " pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:32:26.795005 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:26.794997 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:32:26.795132 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:26.795062 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63f8372e-78ca-4580-8d1e-a57593c38dac-metrics-certs podName:63f8372e-78ca-4580-8d1e-a57593c38dac nodeName:}" failed. No retries permitted until 2026-07-13 18:32:28.795043068 +0000 UTC m=+5.078471652 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63f8372e-78ca-4580-8d1e-a57593c38dac-metrics-certs") pod "network-metrics-daemon-hvwvw" (UID: "63f8372e-78ca-4580-8d1e-a57593c38dac") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:32:26.795476 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:26.795459 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 18:32:26.795565 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:26.795507 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfcad9fb-77bc-460d-9ee0-fbfb5573ba07-original-pull-secret podName:cfcad9fb-77bc-460d-9ee0-fbfb5573ba07 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:28.795492894 +0000 UTC m=+5.078921502 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cfcad9fb-77bc-460d-9ee0-fbfb5573ba07-original-pull-secret") pod "global-pull-secret-syncer-pg9xz" (UID: "cfcad9fb-77bc-460d-9ee0-fbfb5573ba07") : object "kube-system"/"original-pull-secret" not registered Jul 13 18:32:27.121044 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:27.120929 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 18:27:25 +0000 UTC" deadline="2028-04-27 05:22:57.519042594 +0000 UTC" Jul 13 18:32:27.121044 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:27.120994 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15682h50m30.398052757s" Jul 13 18:32:27.262547 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:27.262362 2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 18:32:27.275008 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:27.274976 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:32:27.275184 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:27.275118 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hvwvw" podUID="63f8372e-78ca-4580-8d1e-a57593c38dac" Jul 13 18:32:27.275636 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:27.275609 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:32:27.275756 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:27.275738 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pg9xz" podUID="cfcad9fb-77bc-460d-9ee0-fbfb5573ba07" Jul 13 18:32:27.583669 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:27.583411 2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 18:32:28.278120 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:28.278076 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:32:28.278566 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:28.278212 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r24p4" podUID="241bb069-48c9-418e-9ef8-e2d8ffae0114" Jul 13 18:32:28.710576 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:28.710482 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnwrl\" (UniqueName: \"kubernetes.io/projected/241bb069-48c9-418e-9ef8-e2d8ffae0114-kube-api-access-tnwrl\") pod \"network-check-target-r24p4\" (UID: \"241bb069-48c9-418e-9ef8-e2d8ffae0114\") " pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:32:28.710788 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:28.710721 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 18:32:28.710788 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:28.710748 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 18:32:28.710788 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:28.710765 2582 projected.go:196] Error preparing data for projected volume kube-api-access-tnwrl for pod openshift-network-diagnostics/network-check-target-r24p4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:32:28.710967 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:28.710830 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/241bb069-48c9-418e-9ef8-e2d8ffae0114-kube-api-access-tnwrl podName:241bb069-48c9-418e-9ef8-e2d8ffae0114 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:32.710809074 +0000 UTC m=+8.994237680 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-tnwrl" (UniqueName: "kubernetes.io/projected/241bb069-48c9-418e-9ef8-e2d8ffae0114-kube-api-access-tnwrl") pod "network-check-target-r24p4" (UID: "241bb069-48c9-418e-9ef8-e2d8ffae0114") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:32:28.811853 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:28.811809 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cfcad9fb-77bc-460d-9ee0-fbfb5573ba07-original-pull-secret\") pod \"global-pull-secret-syncer-pg9xz\" (UID: \"cfcad9fb-77bc-460d-9ee0-fbfb5573ba07\") " pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:32:28.812056 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:28.811905 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63f8372e-78ca-4580-8d1e-a57593c38dac-metrics-certs\") pod \"network-metrics-daemon-hvwvw\" (UID: \"63f8372e-78ca-4580-8d1e-a57593c38dac\") " pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:32:28.812156 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:28.812098 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:32:28.812239 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:28.812164 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63f8372e-78ca-4580-8d1e-a57593c38dac-metrics-certs podName:63f8372e-78ca-4580-8d1e-a57593c38dac nodeName:}" failed. No retries permitted until 2026-07-13 18:32:32.81214286 +0000 UTC m=+9.095571445 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63f8372e-78ca-4580-8d1e-a57593c38dac-metrics-certs") pod "network-metrics-daemon-hvwvw" (UID: "63f8372e-78ca-4580-8d1e-a57593c38dac") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:32:28.812606 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:28.812583 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 18:32:28.812735 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:28.812642 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfcad9fb-77bc-460d-9ee0-fbfb5573ba07-original-pull-secret podName:cfcad9fb-77bc-460d-9ee0-fbfb5573ba07 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:32.81262684 +0000 UTC m=+9.096055424 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cfcad9fb-77bc-460d-9ee0-fbfb5573ba07-original-pull-secret") pod "global-pull-secret-syncer-pg9xz" (UID: "cfcad9fb-77bc-460d-9ee0-fbfb5573ba07") : object "kube-system"/"original-pull-secret" not registered Jul 13 18:32:29.275199 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:29.274911 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:32:29.275382 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:29.275334 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pg9xz" podUID="cfcad9fb-77bc-460d-9ee0-fbfb5573ba07" Jul 13 18:32:29.275812 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:29.275786 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:32:29.275925 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:29.275904 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hvwvw" podUID="63f8372e-78ca-4580-8d1e-a57593c38dac" Jul 13 18:32:30.277624 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:30.277440 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:32:30.277624 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:30.277564 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r24p4" podUID="241bb069-48c9-418e-9ef8-e2d8ffae0114" Jul 13 18:32:31.275086 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:31.275050 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:32:31.275254 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:31.275093 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:32:31.275254 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:31.275172 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pg9xz" podUID="cfcad9fb-77bc-460d-9ee0-fbfb5573ba07" Jul 13 18:32:31.275382 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:31.275325 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hvwvw" podUID="63f8372e-78ca-4580-8d1e-a57593c38dac" Jul 13 18:32:32.276932 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:32.276852 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:32:32.277298 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:32.276982 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r24p4" podUID="241bb069-48c9-418e-9ef8-e2d8ffae0114" Jul 13 18:32:32.748584 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:32.748497 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnwrl\" (UniqueName: \"kubernetes.io/projected/241bb069-48c9-418e-9ef8-e2d8ffae0114-kube-api-access-tnwrl\") pod \"network-check-target-r24p4\" (UID: \"241bb069-48c9-418e-9ef8-e2d8ffae0114\") " pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:32:32.748755 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:32.748700 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 18:32:32.748755 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:32.748724 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 18:32:32.748755 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:32.748737 2582 projected.go:196] Error preparing data for projected volume kube-api-access-tnwrl for pod openshift-network-diagnostics/network-check-target-r24p4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:32:32.748937 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:32.748801 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/241bb069-48c9-418e-9ef8-e2d8ffae0114-kube-api-access-tnwrl podName:241bb069-48c9-418e-9ef8-e2d8ffae0114 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:40.748781648 +0000 UTC m=+17.032210251 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-tnwrl" (UniqueName: "kubernetes.io/projected/241bb069-48c9-418e-9ef8-e2d8ffae0114-kube-api-access-tnwrl") pod "network-check-target-r24p4" (UID: "241bb069-48c9-418e-9ef8-e2d8ffae0114") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:32:32.849524 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:32.848813 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63f8372e-78ca-4580-8d1e-a57593c38dac-metrics-certs\") pod \"network-metrics-daemon-hvwvw\" (UID: \"63f8372e-78ca-4580-8d1e-a57593c38dac\") " pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:32:32.849524 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:32.848878 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cfcad9fb-77bc-460d-9ee0-fbfb5573ba07-original-pull-secret\") pod \"global-pull-secret-syncer-pg9xz\" (UID: \"cfcad9fb-77bc-460d-9ee0-fbfb5573ba07\") " pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:32:32.849524 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:32.848992 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 18:32:32.849524 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:32.849055 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfcad9fb-77bc-460d-9ee0-fbfb5573ba07-original-pull-secret podName:cfcad9fb-77bc-460d-9ee0-fbfb5573ba07 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:40.849033928 +0000 UTC m=+17.132462528 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cfcad9fb-77bc-460d-9ee0-fbfb5573ba07-original-pull-secret") pod "global-pull-secret-syncer-pg9xz" (UID: "cfcad9fb-77bc-460d-9ee0-fbfb5573ba07") : object "kube-system"/"original-pull-secret" not registered Jul 13 18:32:32.849524 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:32.849145 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:32:32.849524 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:32.849183 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63f8372e-78ca-4580-8d1e-a57593c38dac-metrics-certs podName:63f8372e-78ca-4580-8d1e-a57593c38dac nodeName:}" failed. No retries permitted until 2026-07-13 18:32:40.849170136 +0000 UTC m=+17.132598732 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63f8372e-78ca-4580-8d1e-a57593c38dac-metrics-certs") pod "network-metrics-daemon-hvwvw" (UID: "63f8372e-78ca-4580-8d1e-a57593c38dac") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:32:33.275157 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:33.275122 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:32:33.275309 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:33.275162 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:32:33.275309 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:33.275265 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pg9xz" podUID="cfcad9fb-77bc-460d-9ee0-fbfb5573ba07" Jul 13 18:32:33.275450 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:33.275420 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hvwvw" podUID="63f8372e-78ca-4580-8d1e-a57593c38dac" Jul 13 18:32:34.275595 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:34.275561 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:32:34.276040 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:34.275691 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r24p4" podUID="241bb069-48c9-418e-9ef8-e2d8ffae0114" Jul 13 18:32:35.275414 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:35.275373 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:32:35.275565 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:35.275519 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hvwvw" podUID="63f8372e-78ca-4580-8d1e-a57593c38dac" Jul 13 18:32:35.275906 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:35.275884 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:32:35.276296 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:35.275986 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pg9xz" podUID="cfcad9fb-77bc-460d-9ee0-fbfb5573ba07" Jul 13 18:32:36.275042 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:36.275006 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:32:36.275224 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:36.275145 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r24p4" podUID="241bb069-48c9-418e-9ef8-e2d8ffae0114" Jul 13 18:32:37.275062 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:37.275034 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:32:37.275437 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:37.275137 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hvwvw" podUID="63f8372e-78ca-4580-8d1e-a57593c38dac" Jul 13 18:32:37.275437 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:37.275190 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:32:37.275437 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:37.275280 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pg9xz" podUID="cfcad9fb-77bc-460d-9ee0-fbfb5573ba07" Jul 13 18:32:38.275109 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:38.275035 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:32:38.275493 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:38.275169 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r24p4" podUID="241bb069-48c9-418e-9ef8-e2d8ffae0114" Jul 13 18:32:39.274939 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:39.274902 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:32:39.275123 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:39.274902 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:32:39.275123 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:39.275046 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hvwvw" podUID="63f8372e-78ca-4580-8d1e-a57593c38dac" Jul 13 18:32:39.275123 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:39.275099 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pg9xz" podUID="cfcad9fb-77bc-460d-9ee0-fbfb5573ba07" Jul 13 18:32:40.274633 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:40.274585 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:32:40.274822 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:40.274725 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r24p4" podUID="241bb069-48c9-418e-9ef8-e2d8ffae0114" Jul 13 18:32:40.814664 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:40.814623 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnwrl\" (UniqueName: \"kubernetes.io/projected/241bb069-48c9-418e-9ef8-e2d8ffae0114-kube-api-access-tnwrl\") pod \"network-check-target-r24p4\" (UID: \"241bb069-48c9-418e-9ef8-e2d8ffae0114\") " pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:32:40.815169 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:40.814833 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 18:32:40.815169 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:40.814862 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 18:32:40.815169 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:40.814878 2582 projected.go:196] Error preparing data for projected volume kube-api-access-tnwrl for pod openshift-network-diagnostics/network-check-target-r24p4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:32:40.815169 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:40.814947 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/241bb069-48c9-418e-9ef8-e2d8ffae0114-kube-api-access-tnwrl podName:241bb069-48c9-418e-9ef8-e2d8ffae0114 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:56.814925475 +0000 UTC m=+33.098354075 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-tnwrl" (UniqueName: "kubernetes.io/projected/241bb069-48c9-418e-9ef8-e2d8ffae0114-kube-api-access-tnwrl") pod "network-check-target-r24p4" (UID: "241bb069-48c9-418e-9ef8-e2d8ffae0114") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:32:40.914999 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:40.914958 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63f8372e-78ca-4580-8d1e-a57593c38dac-metrics-certs\") pod \"network-metrics-daemon-hvwvw\" (UID: \"63f8372e-78ca-4580-8d1e-a57593c38dac\") " pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:32:40.915164 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:40.915059 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cfcad9fb-77bc-460d-9ee0-fbfb5573ba07-original-pull-secret\") pod \"global-pull-secret-syncer-pg9xz\" (UID: \"cfcad9fb-77bc-460d-9ee0-fbfb5573ba07\") " pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:32:40.915164 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:40.915136 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:32:40.915264 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:40.915169 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 18:32:40.915264 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:40.915238 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63f8372e-78ca-4580-8d1e-a57593c38dac-metrics-certs podName:63f8372e-78ca-4580-8d1e-a57593c38dac nodeName:}" failed. No retries permitted until 2026-07-13 18:32:56.915214837 +0000 UTC m=+33.198643421 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63f8372e-78ca-4580-8d1e-a57593c38dac-metrics-certs") pod "network-metrics-daemon-hvwvw" (UID: "63f8372e-78ca-4580-8d1e-a57593c38dac") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:32:40.915264 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:40.915264 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfcad9fb-77bc-460d-9ee0-fbfb5573ba07-original-pull-secret podName:cfcad9fb-77bc-460d-9ee0-fbfb5573ba07 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:56.915250554 +0000 UTC m=+33.198679143 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cfcad9fb-77bc-460d-9ee0-fbfb5573ba07-original-pull-secret") pod "global-pull-secret-syncer-pg9xz" (UID: "cfcad9fb-77bc-460d-9ee0-fbfb5573ba07") : object "kube-system"/"original-pull-secret" not registered Jul 13 18:32:41.275447 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:41.275362 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:32:41.275673 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:41.275384 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:32:41.275673 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:41.275497 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hvwvw" podUID="63f8372e-78ca-4580-8d1e-a57593c38dac" Jul 13 18:32:41.275673 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:41.275527 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pg9xz" podUID="cfcad9fb-77bc-460d-9ee0-fbfb5573ba07" Jul 13 18:32:42.275509 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:42.275475 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:32:42.276018 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:42.275627 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r24p4" podUID="241bb069-48c9-418e-9ef8-e2d8ffae0114" Jul 13 18:32:43.274553 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:43.274522 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:32:43.274757 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:43.274686 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hvwvw" podUID="63f8372e-78ca-4580-8d1e-a57593c38dac" Jul 13 18:32:43.274813 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:43.274758 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:32:43.274907 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:43.274883 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pg9xz" podUID="cfcad9fb-77bc-460d-9ee0-fbfb5573ba07" Jul 13 18:32:44.275359 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:44.275322 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:32:44.275839 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:44.275423 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r24p4" podUID="241bb069-48c9-418e-9ef8-e2d8ffae0114" Jul 13 18:32:45.274767 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:45.274729 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:32:45.274974 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:45.274864 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hvwvw" podUID="63f8372e-78ca-4580-8d1e-a57593c38dac" Jul 13 18:32:45.275274 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:45.275256 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:32:45.275377 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:45.275359 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pg9xz" podUID="cfcad9fb-77bc-460d-9ee0-fbfb5573ba07" Jul 13 18:32:46.274826 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:46.274805 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:32:46.275095 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:46.274904 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r24p4" podUID="241bb069-48c9-418e-9ef8-e2d8ffae0114" Jul 13 18:32:46.430428 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:46.430398 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-191.ec2.internal" event={"ID":"717de0a249da5500e718fed7ba33a501","Type":"ContainerStarted","Data":"14e88d0e59c0b7f9e94513257b264634c4442169e2791a8dad81f6414f209e6d"} Jul 13 18:32:46.454968 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:46.454925 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-191.ec2.internal" podStartSLOduration=21.454911458 podStartE2EDuration="21.454911458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:32:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:32:46.454426811 +0000 UTC m=+22.737855411" watchObservedRunningTime="2026-07-13 18:32:46.454911458 +0000 UTC m=+22.738340059" Jul 13 18:32:47.275730 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:47.275498 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:32:47.276452 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:47.275852 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hvwvw" podUID="63f8372e-78ca-4580-8d1e-a57593c38dac" Jul 13 18:32:47.276452 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:47.275545 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:32:47.276452 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:47.276053 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pg9xz" podUID="cfcad9fb-77bc-460d-9ee0-fbfb5573ba07" Jul 13 18:32:47.434432 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:47.434165 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-w59t6" event={"ID":"518044c4-7583-4f6b-b75c-710afb3ca571","Type":"ContainerStarted","Data":"c7561d883687932fe6eedd45fc0b078edd51c3c6d9e3a9933463e55c0048b952"} Jul 13 18:32:47.435557 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:47.435511 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7x978" event={"ID":"a25a1123-769c-46d1-90a7-ceaf4dc36537","Type":"ContainerStarted","Data":"e746aa13726f9a2d1fbfbafe6565d916eecb0134873fa304bdc9a26ab62fd645"} Jul 13 18:32:47.438493 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:47.438450 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-191.ec2.internal" event={"ID":"ac6f50c32a4e57fee7b6cab885240e5e","Type":"ContainerStarted","Data":"e9919f7ed75e93d9ddea4e2e0e7313b034deb50b391867bcc4964bb7bfcfc1f5"} Jul 13 18:32:47.440982 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:47.440954 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5qgnw" event={"ID":"5fee7e65-59ed-4225-b607-67c9a5218be4","Type":"ContainerStarted","Data":"973f2461a1eef8af753f446afed0e48260bc5a718a76b682a39ebfccb05cb9a8"} Jul 13 18:32:47.444357 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:47.444290 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nt9r9" event={"ID":"36baa7e8-9898-491d-a961-c364c6e6af25","Type":"ContainerStarted","Data":"95520860d42b78a91515f69bd7093a77fc36675e537870af2c79ce5f7406226d"} Jul 13 18:32:47.448186 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:47.447671 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" event={"ID":"e85e7a89-c195-4f67-9843-c0f017468d83","Type":"ContainerStarted","Data":"07d2182954bf95f222ef534fc170abffabd6c89d55d174d534ac66503c65dab8"} Jul 13 18:32:47.449711 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:47.449676 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4p7qj" event={"ID":"f8804320-8ca2-4737-b2cc-b7e585541b30","Type":"ContainerStarted","Data":"c7b3cd1df337cbdcc9ec7b410aa9e2d1b6af6c4b49411ee7e72425bff7e89563"} Jul 13 18:32:47.451252 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:47.451230 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f7lnn" event={"ID":"c402e9df-0605-412f-89d2-7aeaa8cb1277","Type":"ContainerStarted","Data":"8a5841b3138e969ff3e926c300843474ea19aea5b5367d6e18b86309173fe292"} Jul 13 18:32:47.456697 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:47.455967 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-w59t6" podStartSLOduration=2.8231008859999998 podStartE2EDuration="23.455951473s" totalImagesPullingTime="20.632850587s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:32:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:32:47.455354553 +0000 UTC m=+23.738783158" watchObservedRunningTime="2026-07-13 18:32:47.455951473 +0000 UTC m=+23.739380078" Jul 13 18:32:47.459766 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:47.459747 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8l7w6_b14ef695-c589-4fca-bd12-2e951f04e466/ovn-acl-logging/0.log" Jul 13 18:32:47.460151 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:47.460124 2582 generic.go:356] "Generic (PLEG): container finished" podID="b14ef695-c589-4fca-bd12-2e951f04e466" containerID="4a0c469fe573e18f36173ca041589204d739f571263792680ad0a2607558519f" exitCode=1 Jul 13 18:32:47.460251 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:47.460184 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" event={"ID":"b14ef695-c589-4fca-bd12-2e951f04e466","Type":"ContainerStarted","Data":"4a0a09a30109d573237d2267701d8546d52abddecb11f18225f35fc2b567927f"} Jul 13 18:32:47.460311 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:47.460258 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" event={"ID":"b14ef695-c589-4fca-bd12-2e951f04e466","Type":"ContainerStarted","Data":"e0ad9453526f61223914111eef83a230675da6374adefaf9e7a09d92c0d7635c"} Jul 13 18:32:47.460311 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:47.460304 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" event={"ID":"b14ef695-c589-4fca-bd12-2e951f04e466","Type":"ContainerStarted","Data":"15d49dfc942bfe38612251a01e4c13c634953840c8e3ee1192268ac0eded2644"} Jul 13 18:32:47.460412 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:47.460338 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" event={"ID":"b14ef695-c589-4fca-bd12-2e951f04e466","Type":"ContainerStarted","Data":"7c9d596a2dc82e68c4cbff426cc5340dc4a49fa5eb1bf7f47c528f2f2b226c38"} Jul 13 18:32:47.460412 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:47.460374 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" event={"ID":"b14ef695-c589-4fca-bd12-2e951f04e466","Type":"ContainerDied","Data":"4a0c469fe573e18f36173ca041589204d739f571263792680ad0a2607558519f"} Jul 13 18:32:47.460510 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:47.460422 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" event={"ID":"b14ef695-c589-4fca-bd12-2e951f04e466","Type":"ContainerStarted","Data":"39e1d50bed95c2dfa35124378bd5a417c90e2af8343cb74811fd0c02ce15356a"} Jul 13 18:32:47.471565 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:47.471532 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-7x978" podStartSLOduration=2.8808488370000003 podStartE2EDuration="23.471519046s" totalImagesPullingTime="20.590670209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:32:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:32:47.471517882 +0000 UTC m=+23.754946483" watchObservedRunningTime="2026-07-13 18:32:47.471519046 +0000 UTC m=+23.754947648" Jul 13 18:32:47.514034 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:47.513984 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-v4m5p" podStartSLOduration=2.893228181 podStartE2EDuration="23.513968688s" totalImagesPullingTime="20.620740507s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:32:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:32:47.513801094 +0000 UTC m=+23.797229696" watchObservedRunningTime="2026-07-13 18:32:47.513968688 +0000 UTC m=+23.797397290" Jul 13 18:32:47.545470 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:47.545425 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-nt9r9" podStartSLOduration=2.999521877 podStartE2EDuration="23.545407291s" totalImagesPullingTime="20.545885414s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:32:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:32:47.545216585 +0000 UTC m=+23.828645187" watchObservedRunningTime="2026-07-13 18:32:47.545407291 +0000 UTC m=+23.828835892" Jul 13 18:32:47.620959 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:47.620921 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:32:47.679311 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:47.679279 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:32:47.914826 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:47.914784 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:32:47.988912 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:47.988865 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:32:48.202492 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:48.202459 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-7x978" Jul 13 18:32:48.203479 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:48.203459 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-7x978" Jul 13 18:32:48.276139 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:48.276118 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:32:48.276484 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:48.276209 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r24p4" podUID="241bb069-48c9-418e-9ef8-e2d8ffae0114" Jul 13 18:32:48.463706 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:48.463677 2582 generic.go:356] "Generic (PLEG): container finished" podID="ac6f50c32a4e57fee7b6cab885240e5e" containerID="e9919f7ed75e93d9ddea4e2e0e7313b034deb50b391867bcc4964bb7bfcfc1f5" exitCode=0 Jul 13 18:32:48.463859 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:48.463764 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-191.ec2.internal" event={"ID":"ac6f50c32a4e57fee7b6cab885240e5e","Type":"ContainerDied","Data":"e9919f7ed75e93d9ddea4e2e0e7313b034deb50b391867bcc4964bb7bfcfc1f5"} Jul 13 18:32:48.465291 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:48.465266 2582 generic.go:356] "Generic (PLEG): container finished" podID="c402e9df-0605-412f-89d2-7aeaa8cb1277" containerID="8a5841b3138e969ff3e926c300843474ea19aea5b5367d6e18b86309173fe292" exitCode=0 Jul 13 18:32:48.465416 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:48.465330 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f7lnn" event={"ID":"c402e9df-0605-412f-89d2-7aeaa8cb1277","Type":"ContainerDied","Data":"8a5841b3138e969ff3e926c300843474ea19aea5b5367d6e18b86309173fe292"} Jul 13 18:32:48.466968 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:48.466921 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-v6knc" event={"ID":"549c7399-8092-48f5-9596-048204fd4a82","Type":"ContainerStarted","Data":"9a11a147dab9f39c2b95808b83f6496a4f0791cd452f73fceefcec6ef10d85d9"} Jul 13 18:32:48.467298 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:48.467279 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-7x978" Jul 13 18:32:48.467789 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:48.467774 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-7x978" Jul 13 18:32:48.505499 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:48.505454 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-v6knc" podStartSLOduration=3.80257784 podStartE2EDuration="24.505442252s" totalImagesPullingTime="20.702864412s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:32:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:32:48.505136172 +0000 UTC m=+24.788564776" watchObservedRunningTime="2026-07-13 18:32:48.505442252 +0000 UTC m=+24.788870854" Jul 13 18:32:48.541706 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:48.541663 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5qgnw" podStartSLOduration=3.019087798 podStartE2EDuration="23.541648433s" totalImagesPullingTime="20.522560635s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:32:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:32:48.541128501 +0000 UTC m=+24.824557104" watchObservedRunningTime="2026-07-13 18:32:48.541648433 +0000 UTC m=+24.825077035" Jul 13 18:32:48.612456 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:48.612436 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:32:48.692582 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:48.692563 2582 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 13 18:32:48.707136 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:48.707082 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:32:48.805286 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:48.805205 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:32:48.886551 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:48.886511 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:32:49.082253 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:49.082181 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:32:49.153453 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:49.153130 2582 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-13T18:32:48.692578172Z","UUID":"8cc8ff1e-ec28-49eb-b4d3-b83294364b9c","Handler":null,"Name":"","Endpoint":""} Jul 13 18:32:49.154979 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:49.154954 2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 13 18:32:49.155094 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:49.155001 2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 13 18:32:49.168640 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:49.168477 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:32:49.274789 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:49.274760 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:32:49.275066 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:49.274908 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hvwvw" podUID="63f8372e-78ca-4580-8d1e-a57593c38dac" Jul 13 18:32:49.275066 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:49.274978 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:32:49.275066 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:49.275043 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pg9xz" podUID="cfcad9fb-77bc-460d-9ee0-fbfb5573ba07" Jul 13 18:32:49.471090 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:49.471039 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-191.ec2.internal" event={"ID":"ac6f50c32a4e57fee7b6cab885240e5e","Type":"ContainerStarted","Data":"5f57f398b7f20000391313dcccd8540de482d1a2adc64f52e9c0dc51d4c8bfc1"} Jul 13 18:32:49.474066 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:49.474039 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4p7qj" event={"ID":"f8804320-8ca2-4737-b2cc-b7e585541b30","Type":"ContainerStarted","Data":"d9c783631085c7bf4e7a3e789a80d8510893af8ce9e6c14977d59b7b04ffc7c5"} Jul 13 18:32:50.275311 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:50.275234 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:32:50.275459 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:50.275371 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r24p4" podUID="241bb069-48c9-418e-9ef8-e2d8ffae0114" Jul 13 18:32:50.477992 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:50.477787 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4p7qj" event={"ID":"f8804320-8ca2-4737-b2cc-b7e585541b30","Type":"ContainerStarted","Data":"60d7e990b8ec021cbf36f930608ed31d827f60a7613ca146399beb477e7e0a37"} Jul 13 18:32:50.480929 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:50.480910 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8l7w6_b14ef695-c589-4fca-bd12-2e951f04e466/ovn-acl-logging/0.log" Jul 13 18:32:50.481375 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:50.481350 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" event={"ID":"b14ef695-c589-4fca-bd12-2e951f04e466","Type":"ContainerStarted","Data":"da6df9aa86eb281b36f613684ae67c20ae5f70b49ec38306d4c22ec811fe0964"} Jul 13 18:32:50.502813 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:50.502768 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4p7qj" podStartSLOduration=3.586893349 podStartE2EDuration="26.502748549s" totalImagesPullingTime="22.9158552s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:32:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:32:50.50270273 +0000 UTC m=+26.786131352" watchObservedRunningTime="2026-07-13 18:32:50.502748549 +0000 UTC m=+26.786177152" Jul 13 18:32:50.503797 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:50.503754 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-191.ec2.internal" podStartSLOduration=25.50374229 podStartE2EDuration="25.50374229s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:32:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:32:49.490928314 +0000 UTC m=+25.774356918" watchObservedRunningTime="2026-07-13 18:32:50.50374229 +0000 UTC m=+26.787170896" Jul 13 18:32:51.173466 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:51.173429 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:32:51.274751 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:51.274715 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:32:51.274906 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:51.274855 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hvwvw" podUID="63f8372e-78ca-4580-8d1e-a57593c38dac" Jul 13 18:32:51.274954 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:51.274906 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:32:51.275049 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:51.275028 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pg9xz" podUID="cfcad9fb-77bc-460d-9ee0-fbfb5573ba07" Jul 13 18:32:51.378884 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:51.378846 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:32:52.275159 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:52.275128 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:32:52.275585 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:52.275259 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r24p4" podUID="241bb069-48c9-418e-9ef8-e2d8ffae0114" Jul 13 18:32:53.154262 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:53.154069 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:32:53.253305 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:53.253276 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:32:53.274650 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:53.274621 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:32:53.274788 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:53.274655 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:32:53.274788 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:53.274745 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hvwvw" podUID="63f8372e-78ca-4580-8d1e-a57593c38dac" Jul 13 18:32:53.274884 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:53.274866 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pg9xz" podUID="cfcad9fb-77bc-460d-9ee0-fbfb5573ba07" Jul 13 18:32:53.490364 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:53.490328 2582 generic.go:356] "Generic (PLEG): container finished" podID="c402e9df-0605-412f-89d2-7aeaa8cb1277" containerID="6671f5a2e10059e6710678794af91cc893db23cd188d0df6fba829e4e4403d93" exitCode=0 Jul 13 18:32:53.491237 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:53.490412 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f7lnn" event={"ID":"c402e9df-0605-412f-89d2-7aeaa8cb1277","Type":"ContainerDied","Data":"6671f5a2e10059e6710678794af91cc893db23cd188d0df6fba829e4e4403d93"} Jul 13 18:32:53.493847 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:53.493831 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8l7w6_b14ef695-c589-4fca-bd12-2e951f04e466/ovn-acl-logging/0.log" Jul 13 18:32:53.494169 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:53.494150 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" event={"ID":"b14ef695-c589-4fca-bd12-2e951f04e466","Type":"ContainerStarted","Data":"bfa77e0c8a40fd70d20d2b666caad6e72d200aea91e349f5898dcaf7172b3583"} Jul 13 18:32:53.494413 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:53.494399 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:53.494473 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:53.494449 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:53.494565 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:53.494551 2582 scope.go:117] "RemoveContainer" containerID="4a0c469fe573e18f36173ca041589204d739f571263792680ad0a2607558519f" Jul 13 18:32:53.510769 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:53.510747 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:53.564512 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:53.564487 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:32:53.650961 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:53.650927 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:32:53.830775 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:53.830689 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:32:53.960953 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:53.960910 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:32:54.053914 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:54.053882 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:32:54.276448 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:54.276414 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:32:54.276675 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:54.276529 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r24p4" podUID="241bb069-48c9-418e-9ef8-e2d8ffae0114" Jul 13 18:32:54.500036 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:54.500003 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8l7w6_b14ef695-c589-4fca-bd12-2e951f04e466/ovn-acl-logging/0.log" Jul 13 18:32:54.500402 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:54.500329 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" event={"ID":"b14ef695-c589-4fca-bd12-2e951f04e466","Type":"ContainerStarted","Data":"7df77a4c175a50748f7ac9c797429e620f2d978091bec152481258aef06e42c2"} Jul 13 18:32:54.500677 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:54.500658 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:54.515920 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:54.515898 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:32:54.536964 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:54.536899 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" podStartSLOduration=9.290381129 podStartE2EDuration="30.53688392s" totalImagesPullingTime="21.246502791s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:32:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:32:54.535089305 +0000 UTC m=+30.818517907" watchObservedRunningTime="2026-07-13 18:32:54.53688392 +0000 UTC m=+30.820312522" Jul 13 18:32:55.245115 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:55.245085 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pg9xz"] Jul 13 18:32:55.245280 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:55.245191 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:32:55.245333 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:55.245281 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pg9xz" podUID="cfcad9fb-77bc-460d-9ee0-fbfb5573ba07" Jul 13 18:32:55.248298 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:55.248274 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hvwvw"] Jul 13 18:32:55.248420 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:55.248361 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:32:55.248468 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:55.248431 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hvwvw" podUID="63f8372e-78ca-4580-8d1e-a57593c38dac" Jul 13 18:32:55.264627 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:55.264577 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r24p4"] Jul 13 18:32:55.264721 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:55.264695 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:32:55.264799 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:55.264774 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r24p4" podUID="241bb069-48c9-418e-9ef8-e2d8ffae0114" Jul 13 18:32:55.433617 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:55.433519 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:32:55.504166 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:55.504134 2582 generic.go:356] "Generic (PLEG): container finished" podID="c402e9df-0605-412f-89d2-7aeaa8cb1277" containerID="8ad995fadccf5e3777652b14b790b749102e10d594b9d646acb8da08b7b768cc" exitCode=0 Jul 13 18:32:55.504518 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:55.504214 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f7lnn" event={"ID":"c402e9df-0605-412f-89d2-7aeaa8cb1277","Type":"ContainerDied","Data":"8ad995fadccf5e3777652b14b790b749102e10d594b9d646acb8da08b7b768cc"} Jul 13 18:32:56.112142 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:56.112117 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nt9r9_36baa7e8-9898-491d-a961-c364c6e6af25/dns-node-resolver/0.log" Jul 13 18:32:56.509574 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:56.509417 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f7lnn" event={"ID":"c402e9df-0605-412f-89d2-7aeaa8cb1277","Type":"ContainerStarted","Data":"fb2455f15e368e8dcd9c7d1bc3b537411b3cbb0783726bcd80a07ab5ce88e6c1"} Jul 13 18:32:56.696217 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:56.696187 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:32:56.842463 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:56.842382 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnwrl\" (UniqueName: \"kubernetes.io/projected/241bb069-48c9-418e-9ef8-e2d8ffae0114-kube-api-access-tnwrl\") pod \"network-check-target-r24p4\" (UID: \"241bb069-48c9-418e-9ef8-e2d8ffae0114\") " pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:32:56.842589 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:56.842546 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 18:32:56.842589 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:56.842567 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 18:32:56.842589 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:56.842577 2582 projected.go:196] Error preparing data for projected volume kube-api-access-tnwrl for pod openshift-network-diagnostics/network-check-target-r24p4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:32:56.842715 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:56.842639 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/241bb069-48c9-418e-9ef8-e2d8ffae0114-kube-api-access-tnwrl podName:241bb069-48c9-418e-9ef8-e2d8ffae0114 nodeName:}" failed. No retries permitted until 2026-07-13 18:33:28.842625706 +0000 UTC m=+65.126054287 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-tnwrl" (UniqueName: "kubernetes.io/projected/241bb069-48c9-418e-9ef8-e2d8ffae0114-kube-api-access-tnwrl") pod "network-check-target-r24p4" (UID: "241bb069-48c9-418e-9ef8-e2d8ffae0114") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:32:56.911269 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:56.911244 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5qgnw_5fee7e65-59ed-4225-b607-67c9a5218be4/node-ca/0.log" Jul 13 18:32:56.943505 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:56.943480 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63f8372e-78ca-4580-8d1e-a57593c38dac-metrics-certs\") pod \"network-metrics-daemon-hvwvw\" (UID: \"63f8372e-78ca-4580-8d1e-a57593c38dac\") " pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:32:56.943659 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:56.943528 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cfcad9fb-77bc-460d-9ee0-fbfb5573ba07-original-pull-secret\") pod \"global-pull-secret-syncer-pg9xz\" (UID: \"cfcad9fb-77bc-460d-9ee0-fbfb5573ba07\") " pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:32:56.943659 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:56.943652 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 18:32:56.943775 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:56.943656 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:32:56.943775 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:56.943701 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfcad9fb-77bc-460d-9ee0-fbfb5573ba07-original-pull-secret podName:cfcad9fb-77bc-460d-9ee0-fbfb5573ba07 nodeName:}" failed. No retries permitted until 2026-07-13 18:33:28.943687178 +0000 UTC m=+65.227115758 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cfcad9fb-77bc-460d-9ee0-fbfb5573ba07-original-pull-secret") pod "global-pull-secret-syncer-pg9xz" (UID: "cfcad9fb-77bc-460d-9ee0-fbfb5573ba07") : object "kube-system"/"original-pull-secret" not registered Jul 13 18:32:56.943775 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:56.943718 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63f8372e-78ca-4580-8d1e-a57593c38dac-metrics-certs podName:63f8372e-78ca-4580-8d1e-a57593c38dac nodeName:}" failed. No retries permitted until 2026-07-13 18:33:28.943710121 +0000 UTC m=+65.227138702 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63f8372e-78ca-4580-8d1e-a57593c38dac-metrics-certs") pod "network-metrics-daemon-hvwvw" (UID: "63f8372e-78ca-4580-8d1e-a57593c38dac") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:32:57.274547 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:57.274510 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:32:57.274797 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:57.274618 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:32:57.274797 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:57.274627 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r24p4" podUID="241bb069-48c9-418e-9ef8-e2d8ffae0114" Jul 13 18:32:57.274797 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:57.274623 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:32:57.274797 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:57.274703 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pg9xz" podUID="cfcad9fb-77bc-460d-9ee0-fbfb5573ba07" Jul 13 18:32:57.274797 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:57.274788 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hvwvw" podUID="63f8372e-78ca-4580-8d1e-a57593c38dac" Jul 13 18:32:57.513752 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:57.513717 2582 generic.go:356] "Generic (PLEG): container finished" podID="c402e9df-0605-412f-89d2-7aeaa8cb1277" containerID="fb2455f15e368e8dcd9c7d1bc3b537411b3cbb0783726bcd80a07ab5ce88e6c1" exitCode=0 Jul 13 18:32:57.514240 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:57.513768 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f7lnn" event={"ID":"c402e9df-0605-412f-89d2-7aeaa8cb1277","Type":"ContainerDied","Data":"fb2455f15e368e8dcd9c7d1bc3b537411b3cbb0783726bcd80a07ab5ce88e6c1"} Jul 13 18:32:57.850078 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:57.850047 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:32:57.952074 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:57.952044 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:32:58.052983 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:58.052956 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:32:59.275070 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:59.275042 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:32:59.275535 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:59.275108 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:32:59.275535 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:59.275161 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r24p4" podUID="241bb069-48c9-418e-9ef8-e2d8ffae0114" Jul 13 18:32:59.275535 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:59.275222 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pg9xz" podUID="cfcad9fb-77bc-460d-9ee0-fbfb5573ba07" Jul 13 18:32:59.275535 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:59.275273 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:32:59.275535 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:32:59.275356 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hvwvw" podUID="63f8372e-78ca-4580-8d1e-a57593c38dac" Jul 13 18:32:59.923621 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:32:59.923560 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:00.001326 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:00.001293 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:00.374240 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:00.374148 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:00.613395 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:00.613358 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:00.955036 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:00.954999 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:01.156001 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:01.155965 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:01.275036 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:01.274956 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:33:01.275036 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:01.274995 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:33:01.275250 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:01.274956 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:33:01.275250 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:33:01.275083 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hvwvw" podUID="63f8372e-78ca-4580-8d1e-a57593c38dac" Jul 13 18:33:01.275250 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:33:01.275160 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pg9xz" podUID="cfcad9fb-77bc-460d-9ee0-fbfb5573ba07" Jul 13 18:33:01.275250 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:33:01.275230 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r24p4" podUID="241bb069-48c9-418e-9ef8-e2d8ffae0114" Jul 13 18:33:01.556628 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:01.556531 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:01.761771 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:01.761735 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:01.858249 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:01.858169 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:02.149260 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:02.149232 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:02.528043 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:02.528014 2582 generic.go:356] "Generic (PLEG): container finished" podID="c402e9df-0605-412f-89d2-7aeaa8cb1277" containerID="668eae3b6b36e2af01b5f28de098738bd6232085f9e4afeffed11ffe97d16a30" exitCode=0 Jul 13 18:33:02.528197 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:02.528079 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f7lnn" event={"ID":"c402e9df-0605-412f-89d2-7aeaa8cb1277","Type":"ContainerDied","Data":"668eae3b6b36e2af01b5f28de098738bd6232085f9e4afeffed11ffe97d16a30"} Jul 13 18:33:02.700409 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:02.700381 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:02.797033 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:02.796952 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:03.162741 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:03.162650 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:03.274782 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:03.274749 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:33:03.274920 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:03.274749 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:33:03.274920 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:33:03.274859 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r24p4" podUID="241bb069-48c9-418e-9ef8-e2d8ffae0114" Jul 13 18:33:03.275005 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:03.274750 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:33:03.275035 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:33:03.274924 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pg9xz" podUID="cfcad9fb-77bc-460d-9ee0-fbfb5573ba07" Jul 13 18:33:03.275035 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:33:03.275006 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hvwvw" podUID="63f8372e-78ca-4580-8d1e-a57593c38dac" Jul 13 18:33:03.532658 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:03.532630 2582 generic.go:356] "Generic (PLEG): container finished" podID="c402e9df-0605-412f-89d2-7aeaa8cb1277" containerID="4071d074a3f99a0599f957186e0b4ba317e191ac7dc9d8ba8b46b02d81542e87" exitCode=0 Jul 13 18:33:03.532805 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:03.532688 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f7lnn" event={"ID":"c402e9df-0605-412f-89d2-7aeaa8cb1277","Type":"ContainerDied","Data":"4071d074a3f99a0599f957186e0b4ba317e191ac7dc9d8ba8b46b02d81542e87"} Jul 13 18:33:03.719416 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:03.719377 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:03.819419 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:03.819326 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:03.981129 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:03.981089 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:04.103098 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:04.102990 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:04.204024 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:04.203979 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:04.368548 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:04.368458 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:04.538731 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:04.538697 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f7lnn" event={"ID":"c402e9df-0605-412f-89d2-7aeaa8cb1277","Type":"ContainerStarted","Data":"5d9c17ebe60c5a9725ef246b731cf5f75a3d5b331d910bf0644861c9ff61cc6c"} Jul 13 18:33:04.760060 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:04.760028 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:04.898671 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:04.898571 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:04.989477 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:04.989446 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:05.257315 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:05.257282 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:05.275226 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:05.275184 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:33:05.275317 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:05.275237 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:33:05.275317 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:05.275237 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:33:05.275317 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:33:05.275306 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r24p4" podUID="241bb069-48c9-418e-9ef8-e2d8ffae0114" Jul 13 18:33:05.275422 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:33:05.275398 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hvwvw" podUID="63f8372e-78ca-4580-8d1e-a57593c38dac" Jul 13 18:33:05.275491 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:33:05.275473 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pg9xz" podUID="cfcad9fb-77bc-460d-9ee0-fbfb5573ba07" Jul 13 18:33:05.759177 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:05.759131 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:06.405638 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:06.405575 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:06.581429 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:06.581382 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:06.848912 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:06.848880 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:07.163402 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:07.163314 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:07.275440 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:07.275409 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:33:07.275632 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:07.275495 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:33:07.275632 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:33:07.275497 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r24p4" podUID="241bb069-48c9-418e-9ef8-e2d8ffae0114" Jul 13 18:33:07.275632 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:07.275525 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:33:07.275805 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:33:07.275639 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hvwvw" podUID="63f8372e-78ca-4580-8d1e-a57593c38dac" Jul 13 18:33:07.275805 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:33:07.275689 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pg9xz" podUID="cfcad9fb-77bc-460d-9ee0-fbfb5573ba07" Jul 13 18:33:07.427531 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:07.427453 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:07.981941 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:07.981888 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:08.188681 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:08.188654 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:08.381350 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:08.381250 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:08.656268 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:08.656182 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:09.275438 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:09.275403 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:33:09.275631 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:09.275515 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:33:09.275631 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:33:09.275516 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r24p4" podUID="241bb069-48c9-418e-9ef8-e2d8ffae0114" Jul 13 18:33:09.275631 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:09.275538 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:33:09.275631 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:33:09.275627 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hvwvw" podUID="63f8372e-78ca-4580-8d1e-a57593c38dac" Jul 13 18:33:09.275870 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:33:09.275690 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pg9xz" podUID="cfcad9fb-77bc-460d-9ee0-fbfb5573ba07" Jul 13 18:33:11.274966 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:11.274935 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:33:11.274966 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:11.274950 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:33:11.275465 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:11.274950 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:33:11.275465 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:33:11.275032 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r24p4" podUID="241bb069-48c9-418e-9ef8-e2d8ffae0114" Jul 13 18:33:11.275465 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:33:11.275129 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hvwvw" podUID="63f8372e-78ca-4580-8d1e-a57593c38dac" Jul 13 18:33:11.275465 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:33:11.275197 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pg9xz" podUID="cfcad9fb-77bc-460d-9ee0-fbfb5573ba07" Jul 13 18:33:12.862339 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:12.862295 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:12.972852 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:12.972818 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:13.274804 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:13.274774 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:33:13.274996 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:13.274861 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:33:13.274996 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:33:13.274867 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r24p4" podUID="241bb069-48c9-418e-9ef8-e2d8ffae0114" Jul 13 18:33:13.274996 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:33:13.274956 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pg9xz" podUID="cfcad9fb-77bc-460d-9ee0-fbfb5573ba07" Jul 13 18:33:13.275161 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:13.275000 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:33:13.275161 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:33:13.275092 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hvwvw" podUID="63f8372e-78ca-4580-8d1e-a57593c38dac" Jul 13 18:33:15.274738 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:15.274699 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:33:15.274738 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:15.274723 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:33:15.274738 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:15.274725 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:33:15.275231 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:33:15.274797 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r24p4" podUID="241bb069-48c9-418e-9ef8-e2d8ffae0114" Jul 13 18:33:15.275231 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:33:15.274893 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pg9xz" podUID="cfcad9fb-77bc-460d-9ee0-fbfb5573ba07" Jul 13 18:33:15.275231 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:33:15.274932 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hvwvw" podUID="63f8372e-78ca-4580-8d1e-a57593c38dac" Jul 13 18:33:16.356064 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:16.356027 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:16.575065 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:16.575027 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:16.716016 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:16.715975 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:16.848931 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:16.848895 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:17.275059 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.275026 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:33:17.275216 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.275028 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:33:17.275216 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:33:17.275124 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r24p4" podUID="241bb069-48c9-418e-9ef8-e2d8ffae0114" Jul 13 18:33:17.275216 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.275028 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:33:17.275216 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:33:17.275204 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pg9xz" podUID="cfcad9fb-77bc-460d-9ee0-fbfb5573ba07" Jul 13 18:33:17.275381 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:33:17.275271 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hvwvw" podUID="63f8372e-78ca-4580-8d1e-a57593c38dac" Jul 13 18:33:17.592626 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.592577 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-191.ec2.internal" event="NodeReady" Jul 13 18:33:17.593000 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.592723 2582 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 13 18:33:17.684895 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.684846 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-f7lnn" podStartSLOduration=21.801933347 podStartE2EDuration="53.684828616s" totalImagesPullingTime="31.882895269s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:32:24 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:33:04.576409536 +0000 UTC m=+40.859838137" watchObservedRunningTime="2026-07-13 18:33:17.684828616 +0000 UTC m=+53.968257217" Jul 13 18:33:17.685076 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.685021 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-v5z9m"] Jul 13 18:33:17.707031 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.707002 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zc5rl"] Jul 13 18:33:17.707173 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.707135 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-v5z9m" Jul 13 18:33:17.712818 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.712788 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 13 18:33:17.713001 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.712852 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tnbnw\"" Jul 13 18:33:17.713001 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.712943 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 13 18:33:17.723240 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.723219 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-v5z9m"] Jul 13 18:33:17.723351 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.723252 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zc5rl"] Jul 13 18:33:17.723351 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.723332 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zc5rl" Jul 13 18:33:17.726319 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.726300 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 13 18:33:17.726456 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.726303 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 13 18:33:17.726456 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.726386 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 13 18:33:17.726456 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.726420 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vfssh\"" Jul 13 18:33:17.726734 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.726718 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 13 18:33:17.770913 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.770878 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-vl6xp"] Jul 13 18:33:17.791701 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.791672 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vl6xp"] Jul 13 18:33:17.791853 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.791773 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vl6xp" Jul 13 18:33:17.796529 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.796510 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 13 18:33:17.796975 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.796955 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 13 18:33:17.797101 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.797030 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 13 18:33:17.797158 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.797140 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-phpw8\"" Jul 13 18:33:17.803452 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.803433 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a8906f23-73c2-4ee7-9202-ec4564ef44d3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zc5rl\" (UID: \"a8906f23-73c2-4ee7-9202-ec4564ef44d3\") " pod="openshift-insights/insights-runtime-extractor-zc5rl" Jul 13 18:33:17.803564 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.803467 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvcds\" (UniqueName: \"kubernetes.io/projected/8fcf3f48-d491-4064-a7b6-b860ab39e829-kube-api-access-hvcds\") pod \"dns-default-v5z9m\" (UID: \"8fcf3f48-d491-4064-a7b6-b860ab39e829\") " pod="openshift-dns/dns-default-v5z9m" Jul 13 18:33:17.803564 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.803486 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8fcf3f48-d491-4064-a7b6-b860ab39e829-config-volume\") pod \"dns-default-v5z9m\" (UID: \"8fcf3f48-d491-4064-a7b6-b860ab39e829\") " pod="openshift-dns/dns-default-v5z9m" Jul 13 18:33:17.803564 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.803502 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a8906f23-73c2-4ee7-9202-ec4564ef44d3-data-volume\") pod \"insights-runtime-extractor-zc5rl\" (UID: \"a8906f23-73c2-4ee7-9202-ec4564ef44d3\") " pod="openshift-insights/insights-runtime-extractor-zc5rl" Jul 13 18:33:17.803564 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.803549 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a8906f23-73c2-4ee7-9202-ec4564ef44d3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zc5rl\" (UID: \"a8906f23-73c2-4ee7-9202-ec4564ef44d3\") " pod="openshift-insights/insights-runtime-extractor-zc5rl" Jul 13 18:33:17.803721 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.803586 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8fcf3f48-d491-4064-a7b6-b860ab39e829-metrics-tls\") pod \"dns-default-v5z9m\" (UID: \"8fcf3f48-d491-4064-a7b6-b860ab39e829\") " pod="openshift-dns/dns-default-v5z9m" Jul 13 18:33:17.803721 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.803645 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a8906f23-73c2-4ee7-9202-ec4564ef44d3-crio-socket\") pod \"insights-runtime-extractor-zc5rl\" (UID: \"a8906f23-73c2-4ee7-9202-ec4564ef44d3\") " pod="openshift-insights/insights-runtime-extractor-zc5rl" Jul 13 18:33:17.803721 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.803662 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkvqn\" (UniqueName: \"kubernetes.io/projected/a8906f23-73c2-4ee7-9202-ec4564ef44d3-kube-api-access-mkvqn\") pod \"insights-runtime-extractor-zc5rl\" (UID: \"a8906f23-73c2-4ee7-9202-ec4564ef44d3\") " pod="openshift-insights/insights-runtime-extractor-zc5rl" Jul 13 18:33:17.803721 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.803686 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fcf3f48-d491-4064-a7b6-b860ab39e829-tmp-dir\") pod \"dns-default-v5z9m\" (UID: \"8fcf3f48-d491-4064-a7b6-b860ab39e829\") " pod="openshift-dns/dns-default-v5z9m" Jul 13 18:33:17.868806 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.868710 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:17.904014 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.903980 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8fcf3f48-d491-4064-a7b6-b860ab39e829-metrics-tls\") pod \"dns-default-v5z9m\" (UID: \"8fcf3f48-d491-4064-a7b6-b860ab39e829\") " pod="openshift-dns/dns-default-v5z9m" Jul 13 18:33:17.904168 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.904023 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a8906f23-73c2-4ee7-9202-ec4564ef44d3-crio-socket\") pod \"insights-runtime-extractor-zc5rl\" (UID: \"a8906f23-73c2-4ee7-9202-ec4564ef44d3\") " pod="openshift-insights/insights-runtime-extractor-zc5rl" Jul 13 18:33:17.904168 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.904042 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkvqn\" (UniqueName: \"kubernetes.io/projected/a8906f23-73c2-4ee7-9202-ec4564ef44d3-kube-api-access-mkvqn\") pod \"insights-runtime-extractor-zc5rl\" (UID: \"a8906f23-73c2-4ee7-9202-ec4564ef44d3\") " pod="openshift-insights/insights-runtime-extractor-zc5rl" Jul 13 18:33:17.904168 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.904161 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fcf3f48-d491-4064-a7b6-b860ab39e829-tmp-dir\") pod \"dns-default-v5z9m\" (UID: \"8fcf3f48-d491-4064-a7b6-b860ab39e829\") " pod="openshift-dns/dns-default-v5z9m" Jul 13 18:33:17.904308 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.904221 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a8906f23-73c2-4ee7-9202-ec4564ef44d3-crio-socket\") pod \"insights-runtime-extractor-zc5rl\" (UID: \"a8906f23-73c2-4ee7-9202-ec4564ef44d3\") " pod="openshift-insights/insights-runtime-extractor-zc5rl" Jul 13 18:33:17.904308 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.904221 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a8906f23-73c2-4ee7-9202-ec4564ef44d3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zc5rl\" (UID: \"a8906f23-73c2-4ee7-9202-ec4564ef44d3\") " pod="openshift-insights/insights-runtime-extractor-zc5rl" Jul 13 18:33:17.904308 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.904298 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvcds\" (UniqueName: \"kubernetes.io/projected/8fcf3f48-d491-4064-a7b6-b860ab39e829-kube-api-access-hvcds\") pod \"dns-default-v5z9m\" (UID: \"8fcf3f48-d491-4064-a7b6-b860ab39e829\") " pod="openshift-dns/dns-default-v5z9m" Jul 13 18:33:17.904442 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.904327 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8fcf3f48-d491-4064-a7b6-b860ab39e829-config-volume\") pod \"dns-default-v5z9m\" (UID: \"8fcf3f48-d491-4064-a7b6-b860ab39e829\") " pod="openshift-dns/dns-default-v5z9m" Jul 13 18:33:17.904442 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.904347 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a8906f23-73c2-4ee7-9202-ec4564ef44d3-data-volume\") pod \"insights-runtime-extractor-zc5rl\" (UID: \"a8906f23-73c2-4ee7-9202-ec4564ef44d3\") " pod="openshift-insights/insights-runtime-extractor-zc5rl" Jul 13 18:33:17.904442 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.904369 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/051cc321-d24b-4f45-b2e5-477428647fe3-cert\") pod \"ingress-canary-vl6xp\" (UID: \"051cc321-d24b-4f45-b2e5-477428647fe3\") " pod="openshift-ingress-canary/ingress-canary-vl6xp" Jul 13 18:33:17.904585 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.904522 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a8906f23-73c2-4ee7-9202-ec4564ef44d3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zc5rl\" (UID: \"a8906f23-73c2-4ee7-9202-ec4564ef44d3\") " pod="openshift-insights/insights-runtime-extractor-zc5rl" Jul 13 18:33:17.904668 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.904586 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvr9z\" (UniqueName: \"kubernetes.io/projected/051cc321-d24b-4f45-b2e5-477428647fe3-kube-api-access-gvr9z\") pod \"ingress-canary-vl6xp\" (UID: \"051cc321-d24b-4f45-b2e5-477428647fe3\") " pod="openshift-ingress-canary/ingress-canary-vl6xp" Jul 13 18:33:17.904742 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.904721 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a8906f23-73c2-4ee7-9202-ec4564ef44d3-data-volume\") pod \"insights-runtime-extractor-zc5rl\" (UID: \"a8906f23-73c2-4ee7-9202-ec4564ef44d3\") " pod="openshift-insights/insights-runtime-extractor-zc5rl" Jul 13 18:33:17.904898 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.904874 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fcf3f48-d491-4064-a7b6-b860ab39e829-tmp-dir\") pod \"dns-default-v5z9m\" (UID: \"8fcf3f48-d491-4064-a7b6-b860ab39e829\") " pod="openshift-dns/dns-default-v5z9m" Jul 13 18:33:17.905005 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.904904 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a8906f23-73c2-4ee7-9202-ec4564ef44d3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zc5rl\" (UID: \"a8906f23-73c2-4ee7-9202-ec4564ef44d3\") " pod="openshift-insights/insights-runtime-extractor-zc5rl" Jul 13 18:33:17.905005 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.904954 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8fcf3f48-d491-4064-a7b6-b860ab39e829-config-volume\") pod \"dns-default-v5z9m\" (UID: \"8fcf3f48-d491-4064-a7b6-b860ab39e829\") " pod="openshift-dns/dns-default-v5z9m" Jul 13 18:33:17.908395 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.908375 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a8906f23-73c2-4ee7-9202-ec4564ef44d3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zc5rl\" (UID: \"a8906f23-73c2-4ee7-9202-ec4564ef44d3\") " pod="openshift-insights/insights-runtime-extractor-zc5rl" Jul 13 18:33:17.908509 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.908404 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8fcf3f48-d491-4064-a7b6-b860ab39e829-metrics-tls\") pod \"dns-default-v5z9m\" (UID: \"8fcf3f48-d491-4064-a7b6-b860ab39e829\") " pod="openshift-dns/dns-default-v5z9m" Jul 13 18:33:17.914563 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.914541 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkvqn\" (UniqueName: \"kubernetes.io/projected/a8906f23-73c2-4ee7-9202-ec4564ef44d3-kube-api-access-mkvqn\") pod \"insights-runtime-extractor-zc5rl\" (UID: \"a8906f23-73c2-4ee7-9202-ec4564ef44d3\") " pod="openshift-insights/insights-runtime-extractor-zc5rl" Jul 13 18:33:17.914706 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:17.914686 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvcds\" (UniqueName: \"kubernetes.io/projected/8fcf3f48-d491-4064-a7b6-b860ab39e829-kube-api-access-hvcds\") pod \"dns-default-v5z9m\" (UID: \"8fcf3f48-d491-4064-a7b6-b860ab39e829\") " pod="openshift-dns/dns-default-v5z9m" Jul 13 18:33:18.005977 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:18.005932 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/051cc321-d24b-4f45-b2e5-477428647fe3-cert\") pod \"ingress-canary-vl6xp\" (UID: \"051cc321-d24b-4f45-b2e5-477428647fe3\") " pod="openshift-ingress-canary/ingress-canary-vl6xp" Jul 13 18:33:18.005977 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:18.005983 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvr9z\" (UniqueName: \"kubernetes.io/projected/051cc321-d24b-4f45-b2e5-477428647fe3-kube-api-access-gvr9z\") pod \"ingress-canary-vl6xp\" (UID: \"051cc321-d24b-4f45-b2e5-477428647fe3\") " pod="openshift-ingress-canary/ingress-canary-vl6xp" Jul 13 18:33:18.008458 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:18.008428 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/051cc321-d24b-4f45-b2e5-477428647fe3-cert\") pod \"ingress-canary-vl6xp\" (UID: \"051cc321-d24b-4f45-b2e5-477428647fe3\") " pod="openshift-ingress-canary/ingress-canary-vl6xp" Jul 13 18:33:18.015432 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:18.015409 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvr9z\" (UniqueName: \"kubernetes.io/projected/051cc321-d24b-4f45-b2e5-477428647fe3-kube-api-access-gvr9z\") pod \"ingress-canary-vl6xp\" (UID: \"051cc321-d24b-4f45-b2e5-477428647fe3\") " pod="openshift-ingress-canary/ingress-canary-vl6xp" Jul 13 18:33:18.018268 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:18.018247 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-v5z9m" Jul 13 18:33:18.031403 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:18.031374 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zc5rl" Jul 13 18:33:18.036038 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:18.036012 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:18.101443 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:18.101414 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vl6xp" Jul 13 18:33:18.248508 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:18.248189 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-v5z9m"] Jul 13 18:33:18.252046 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:33:18.252019 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fcf3f48_d491_4064_a7b6_b860ab39e829.slice/crio-326fa1409c54f078baf521f0e57cc0d977b1baa6eae393130f241c11c07d60bb WatchSource:0}: Error finding container 326fa1409c54f078baf521f0e57cc0d977b1baa6eae393130f241c11c07d60bb: Status 404 returned error can't find the container with id 326fa1409c54f078baf521f0e57cc0d977b1baa6eae393130f241c11c07d60bb Jul 13 18:33:18.255969 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:18.255947 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zc5rl"] Jul 13 18:33:18.260192 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:33:18.260156 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8906f23_73c2_4ee7_9202_ec4564ef44d3.slice/crio-389b76901dd68fcf3b07adc40c03da304cbb955c70f770080b6453a26b4d98a5 WatchSource:0}: Error finding container 389b76901dd68fcf3b07adc40c03da304cbb955c70f770080b6453a26b4d98a5: Status 404 returned error can't find the container with id 389b76901dd68fcf3b07adc40c03da304cbb955c70f770080b6453a26b4d98a5 Jul 13 18:33:18.290100 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:18.290075 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vl6xp"] Jul 13 18:33:18.292810 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:33:18.292785 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod051cc321_d24b_4f45_b2e5_477428647fe3.slice/crio-68f38e218b7a73b1bf1f626f25e297fcd3f2165f9692ce89130ab1d7ab6aa4e9 WatchSource:0}: Error finding container 68f38e218b7a73b1bf1f626f25e297fcd3f2165f9692ce89130ab1d7ab6aa4e9: Status 404 returned error can't find the container with id 68f38e218b7a73b1bf1f626f25e297fcd3f2165f9692ce89130ab1d7ab6aa4e9 Jul 13 18:33:18.568505 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:18.568470 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v5z9m" event={"ID":"8fcf3f48-d491-4064-a7b6-b860ab39e829","Type":"ContainerStarted","Data":"326fa1409c54f078baf521f0e57cc0d977b1baa6eae393130f241c11c07d60bb"} Jul 13 18:33:18.569541 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:18.569517 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zc5rl" event={"ID":"a8906f23-73c2-4ee7-9202-ec4564ef44d3","Type":"ContainerStarted","Data":"389b76901dd68fcf3b07adc40c03da304cbb955c70f770080b6453a26b4d98a5"} Jul 13 18:33:18.570463 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:18.570439 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vl6xp" event={"ID":"051cc321-d24b-4f45-b2e5-477428647fe3","Type":"ContainerStarted","Data":"68f38e218b7a73b1bf1f626f25e297fcd3f2165f9692ce89130ab1d7ab6aa4e9"} Jul 13 18:33:19.274773 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:19.274700 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:33:19.274773 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:19.274717 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:33:19.275302 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:19.274700 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:33:19.279086 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:19.279058 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 13 18:33:19.279225 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:19.279088 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 13 18:33:19.279225 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:19.279109 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bwd4z\"" Jul 13 18:33:19.279225 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:19.279205 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 13 18:33:19.279420 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:19.279254 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ql2kw\"" Jul 13 18:33:19.279420 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:19.279345 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 13 18:33:21.581705 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:21.581588 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vl6xp" event={"ID":"051cc321-d24b-4f45-b2e5-477428647fe3","Type":"ContainerStarted","Data":"77e4651b118a85e759478756bdc0e03e8d41e2e2fa398980e461f2da88e7fcdb"} Jul 13 18:33:21.584433 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:21.584295 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v5z9m" event={"ID":"8fcf3f48-d491-4064-a7b6-b860ab39e829","Type":"ContainerStarted","Data":"4910dae2ff567e1632da0d90efdd54a6aac1c6e259f72bc50365a12a30aabce2"} Jul 13 18:33:21.584433 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:21.584368 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v5z9m" event={"ID":"8fcf3f48-d491-4064-a7b6-b860ab39e829","Type":"ContainerStarted","Data":"db377188a6075667038eb2cc5721d4d1eb558d85f13dd59b2902828ad08fda52"} Jul 13 18:33:21.584756 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:21.584716 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-v5z9m" Jul 13 18:33:21.633625 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:21.633553 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-vl6xp" podStartSLOduration=1.7088131560000002 podStartE2EDuration="4.633532301s" totalImagesPullingTime="2.924719145s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:33:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:33:21.603688544 +0000 UTC m=+57.887117146" watchObservedRunningTime="2026-07-13 18:33:21.633532301 +0000 UTC m=+57.916960881" Jul 13 18:33:23.100987 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:23.100953 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:23.200122 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:23.200087 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:23.430647 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:23.430520 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:23.530897 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:23.530864 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:23.610342 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:23.610274 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-v5z9m" podStartSLOduration=3.64533406 podStartE2EDuration="6.61025526s" totalImagesPullingTime="2.9649212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:33:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:33:21.633792641 +0000 UTC m=+57.917221242" watchObservedRunningTime="2026-07-13 18:33:23.61025526 +0000 UTC m=+59.893683864" Jul 13 18:33:23.611191 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:23.611168 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bzv49"] Jul 13 18:33:23.612164 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:23.612135 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:23.628562 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:23.628542 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bzv49"] Jul 13 18:33:23.628692 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:23.628645 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bzv49" Jul 13 18:33:23.631309 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:23.631284 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 13 18:33:23.631398 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:23.631360 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-rkvtc\"" Jul 13 18:33:23.657473 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:23.657433 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f3baa39d-9895-4c3e-afe0-e3b1633cd340-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-bzv49\" (UID: \"f3baa39d-9895-4c3e-afe0-e3b1633cd340\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bzv49" Jul 13 18:33:23.758343 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:23.758309 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f3baa39d-9895-4c3e-afe0-e3b1633cd340-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-bzv49\" (UID: \"f3baa39d-9895-4c3e-afe0-e3b1633cd340\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bzv49" Jul 13 18:33:23.758514 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:33:23.758482 2582 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 13 18:33:23.758580 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:33:23.758568 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3baa39d-9895-4c3e-afe0-e3b1633cd340-tls-certificates podName:f3baa39d-9895-4c3e-afe0-e3b1633cd340 nodeName:}" failed. No retries permitted until 2026-07-13 18:33:24.258544715 +0000 UTC m=+60.541973299 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/f3baa39d-9895-4c3e-afe0-e3b1633cd340-tls-certificates") pod "prometheus-operator-admission-webhook-77d98bc786-bzv49" (UID: "f3baa39d-9895-4c3e-afe0-e3b1633cd340") : secret "prometheus-operator-admission-webhook-tls" not found Jul 13 18:33:24.002679 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:24.002639 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:24.263684 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:24.263646 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f3baa39d-9895-4c3e-afe0-e3b1633cd340-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-bzv49\" (UID: \"f3baa39d-9895-4c3e-afe0-e3b1633cd340\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bzv49" Jul 13 18:33:24.266459 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:24.266429 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f3baa39d-9895-4c3e-afe0-e3b1633cd340-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-bzv49\" (UID: \"f3baa39d-9895-4c3e-afe0-e3b1633cd340\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bzv49" Jul 13 18:33:24.541222 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:24.541153 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-rkvtc\"" Jul 13 18:33:24.548797 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:24.548770 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bzv49" Jul 13 18:33:24.690641 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:24.690596 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bzv49"] Jul 13 18:33:24.693893 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:33:24.693867 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3baa39d_9895_4c3e_afe0_e3b1633cd340.slice/crio-17f1e860d2115a11892ead4166e5ec8e41f4de59d9d0b592d09973e4a90d52c6 WatchSource:0}: Error finding container 17f1e860d2115a11892ead4166e5ec8e41f4de59d9d0b592d09973e4a90d52c6: Status 404 returned error can't find the container with id 17f1e860d2115a11892ead4166e5ec8e41f4de59d9d0b592d09973e4a90d52c6 Jul 13 18:33:25.597691 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:25.597648 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zc5rl" event={"ID":"a8906f23-73c2-4ee7-9202-ec4564ef44d3","Type":"ContainerStarted","Data":"a6227795268c80e50b0cf0e2a481200e35f2fbd8cf21113647c8ce5a150ec22a"} Jul 13 18:33:25.599125 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:25.599099 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bzv49" event={"ID":"f3baa39d-9895-4c3e-afe0-e3b1633cd340","Type":"ContainerStarted","Data":"17f1e860d2115a11892ead4166e5ec8e41f4de59d9d0b592d09973e4a90d52c6"} Jul 13 18:33:26.326849 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:26.326813 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:26.474848 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:26.474817 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:26.527203 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:26.527178 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8l7w6" Jul 13 18:33:26.603669 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:26.603567 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bzv49" event={"ID":"f3baa39d-9895-4c3e-afe0-e3b1633cd340","Type":"ContainerStarted","Data":"4b21210fe78cde400634483aedda2f3c91d1d37d52c33b18626c7cd79872f079"} Jul 13 18:33:26.604087 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:26.603797 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bzv49" Jul 13 18:33:26.608314 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:26.608293 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bzv49" Jul 13 18:33:26.628071 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:26.628018 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bzv49" podStartSLOduration=1.984785322 podStartE2EDuration="3.628003845s" totalImagesPullingTime="1.643218523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:33:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:33:26.627499966 +0000 UTC m=+62.910928593" watchObservedRunningTime="2026-07-13 18:33:26.628003845 +0000 UTC m=+62.911432491" Jul 13 18:33:26.702983 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:26.702948 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:26.807152 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:26.807072 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:27.668381 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:27.668343 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:27.715170 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:27.715139 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-qc4ml"] Jul 13 18:33:27.717644 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:27.717627 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-qc4ml" Jul 13 18:33:27.722204 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:27.722178 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 13 18:33:27.722338 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:27.722316 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 13 18:33:27.723656 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:27.723633 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 13 18:33:27.723656 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:27.723646 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 13 18:33:27.723851 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:27.723830 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 13 18:33:27.723851 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:27.723846 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-rgfw7\"" Jul 13 18:33:27.737762 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:27.737740 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-qc4ml"] Jul 13 18:33:27.770986 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:27.770948 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:27.788909 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:27.788884 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4923aacc-d13a-4892-9cde-a2425d9f36b4-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-qc4ml\" (UID: \"4923aacc-d13a-4892-9cde-a2425d9f36b4\") " pod="openshift-monitoring/prometheus-operator-64785b6675-qc4ml" Jul 13 18:33:27.788999 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:27.788924 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-df8k4\" (UniqueName: \"kubernetes.io/projected/4923aacc-d13a-4892-9cde-a2425d9f36b4-kube-api-access-df8k4\") pod \"prometheus-operator-64785b6675-qc4ml\" (UID: \"4923aacc-d13a-4892-9cde-a2425d9f36b4\") " pod="openshift-monitoring/prometheus-operator-64785b6675-qc4ml" Jul 13 18:33:27.789035 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:27.789000 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4923aacc-d13a-4892-9cde-a2425d9f36b4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-qc4ml\" (UID: \"4923aacc-d13a-4892-9cde-a2425d9f36b4\") " pod="openshift-monitoring/prometheus-operator-64785b6675-qc4ml" Jul 13 18:33:27.789035 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:27.789027 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4923aacc-d13a-4892-9cde-a2425d9f36b4-metrics-client-ca\") pod \"prometheus-operator-64785b6675-qc4ml\" (UID: \"4923aacc-d13a-4892-9cde-a2425d9f36b4\") " pod="openshift-monitoring/prometheus-operator-64785b6675-qc4ml" Jul 13 18:33:27.876982 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:27.876922 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:27.889926 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:27.889903 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4923aacc-d13a-4892-9cde-a2425d9f36b4-metrics-client-ca\") pod \"prometheus-operator-64785b6675-qc4ml\" (UID: \"4923aacc-d13a-4892-9cde-a2425d9f36b4\") " pod="openshift-monitoring/prometheus-operator-64785b6675-qc4ml" Jul 13 18:33:27.890029 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:27.889949 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4923aacc-d13a-4892-9cde-a2425d9f36b4-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-qc4ml\" (UID: \"4923aacc-d13a-4892-9cde-a2425d9f36b4\") " pod="openshift-monitoring/prometheus-operator-64785b6675-qc4ml" Jul 13 18:33:27.890029 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:27.889983 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-df8k4\" (UniqueName: \"kubernetes.io/projected/4923aacc-d13a-4892-9cde-a2425d9f36b4-kube-api-access-df8k4\") pod \"prometheus-operator-64785b6675-qc4ml\" (UID: \"4923aacc-d13a-4892-9cde-a2425d9f36b4\") " pod="openshift-monitoring/prometheus-operator-64785b6675-qc4ml" Jul 13 18:33:27.890191 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:27.890169 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4923aacc-d13a-4892-9cde-a2425d9f36b4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-qc4ml\" (UID: \"4923aacc-d13a-4892-9cde-a2425d9f36b4\") " pod="openshift-monitoring/prometheus-operator-64785b6675-qc4ml" Jul 13 18:33:27.890629 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:27.890588 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4923aacc-d13a-4892-9cde-a2425d9f36b4-metrics-client-ca\") pod \"prometheus-operator-64785b6675-qc4ml\" (UID: \"4923aacc-d13a-4892-9cde-a2425d9f36b4\") " pod="openshift-monitoring/prometheus-operator-64785b6675-qc4ml" Jul 13 18:33:27.892419 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:27.892393 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4923aacc-d13a-4892-9cde-a2425d9f36b4-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-qc4ml\" (UID: \"4923aacc-d13a-4892-9cde-a2425d9f36b4\") " pod="openshift-monitoring/prometheus-operator-64785b6675-qc4ml" Jul 13 18:33:27.892506 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:27.892476 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4923aacc-d13a-4892-9cde-a2425d9f36b4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-qc4ml\" (UID: \"4923aacc-d13a-4892-9cde-a2425d9f36b4\") " pod="openshift-monitoring/prometheus-operator-64785b6675-qc4ml" Jul 13 18:33:27.906444 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:27.906419 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-df8k4\" (UniqueName: \"kubernetes.io/projected/4923aacc-d13a-4892-9cde-a2425d9f36b4-kube-api-access-df8k4\") pod \"prometheus-operator-64785b6675-qc4ml\" (UID: \"4923aacc-d13a-4892-9cde-a2425d9f36b4\") " pod="openshift-monitoring/prometheus-operator-64785b6675-qc4ml" Jul 13 18:33:28.026325 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:28.026281 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-qc4ml" Jul 13 18:33:28.097680 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:28.097624 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:28.153566 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:28.153536 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-qc4ml"] Jul 13 18:33:28.156765 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:33:28.156737 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4923aacc_d13a_4892_9cde_a2425d9f36b4.slice/crio-906f7f5874223417297e586ed860f3abf46de6cadd380c8bf9614594ea0a35ea WatchSource:0}: Error finding container 906f7f5874223417297e586ed860f3abf46de6cadd380c8bf9614594ea0a35ea: Status 404 returned error can't find the container with id 906f7f5874223417297e586ed860f3abf46de6cadd380c8bf9614594ea0a35ea Jul 13 18:33:28.198903 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:28.198858 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:28.611170 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:28.611137 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-qc4ml" event={"ID":"4923aacc-d13a-4892-9cde-a2425d9f36b4","Type":"ContainerStarted","Data":"906f7f5874223417297e586ed860f3abf46de6cadd380c8bf9614594ea0a35ea"} Jul 13 18:33:28.897660 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:28.897226 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnwrl\" (UniqueName: \"kubernetes.io/projected/241bb069-48c9-418e-9ef8-e2d8ffae0114-kube-api-access-tnwrl\") pod \"network-check-target-r24p4\" (UID: \"241bb069-48c9-418e-9ef8-e2d8ffae0114\") " pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:33:28.900815 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:28.900792 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 13 18:33:28.910557 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:28.910535 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 13 18:33:28.921213 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:28.921186 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnwrl\" (UniqueName: \"kubernetes.io/projected/241bb069-48c9-418e-9ef8-e2d8ffae0114-kube-api-access-tnwrl\") pod \"network-check-target-r24p4\" (UID: \"241bb069-48c9-418e-9ef8-e2d8ffae0114\") " pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:33:28.998441 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:28.998400 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cfcad9fb-77bc-460d-9ee0-fbfb5573ba07-original-pull-secret\") pod \"global-pull-secret-syncer-pg9xz\" (UID: \"cfcad9fb-77bc-460d-9ee0-fbfb5573ba07\") " pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:33:28.998596 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:28.998481 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63f8372e-78ca-4580-8d1e-a57593c38dac-metrics-certs\") pod \"network-metrics-daemon-hvwvw\" (UID: \"63f8372e-78ca-4580-8d1e-a57593c38dac\") " pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:33:29.002292 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:29.002271 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 13 18:33:29.002573 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:29.002554 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 13 18:33:29.010979 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:29.010952 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63f8372e-78ca-4580-8d1e-a57593c38dac-metrics-certs\") pod \"network-metrics-daemon-hvwvw\" (UID: \"63f8372e-78ca-4580-8d1e-a57593c38dac\") " pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:33:29.011067 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:29.010963 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cfcad9fb-77bc-460d-9ee0-fbfb5573ba07-original-pull-secret\") pod \"global-pull-secret-syncer-pg9xz\" (UID: \"cfcad9fb-77bc-460d-9ee0-fbfb5573ba07\") " pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:33:29.192336 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:29.192267 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bwd4z\"" Jul 13 18:33:29.199832 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:29.199809 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hvwvw" Jul 13 18:33:29.201336 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:29.201319 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ql2kw\"" Jul 13 18:33:29.203503 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:29.203483 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pg9xz" Jul 13 18:33:29.209229 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:29.209212 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:33:29.366368 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:29.366340 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pg9xz"] Jul 13 18:33:29.597091 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:29.597064 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hvwvw"] Jul 13 18:33:29.600816 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:29.600712 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r24p4"] Jul 13 18:33:29.632964 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:33:29.632922 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod241bb069_48c9_418e_9ef8_e2d8ffae0114.slice/crio-19bd2dac826bbaaf56fd257d6dfd5d34918ab2b903be6d0521760caaa070b717 WatchSource:0}: Error finding container 19bd2dac826bbaaf56fd257d6dfd5d34918ab2b903be6d0521760caaa070b717: Status 404 returned error can't find the container with id 19bd2dac826bbaaf56fd257d6dfd5d34918ab2b903be6d0521760caaa070b717 Jul 13 18:33:29.633494 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:33:29.633468 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63f8372e_78ca_4580_8d1e_a57593c38dac.slice/crio-ca05216f4214fdd7abafa5b75761350c9b8d433ab28aca9d03ee0b4db8da7570 WatchSource:0}: Error finding container ca05216f4214fdd7abafa5b75761350c9b8d433ab28aca9d03ee0b4db8da7570: Status 404 returned error can't find the container with id ca05216f4214fdd7abafa5b75761350c9b8d433ab28aca9d03ee0b4db8da7570 Jul 13 18:33:29.634408 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:33:29.634258 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfcad9fb_77bc_460d_9ee0_fbfb5573ba07.slice/crio-e66ef068d32cf732fed9ad771d5a30ba647dee77112638c93e57ad97f920fce3 WatchSource:0}: Error finding container e66ef068d32cf732fed9ad771d5a30ba647dee77112638c93e57ad97f920fce3: Status 404 returned error can't find the container with id e66ef068d32cf732fed9ad771d5a30ba647dee77112638c93e57ad97f920fce3 Jul 13 18:33:30.169038 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:30.168949 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:30.622630 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:30.622576 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zc5rl" event={"ID":"a8906f23-73c2-4ee7-9202-ec4564ef44d3","Type":"ContainerStarted","Data":"a49ac2fed4dc4e6d9310cf7a40cd5032efbe9fcc598da175bc8aea5a95ed8429"} Jul 13 18:33:30.624778 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:30.624741 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-qc4ml" event={"ID":"4923aacc-d13a-4892-9cde-a2425d9f36b4","Type":"ContainerStarted","Data":"dfdda188c7238964c2874a24dbd9ae2bf48b5032cdd9395b0d8a2405066a13a7"} Jul 13 18:33:30.624909 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:30.624795 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-qc4ml" event={"ID":"4923aacc-d13a-4892-9cde-a2425d9f36b4","Type":"ContainerStarted","Data":"742d233882810473ec22499350098fb9d5a3d6093c24b2d284f347ae06924d1d"} Jul 13 18:33:30.627513 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:30.627330 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hvwvw" event={"ID":"63f8372e-78ca-4580-8d1e-a57593c38dac","Type":"ContainerStarted","Data":"ca05216f4214fdd7abafa5b75761350c9b8d433ab28aca9d03ee0b4db8da7570"} Jul 13 18:33:30.629031 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:30.628972 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r24p4" event={"ID":"241bb069-48c9-418e-9ef8-e2d8ffae0114","Type":"ContainerStarted","Data":"19bd2dac826bbaaf56fd257d6dfd5d34918ab2b903be6d0521760caaa070b717"} Jul 13 18:33:30.630509 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:30.630484 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pg9xz" event={"ID":"cfcad9fb-77bc-460d-9ee0-fbfb5573ba07","Type":"ContainerStarted","Data":"e66ef068d32cf732fed9ad771d5a30ba647dee77112638c93e57ad97f920fce3"} Jul 13 18:33:31.639082 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:31.639043 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hvwvw" event={"ID":"63f8372e-78ca-4580-8d1e-a57593c38dac","Type":"ContainerStarted","Data":"8e0dde9bba41f214edb046f17be706c2c234fbe37183761f8206a278800ddb11"} Jul 13 18:33:31.639533 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:31.639104 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hvwvw" event={"ID":"63f8372e-78ca-4580-8d1e-a57593c38dac","Type":"ContainerStarted","Data":"fe7ca2beab2c252fe848aae0f847c0148967cf1e0cebc21087dc0b64e8a00de9"} Jul 13 18:33:31.674879 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:31.674824 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-qc4ml" podStartSLOduration=3.0981502499999998 podStartE2EDuration="4.674805059s" totalImagesPullingTime="1.576654809s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:33:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:33:30.661361481 +0000 UTC m=+66.944790084" watchObservedRunningTime="2026-07-13 18:33:31.674805059 +0000 UTC m=+67.958233660" Jul 13 18:33:31.675118 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:31.675086 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-hvwvw" podStartSLOduration=66.775631696 podStartE2EDuration="1m7.675074342s" totalImagesPullingTime="899.442646ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:32:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:33:31.673176176 +0000 UTC m=+67.956604778" watchObservedRunningTime="2026-07-13 18:33:31.675074342 +0000 UTC m=+67.958502979" Jul 13 18:33:31.934898 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:31.934767 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:32.226034 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:32.225995 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:32.592782 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:32.592705 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-v5z9m" Jul 13 18:33:32.997164 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:32.997127 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:33.234661 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.234622 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:33.558084 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.558048 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:33.561416 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.561390 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-bggrq"] Jul 13 18:33:33.579399 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.579363 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-bggrq" Jul 13 18:33:33.583329 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.583306 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 13 18:33:33.583456 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.583420 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-hvtdn\"" Jul 13 18:33:33.583513 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.583308 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 13 18:33:33.590165 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.590078 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-bggrq"] Jul 13 18:33:33.635465 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.635422 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hq4nn\" (UniqueName: \"kubernetes.io/projected/e657c3c1-fb9d-4da0-b88f-6db7f2510f95-kube-api-access-hq4nn\") pod \"downloads-68c9cbb497-bggrq\" (UID: \"e657c3c1-fb9d-4da0-b88f-6db7f2510f95\") " pod="openshift-console/downloads-68c9cbb497-bggrq" Jul 13 18:33:33.652080 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.652034 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-zd5xm"] Jul 13 18:33:33.671203 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.671182 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zd5xm" Jul 13 18:33:33.680312 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.680287 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 13 18:33:33.680447 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.680360 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 13 18:33:33.681879 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.681497 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 13 18:33:33.681879 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.681734 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-jxvkr\"" Jul 13 18:33:33.682702 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.682677 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-vrgqz"] Jul 13 18:33:33.707504 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.707472 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-vrgqz" Jul 13 18:33:33.710039 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.710005 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-8zgr7"] Jul 13 18:33:33.711934 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.711900 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 13 18:33:33.712203 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.712185 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 13 18:33:33.712939 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.712921 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-444mg\"" Jul 13 18:33:33.730105 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.730075 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:33.733669 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.733649 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-vrgqz"] Jul 13 18:33:33.733789 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.733773 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-8zgr7" Jul 13 18:33:33.736132 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.736113 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/58ca72a5-ee9b-44f0-9cf2-c503f37d929b-node-exporter-tls\") pod \"node-exporter-zd5xm\" (UID: \"58ca72a5-ee9b-44f0-9cf2-c503f37d929b\") " pod="openshift-monitoring/node-exporter-zd5xm" Jul 13 18:33:33.736245 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.736138 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/58ca72a5-ee9b-44f0-9cf2-c503f37d929b-sys\") pod \"node-exporter-zd5xm\" (UID: \"58ca72a5-ee9b-44f0-9cf2-c503f37d929b\") " pod="openshift-monitoring/node-exporter-zd5xm" Jul 13 18:33:33.736245 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.736156 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58ca72a5-ee9b-44f0-9cf2-c503f37d929b-metrics-client-ca\") pod \"node-exporter-zd5xm\" (UID: \"58ca72a5-ee9b-44f0-9cf2-c503f37d929b\") " pod="openshift-monitoring/node-exporter-zd5xm" Jul 13 18:33:33.736245 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.736193 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hq4nn\" (UniqueName: \"kubernetes.io/projected/e657c3c1-fb9d-4da0-b88f-6db7f2510f95-kube-api-access-hq4nn\") pod \"downloads-68c9cbb497-bggrq\" (UID: \"e657c3c1-fb9d-4da0-b88f-6db7f2510f95\") " pod="openshift-console/downloads-68c9cbb497-bggrq" Jul 13 18:33:33.736245 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.736226 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/58ca72a5-ee9b-44f0-9cf2-c503f37d929b-node-exporter-accelerators-collector-config\") pod \"node-exporter-zd5xm\" (UID: \"58ca72a5-ee9b-44f0-9cf2-c503f37d929b\") " pod="openshift-monitoring/node-exporter-zd5xm" Jul 13 18:33:33.736460 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.736259 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/58ca72a5-ee9b-44f0-9cf2-c503f37d929b-root\") pod \"node-exporter-zd5xm\" (UID: \"58ca72a5-ee9b-44f0-9cf2-c503f37d929b\") " pod="openshift-monitoring/node-exporter-zd5xm" Jul 13 18:33:33.736460 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.736283 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/58ca72a5-ee9b-44f0-9cf2-c503f37d929b-node-exporter-textfile\") pod \"node-exporter-zd5xm\" (UID: \"58ca72a5-ee9b-44f0-9cf2-c503f37d929b\") " pod="openshift-monitoring/node-exporter-zd5xm" Jul 13 18:33:33.736460 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.736315 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2s4n\" (UniqueName: \"kubernetes.io/projected/58ca72a5-ee9b-44f0-9cf2-c503f37d929b-kube-api-access-k2s4n\") pod \"node-exporter-zd5xm\" (UID: \"58ca72a5-ee9b-44f0-9cf2-c503f37d929b\") " pod="openshift-monitoring/node-exporter-zd5xm" Jul 13 18:33:33.736460 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.736344 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/58ca72a5-ee9b-44f0-9cf2-c503f37d929b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zd5xm\" (UID: \"58ca72a5-ee9b-44f0-9cf2-c503f37d929b\") " pod="openshift-monitoring/node-exporter-zd5xm" Jul 13 18:33:33.736460 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.736374 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/58ca72a5-ee9b-44f0-9cf2-c503f37d929b-node-exporter-wtmp\") pod \"node-exporter-zd5xm\" (UID: \"58ca72a5-ee9b-44f0-9cf2-c503f37d929b\") " pod="openshift-monitoring/node-exporter-zd5xm" Jul 13 18:33:33.743024 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.743003 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 13 18:33:33.743154 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.743137 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-vzv4r\"" Jul 13 18:33:33.743634 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.743615 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 13 18:33:33.743758 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.743740 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 13 18:33:33.771081 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.771051 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hq4nn\" (UniqueName: \"kubernetes.io/projected/e657c3c1-fb9d-4da0-b88f-6db7f2510f95-kube-api-access-hq4nn\") pod \"downloads-68c9cbb497-bggrq\" (UID: \"e657c3c1-fb9d-4da0-b88f-6db7f2510f95\") " pod="openshift-console/downloads-68c9cbb497-bggrq" Jul 13 18:33:33.780320 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.780277 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-8zgr7"] Jul 13 18:33:33.837017 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.836941 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/58ca72a5-ee9b-44f0-9cf2-c503f37d929b-node-exporter-textfile\") pod \"node-exporter-zd5xm\" (UID: \"58ca72a5-ee9b-44f0-9cf2-c503f37d929b\") " pod="openshift-monitoring/node-exporter-zd5xm" Jul 13 18:33:33.837017 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.836984 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/4fa44170-99b2-4f9a-be6e-415cc519a3f5-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-8zgr7\" (UID: \"4fa44170-99b2-4f9a-be6e-415cc519a3f5\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-8zgr7" Jul 13 18:33:33.837017 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.837016 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/58ca72a5-ee9b-44f0-9cf2-c503f37d929b-node-exporter-wtmp\") pod \"node-exporter-zd5xm\" (UID: \"58ca72a5-ee9b-44f0-9cf2-c503f37d929b\") " pod="openshift-monitoring/node-exporter-zd5xm" Jul 13 18:33:33.837253 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.837042 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4fa44170-99b2-4f9a-be6e-415cc519a3f5-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-8zgr7\" (UID: \"4fa44170-99b2-4f9a-be6e-415cc519a3f5\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-8zgr7" Jul 13 18:33:33.837253 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.837074 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/58ca72a5-ee9b-44f0-9cf2-c503f37d929b-node-exporter-tls\") pod \"node-exporter-zd5xm\" (UID: \"58ca72a5-ee9b-44f0-9cf2-c503f37d929b\") " pod="openshift-monitoring/node-exporter-zd5xm" Jul 13 18:33:33.837253 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.837140 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/58ca72a5-ee9b-44f0-9cf2-c503f37d929b-sys\") pod \"node-exporter-zd5xm\" (UID: \"58ca72a5-ee9b-44f0-9cf2-c503f37d929b\") " pod="openshift-monitoring/node-exporter-zd5xm" Jul 13 18:33:33.837253 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.837172 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4fa44170-99b2-4f9a-be6e-415cc519a3f5-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-8zgr7\" (UID: \"4fa44170-99b2-4f9a-be6e-415cc519a3f5\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-8zgr7" Jul 13 18:33:33.837253 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.837222 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/58ca72a5-ee9b-44f0-9cf2-c503f37d929b-node-exporter-accelerators-collector-config\") pod \"node-exporter-zd5xm\" (UID: \"58ca72a5-ee9b-44f0-9cf2-c503f37d929b\") " pod="openshift-monitoring/node-exporter-zd5xm" Jul 13 18:33:33.837439 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.837252 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4fa44170-99b2-4f9a-be6e-415cc519a3f5-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-8zgr7\" (UID: \"4fa44170-99b2-4f9a-be6e-415cc519a3f5\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-8zgr7" Jul 13 18:33:33.837439 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.837278 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfq6x\" (UniqueName: \"kubernetes.io/projected/b6d0ba1f-a0cd-490b-9a6a-1c573bdffc5e-kube-api-access-mfq6x\") pod \"openshift-state-metrics-84bd68b896-vrgqz\" (UID: \"b6d0ba1f-a0cd-490b-9a6a-1c573bdffc5e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-vrgqz" Jul 13 18:33:33.837439 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.837301 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/58ca72a5-ee9b-44f0-9cf2-c503f37d929b-root\") pod \"node-exporter-zd5xm\" (UID: \"58ca72a5-ee9b-44f0-9cf2-c503f37d929b\") " pod="openshift-monitoring/node-exporter-zd5xm" Jul 13 18:33:33.837439 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.837323 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2s4n\" (UniqueName: \"kubernetes.io/projected/58ca72a5-ee9b-44f0-9cf2-c503f37d929b-kube-api-access-k2s4n\") pod \"node-exporter-zd5xm\" (UID: \"58ca72a5-ee9b-44f0-9cf2-c503f37d929b\") " pod="openshift-monitoring/node-exporter-zd5xm" Jul 13 18:33:33.837439 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.837344 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/58ca72a5-ee9b-44f0-9cf2-c503f37d929b-node-exporter-textfile\") pod \"node-exporter-zd5xm\" (UID: \"58ca72a5-ee9b-44f0-9cf2-c503f37d929b\") " pod="openshift-monitoring/node-exporter-zd5xm" Jul 13 18:33:33.837439 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.837348 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/4fa44170-99b2-4f9a-be6e-415cc519a3f5-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-8zgr7\" (UID: \"4fa44170-99b2-4f9a-be6e-415cc519a3f5\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-8zgr7" Jul 13 18:33:33.837439 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.837378 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/58ca72a5-ee9b-44f0-9cf2-c503f37d929b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zd5xm\" (UID: \"58ca72a5-ee9b-44f0-9cf2-c503f37d929b\") " pod="openshift-monitoring/node-exporter-zd5xm" Jul 13 18:33:33.837439 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.837410 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6d0ba1f-a0cd-490b-9a6a-1c573bdffc5e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-vrgqz\" (UID: \"b6d0ba1f-a0cd-490b-9a6a-1c573bdffc5e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-vrgqz" Jul 13 18:33:33.837814 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.837474 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/58ca72a5-ee9b-44f0-9cf2-c503f37d929b-sys\") pod \"node-exporter-zd5xm\" (UID: \"58ca72a5-ee9b-44f0-9cf2-c503f37d929b\") " pod="openshift-monitoring/node-exporter-zd5xm" Jul 13 18:33:33.837814 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.837477 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/58ca72a5-ee9b-44f0-9cf2-c503f37d929b-node-exporter-wtmp\") pod \"node-exporter-zd5xm\" (UID: \"58ca72a5-ee9b-44f0-9cf2-c503f37d929b\") " pod="openshift-monitoring/node-exporter-zd5xm" Jul 13 18:33:33.837814 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.837525 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/58ca72a5-ee9b-44f0-9cf2-c503f37d929b-root\") pod \"node-exporter-zd5xm\" (UID: \"58ca72a5-ee9b-44f0-9cf2-c503f37d929b\") " pod="openshift-monitoring/node-exporter-zd5xm" Jul 13 18:33:33.837814 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.837590 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvfkr\" (UniqueName: \"kubernetes.io/projected/4fa44170-99b2-4f9a-be6e-415cc519a3f5-kube-api-access-bvfkr\") pod \"kube-state-metrics-785778ff74-8zgr7\" (UID: \"4fa44170-99b2-4f9a-be6e-415cc519a3f5\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-8zgr7" Jul 13 18:33:33.837814 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.837669 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58ca72a5-ee9b-44f0-9cf2-c503f37d929b-metrics-client-ca\") pod \"node-exporter-zd5xm\" (UID: \"58ca72a5-ee9b-44f0-9cf2-c503f37d929b\") " pod="openshift-monitoring/node-exporter-zd5xm" Jul 13 18:33:33.837814 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.837767 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6d0ba1f-a0cd-490b-9a6a-1c573bdffc5e-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-vrgqz\" (UID: \"b6d0ba1f-a0cd-490b-9a6a-1c573bdffc5e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-vrgqz" Jul 13 18:33:33.837814 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.837811 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b6d0ba1f-a0cd-490b-9a6a-1c573bdffc5e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-vrgqz\" (UID: \"b6d0ba1f-a0cd-490b-9a6a-1c573bdffc5e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-vrgqz" Jul 13 18:33:33.838161 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.838030 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/58ca72a5-ee9b-44f0-9cf2-c503f37d929b-node-exporter-accelerators-collector-config\") pod \"node-exporter-zd5xm\" (UID: \"58ca72a5-ee9b-44f0-9cf2-c503f37d929b\") " pod="openshift-monitoring/node-exporter-zd5xm" Jul 13 18:33:33.838292 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.838228 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58ca72a5-ee9b-44f0-9cf2-c503f37d929b-metrics-client-ca\") pod \"node-exporter-zd5xm\" (UID: \"58ca72a5-ee9b-44f0-9cf2-c503f37d929b\") " pod="openshift-monitoring/node-exporter-zd5xm" Jul 13 18:33:33.840322 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.840298 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/58ca72a5-ee9b-44f0-9cf2-c503f37d929b-node-exporter-tls\") pod \"node-exporter-zd5xm\" (UID: \"58ca72a5-ee9b-44f0-9cf2-c503f37d929b\") " pod="openshift-monitoring/node-exporter-zd5xm" Jul 13 18:33:33.840419 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.840306 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/58ca72a5-ee9b-44f0-9cf2-c503f37d929b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zd5xm\" (UID: \"58ca72a5-ee9b-44f0-9cf2-c503f37d929b\") " pod="openshift-monitoring/node-exporter-zd5xm" Jul 13 18:33:33.860215 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.860159 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2s4n\" (UniqueName: \"kubernetes.io/projected/58ca72a5-ee9b-44f0-9cf2-c503f37d929b-kube-api-access-k2s4n\") pod \"node-exporter-zd5xm\" (UID: \"58ca72a5-ee9b-44f0-9cf2-c503f37d929b\") " pod="openshift-monitoring/node-exporter-zd5xm" Jul 13 18:33:33.882358 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.882281 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:33.892358 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.892327 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-bggrq" Jul 13 18:33:33.939090 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.939041 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4fa44170-99b2-4f9a-be6e-415cc519a3f5-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-8zgr7\" (UID: \"4fa44170-99b2-4f9a-be6e-415cc519a3f5\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-8zgr7" Jul 13 18:33:33.939268 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.939099 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4fa44170-99b2-4f9a-be6e-415cc519a3f5-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-8zgr7\" (UID: \"4fa44170-99b2-4f9a-be6e-415cc519a3f5\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-8zgr7" Jul 13 18:33:33.939268 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.939133 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfq6x\" (UniqueName: \"kubernetes.io/projected/b6d0ba1f-a0cd-490b-9a6a-1c573bdffc5e-kube-api-access-mfq6x\") pod \"openshift-state-metrics-84bd68b896-vrgqz\" (UID: \"b6d0ba1f-a0cd-490b-9a6a-1c573bdffc5e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-vrgqz" Jul 13 18:33:33.939268 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.939162 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/4fa44170-99b2-4f9a-be6e-415cc519a3f5-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-8zgr7\" (UID: \"4fa44170-99b2-4f9a-be6e-415cc519a3f5\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-8zgr7" Jul 13 18:33:33.939268 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.939193 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6d0ba1f-a0cd-490b-9a6a-1c573bdffc5e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-vrgqz\" (UID: \"b6d0ba1f-a0cd-490b-9a6a-1c573bdffc5e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-vrgqz" Jul 13 18:33:33.939268 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.939224 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvfkr\" (UniqueName: \"kubernetes.io/projected/4fa44170-99b2-4f9a-be6e-415cc519a3f5-kube-api-access-bvfkr\") pod \"kube-state-metrics-785778ff74-8zgr7\" (UID: \"4fa44170-99b2-4f9a-be6e-415cc519a3f5\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-8zgr7" Jul 13 18:33:33.939268 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.939257 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6d0ba1f-a0cd-490b-9a6a-1c573bdffc5e-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-vrgqz\" (UID: \"b6d0ba1f-a0cd-490b-9a6a-1c573bdffc5e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-vrgqz" Jul 13 18:33:33.939550 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.939281 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b6d0ba1f-a0cd-490b-9a6a-1c573bdffc5e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-vrgqz\" (UID: \"b6d0ba1f-a0cd-490b-9a6a-1c573bdffc5e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-vrgqz" Jul 13 18:33:33.939550 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.939324 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/4fa44170-99b2-4f9a-be6e-415cc519a3f5-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-8zgr7\" (UID: \"4fa44170-99b2-4f9a-be6e-415cc519a3f5\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-8zgr7" Jul 13 18:33:33.939550 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.939359 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4fa44170-99b2-4f9a-be6e-415cc519a3f5-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-8zgr7\" (UID: \"4fa44170-99b2-4f9a-be6e-415cc519a3f5\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-8zgr7" Jul 13 18:33:33.939721 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.939643 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/4fa44170-99b2-4f9a-be6e-415cc519a3f5-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-8zgr7\" (UID: \"4fa44170-99b2-4f9a-be6e-415cc519a3f5\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-8zgr7" Jul 13 18:33:33.940068 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.940044 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4fa44170-99b2-4f9a-be6e-415cc519a3f5-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-8zgr7\" (UID: \"4fa44170-99b2-4f9a-be6e-415cc519a3f5\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-8zgr7" Jul 13 18:33:33.940185 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.940124 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6d0ba1f-a0cd-490b-9a6a-1c573bdffc5e-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-vrgqz\" (UID: \"b6d0ba1f-a0cd-490b-9a6a-1c573bdffc5e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-vrgqz" Jul 13 18:33:33.940458 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.940431 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/4fa44170-99b2-4f9a-be6e-415cc519a3f5-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-8zgr7\" (UID: \"4fa44170-99b2-4f9a-be6e-415cc519a3f5\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-8zgr7" Jul 13 18:33:33.942177 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.942151 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4fa44170-99b2-4f9a-be6e-415cc519a3f5-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-8zgr7\" (UID: \"4fa44170-99b2-4f9a-be6e-415cc519a3f5\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-8zgr7" Jul 13 18:33:33.942656 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.942632 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b6d0ba1f-a0cd-490b-9a6a-1c573bdffc5e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-vrgqz\" (UID: \"b6d0ba1f-a0cd-490b-9a6a-1c573bdffc5e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-vrgqz" Jul 13 18:33:33.942769 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.942721 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4fa44170-99b2-4f9a-be6e-415cc519a3f5-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-8zgr7\" (UID: \"4fa44170-99b2-4f9a-be6e-415cc519a3f5\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-8zgr7" Jul 13 18:33:33.942870 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.942852 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6d0ba1f-a0cd-490b-9a6a-1c573bdffc5e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-vrgqz\" (UID: \"b6d0ba1f-a0cd-490b-9a6a-1c573bdffc5e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-vrgqz" Jul 13 18:33:33.968738 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.968711 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvfkr\" (UniqueName: \"kubernetes.io/projected/4fa44170-99b2-4f9a-be6e-415cc519a3f5-kube-api-access-bvfkr\") pod \"kube-state-metrics-785778ff74-8zgr7\" (UID: \"4fa44170-99b2-4f9a-be6e-415cc519a3f5\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-8zgr7" Jul 13 18:33:33.980450 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.980426 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfq6x\" (UniqueName: \"kubernetes.io/projected/b6d0ba1f-a0cd-490b-9a6a-1c573bdffc5e-kube-api-access-mfq6x\") pod \"openshift-state-metrics-84bd68b896-vrgqz\" (UID: \"b6d0ba1f-a0cd-490b-9a6a-1c573bdffc5e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-vrgqz" Jul 13 18:33:33.985275 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:33.985256 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zd5xm" Jul 13 18:33:34.017935 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.017908 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-vrgqz" Jul 13 18:33:34.044873 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.044850 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-8zgr7" Jul 13 18:33:34.069727 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.069693 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:34.207708 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:33:34.207637 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58ca72a5_ee9b_44f0_9cf2_c503f37d929b.slice/crio-97bf5f0623943f535351d66f24710e019014f8af8e4e196294388ea582a835bc WatchSource:0}: Error finding container 97bf5f0623943f535351d66f24710e019014f8af8e4e196294388ea582a835bc: Status 404 returned error can't find the container with id 97bf5f0623943f535351d66f24710e019014f8af8e4e196294388ea582a835bc Jul 13 18:33:34.409497 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.409466 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-8zgr7"] Jul 13 18:33:34.412781 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:33:34.412533 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fa44170_99b2_4f9a_be6e_415cc519a3f5.slice/crio-706a53013a135bf30129280432453bbccc9653035b06b3fa7e7c452ed4dea429 WatchSource:0}: Error finding container 706a53013a135bf30129280432453bbccc9653035b06b3fa7e7c452ed4dea429: Status 404 returned error can't find the container with id 706a53013a135bf30129280432453bbccc9653035b06b3fa7e7c452ed4dea429 Jul 13 18:33:34.655221 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.655139 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zc5rl" event={"ID":"a8906f23-73c2-4ee7-9202-ec4564ef44d3","Type":"ContainerStarted","Data":"cfc63e86a9cbbd5fec38d22ad6473566db760f1b184da2445719e044aa8d9127"} Jul 13 18:33:34.656395 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.656364 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-8zgr7" event={"ID":"4fa44170-99b2-4f9a-be6e-415cc519a3f5","Type":"ContainerStarted","Data":"706a53013a135bf30129280432453bbccc9653035b06b3fa7e7c452ed4dea429"} Jul 13 18:33:34.658034 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.658007 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r24p4" event={"ID":"241bb069-48c9-418e-9ef8-e2d8ffae0114","Type":"ContainerStarted","Data":"0c3e6a7036c6818308afa256687029738e86449dc29758ca57cb1c2ad63069a0"} Jul 13 18:33:34.658131 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.658077 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:33:34.659553 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.659534 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zd5xm" event={"ID":"58ca72a5-ee9b-44f0-9cf2-c503f37d929b","Type":"ContainerStarted","Data":"97bf5f0623943f535351d66f24710e019014f8af8e4e196294388ea582a835bc"} Jul 13 18:33:34.696473 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.696418 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-vrgqz"] Jul 13 18:33:34.698146 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.698092 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-bggrq"] Jul 13 18:33:34.700645 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:33:34.700596 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6d0ba1f_a0cd_490b_9a6a_1c573bdffc5e.slice/crio-c866c773bf09de719338e518112ee6d7fd762f8bcdcdd0cf262fd6b773e25d64 WatchSource:0}: Error finding container c866c773bf09de719338e518112ee6d7fd762f8bcdcdd0cf262fd6b773e25d64: Status 404 returned error can't find the container with id c866c773bf09de719338e518112ee6d7fd762f8bcdcdd0cf262fd6b773e25d64 Jul 13 18:33:34.701292 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:33:34.701273 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode657c3c1_fb9d_4da0_b88f_6db7f2510f95.slice/crio-10aeba84ba489a31c310a61dd7343fb764cd971555dc4cc8077a712c676d2406 WatchSource:0}: Error finding container 10aeba84ba489a31c310a61dd7343fb764cd971555dc4cc8077a712c676d2406: Status 404 returned error can't find the container with id 10aeba84ba489a31c310a61dd7343fb764cd971555dc4cc8077a712c676d2406 Jul 13 18:33:34.713203 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.713154 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zc5rl" podStartSLOduration=1.8888097849999999 podStartE2EDuration="17.713134842s" totalImagesPullingTime="15.824325057s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:33:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:33:34.711483756 +0000 UTC m=+70.994912384" watchObservedRunningTime="2026-07-13 18:33:34.713134842 +0000 UTC m=+70.996563447" Jul 13 18:33:34.722178 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.722153 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 18:33:34.751547 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.751516 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.756860 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.756834 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 13 18:33:34.759746 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.759727 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 13 18:33:34.762738 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.762715 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 13 18:33:34.762992 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.762970 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 13 18:33:34.763540 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.763390 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-bmbw9\"" Jul 13 18:33:34.763540 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.763425 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 13 18:33:34.763540 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.763488 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 13 18:33:34.763764 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.763726 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 13 18:33:34.764558 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.764379 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 13 18:33:34.764558 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.764464 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 13 18:33:34.779408 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.779363 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-r24p4" podStartSLOduration=66.207920324 podStartE2EDuration="1m10.779345927s" totalImagesPullingTime="4.571425603s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:32:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:33:34.776808604 +0000 UTC m=+71.060237204" watchObservedRunningTime="2026-07-13 18:33:34.779345927 +0000 UTC m=+71.062774528" Jul 13 18:33:34.782838 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.782310 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 18:33:34.848073 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.848004 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f9014714-972e-429c-88bb-30b690ad1215-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.848335 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.848116 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82ksp\" (UniqueName: \"kubernetes.io/projected/f9014714-972e-429c-88bb-30b690ad1215-kube-api-access-82ksp\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.848517 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.848484 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.848811 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.848794 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-web-config\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.848885 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.848844 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-config-volume\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.849047 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.849031 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.849275 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.849071 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.849345 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.849323 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f9014714-972e-429c-88bb-30b690ad1215-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.849527 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.849361 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.849587 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.849552 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9014714-972e-429c-88bb-30b690ad1215-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.849816 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.849794 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f9014714-972e-429c-88bb-30b690ad1215-config-out\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.849952 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.849864 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.850124 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.850105 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f9014714-972e-429c-88bb-30b690ad1215-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.952643 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.951743 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82ksp\" (UniqueName: \"kubernetes.io/projected/f9014714-972e-429c-88bb-30b690ad1215-kube-api-access-82ksp\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.952643 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.951798 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.952643 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.951826 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-web-config\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.952643 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.951851 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-config-volume\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.952643 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.951866 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.952643 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.951883 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.952643 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.951918 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f9014714-972e-429c-88bb-30b690ad1215-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.952643 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.951946 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.952643 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.951971 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9014714-972e-429c-88bb-30b690ad1215-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.952643 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.951997 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f9014714-972e-429c-88bb-30b690ad1215-config-out\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.952643 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.952030 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.952643 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.952049 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f9014714-972e-429c-88bb-30b690ad1215-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.952643 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.952077 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f9014714-972e-429c-88bb-30b690ad1215-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.953810 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.953778 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f9014714-972e-429c-88bb-30b690ad1215-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.953922 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.953856 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9014714-972e-429c-88bb-30b690ad1215-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.956144 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.955829 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f9014714-972e-429c-88bb-30b690ad1215-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.957573 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.957523 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.958153 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.958128 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.959536 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.959021 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f9014714-972e-429c-88bb-30b690ad1215-config-out\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.959536 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.959494 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.959775 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.959709 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-config-volume\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.960261 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.960216 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f9014714-972e-429c-88bb-30b690ad1215-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.960427 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.960373 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-web-config\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.961100 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.961079 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.961543 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.961529 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:34.990063 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:34.990039 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82ksp\" (UniqueName: \"kubernetes.io/projected/f9014714-972e-429c-88bb-30b690ad1215-kube-api-access-82ksp\") pod \"alertmanager-main-0\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:35.103658 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:35.103592 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:33:35.498488 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:35.498453 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 18:33:35.667576 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:35.667455 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-bggrq" event={"ID":"e657c3c1-fb9d-4da0-b88f-6db7f2510f95","Type":"ContainerStarted","Data":"10aeba84ba489a31c310a61dd7343fb764cd971555dc4cc8077a712c676d2406"} Jul 13 18:33:35.670317 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:35.670197 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-vrgqz" event={"ID":"b6d0ba1f-a0cd-490b-9a6a-1c573bdffc5e","Type":"ContainerStarted","Data":"4e3391993ab649a43dd4931d7afc6078396fe281deb2c9a87c700510ed322d65"} Jul 13 18:33:35.670317 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:35.670257 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-vrgqz" event={"ID":"b6d0ba1f-a0cd-490b-9a6a-1c573bdffc5e","Type":"ContainerStarted","Data":"37dd453b30d35e636a5d9c430aea056b60cf61420f47d9f5e9e9ee42b4f70bf6"} Jul 13 18:33:35.670317 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:35.670284 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-vrgqz" event={"ID":"b6d0ba1f-a0cd-490b-9a6a-1c573bdffc5e","Type":"ContainerStarted","Data":"c866c773bf09de719338e518112ee6d7fd762f8bcdcdd0cf262fd6b773e25d64"} Jul 13 18:33:36.382517 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:36.382475 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:36.486458 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:36.486414 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:36.656512 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:36.656428 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:36.894341 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:36.894305 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:36.966330 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:33:36.966300 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9014714_972e_429c_88bb_30b690ad1215.slice/crio-eba088ade4fbfe5f08ceb32d81417b78b1bf1c2940a7b4a2afc122d635082aaf WatchSource:0}: Error finding container eba088ade4fbfe5f08ceb32d81417b78b1bf1c2940a7b4a2afc122d635082aaf: Status 404 returned error can't find the container with id eba088ade4fbfe5f08ceb32d81417b78b1bf1c2940a7b4a2afc122d635082aaf Jul 13 18:33:37.200346 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.200312 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:37.662374 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.662344 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-77f95c75bb-4mxf7"] Jul 13 18:33:37.666471 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.666445 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-77f95c75bb-4mxf7" Jul 13 18:33:37.674859 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.674836 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 13 18:33:37.675765 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.675667 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-6s5jtp6q9hvv9\"" Jul 13 18:33:37.675841 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.675805 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 13 18:33:37.675921 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.675903 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-rnnn7\"" Jul 13 18:33:37.675981 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.675949 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 13 18:33:37.676095 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.676033 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 13 18:33:37.676095 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.676033 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 13 18:33:37.680896 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.680871 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f9014714-972e-429c-88bb-30b690ad1215","Type":"ContainerStarted","Data":"eba088ade4fbfe5f08ceb32d81417b78b1bf1c2940a7b4a2afc122d635082aaf"} Jul 13 18:33:37.696597 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.696502 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-77f95c75bb-4mxf7"] Jul 13 18:33:37.777675 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.777539 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/572dea7e-ab24-47f1-9e10-e099d9e0316e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-77f95c75bb-4mxf7\" (UID: \"572dea7e-ab24-47f1-9e10-e099d9e0316e\") " pod="openshift-monitoring/thanos-querier-77f95c75bb-4mxf7" Jul 13 18:33:37.777675 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.777587 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znvrq\" (UniqueName: \"kubernetes.io/projected/572dea7e-ab24-47f1-9e10-e099d9e0316e-kube-api-access-znvrq\") pod \"thanos-querier-77f95c75bb-4mxf7\" (UID: \"572dea7e-ab24-47f1-9e10-e099d9e0316e\") " pod="openshift-monitoring/thanos-querier-77f95c75bb-4mxf7" Jul 13 18:33:37.777675 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.777645 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/572dea7e-ab24-47f1-9e10-e099d9e0316e-secret-thanos-querier-tls\") pod \"thanos-querier-77f95c75bb-4mxf7\" (UID: \"572dea7e-ab24-47f1-9e10-e099d9e0316e\") " pod="openshift-monitoring/thanos-querier-77f95c75bb-4mxf7" Jul 13 18:33:37.777893 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.777681 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/572dea7e-ab24-47f1-9e10-e099d9e0316e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-77f95c75bb-4mxf7\" (UID: \"572dea7e-ab24-47f1-9e10-e099d9e0316e\") " pod="openshift-monitoring/thanos-querier-77f95c75bb-4mxf7" Jul 13 18:33:37.777893 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.777715 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/572dea7e-ab24-47f1-9e10-e099d9e0316e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-77f95c75bb-4mxf7\" (UID: \"572dea7e-ab24-47f1-9e10-e099d9e0316e\") " pod="openshift-monitoring/thanos-querier-77f95c75bb-4mxf7" Jul 13 18:33:37.777893 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.777776 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/572dea7e-ab24-47f1-9e10-e099d9e0316e-secret-grpc-tls\") pod \"thanos-querier-77f95c75bb-4mxf7\" (UID: \"572dea7e-ab24-47f1-9e10-e099d9e0316e\") " pod="openshift-monitoring/thanos-querier-77f95c75bb-4mxf7" Jul 13 18:33:37.777893 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.777806 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/572dea7e-ab24-47f1-9e10-e099d9e0316e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-77f95c75bb-4mxf7\" (UID: \"572dea7e-ab24-47f1-9e10-e099d9e0316e\") " pod="openshift-monitoring/thanos-querier-77f95c75bb-4mxf7" Jul 13 18:33:37.777893 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.777834 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/572dea7e-ab24-47f1-9e10-e099d9e0316e-metrics-client-ca\") pod \"thanos-querier-77f95c75bb-4mxf7\" (UID: \"572dea7e-ab24-47f1-9e10-e099d9e0316e\") " pod="openshift-monitoring/thanos-querier-77f95c75bb-4mxf7" Jul 13 18:33:37.879341 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.878569 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znvrq\" (UniqueName: \"kubernetes.io/projected/572dea7e-ab24-47f1-9e10-e099d9e0316e-kube-api-access-znvrq\") pod \"thanos-querier-77f95c75bb-4mxf7\" (UID: \"572dea7e-ab24-47f1-9e10-e099d9e0316e\") " pod="openshift-monitoring/thanos-querier-77f95c75bb-4mxf7" Jul 13 18:33:37.879341 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.878654 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/572dea7e-ab24-47f1-9e10-e099d9e0316e-secret-thanos-querier-tls\") pod \"thanos-querier-77f95c75bb-4mxf7\" (UID: \"572dea7e-ab24-47f1-9e10-e099d9e0316e\") " pod="openshift-monitoring/thanos-querier-77f95c75bb-4mxf7" Jul 13 18:33:37.879341 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.878693 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/572dea7e-ab24-47f1-9e10-e099d9e0316e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-77f95c75bb-4mxf7\" (UID: \"572dea7e-ab24-47f1-9e10-e099d9e0316e\") " pod="openshift-monitoring/thanos-querier-77f95c75bb-4mxf7" Jul 13 18:33:37.879341 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.878731 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/572dea7e-ab24-47f1-9e10-e099d9e0316e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-77f95c75bb-4mxf7\" (UID: \"572dea7e-ab24-47f1-9e10-e099d9e0316e\") " pod="openshift-monitoring/thanos-querier-77f95c75bb-4mxf7" Jul 13 18:33:37.879341 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.878798 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/572dea7e-ab24-47f1-9e10-e099d9e0316e-secret-grpc-tls\") pod \"thanos-querier-77f95c75bb-4mxf7\" (UID: \"572dea7e-ab24-47f1-9e10-e099d9e0316e\") " pod="openshift-monitoring/thanos-querier-77f95c75bb-4mxf7" Jul 13 18:33:37.879341 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.878830 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/572dea7e-ab24-47f1-9e10-e099d9e0316e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-77f95c75bb-4mxf7\" (UID: \"572dea7e-ab24-47f1-9e10-e099d9e0316e\") " pod="openshift-monitoring/thanos-querier-77f95c75bb-4mxf7" Jul 13 18:33:37.879341 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.878854 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/572dea7e-ab24-47f1-9e10-e099d9e0316e-metrics-client-ca\") pod \"thanos-querier-77f95c75bb-4mxf7\" (UID: \"572dea7e-ab24-47f1-9e10-e099d9e0316e\") " pod="openshift-monitoring/thanos-querier-77f95c75bb-4mxf7" Jul 13 18:33:37.879341 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.878907 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/572dea7e-ab24-47f1-9e10-e099d9e0316e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-77f95c75bb-4mxf7\" (UID: \"572dea7e-ab24-47f1-9e10-e099d9e0316e\") " pod="openshift-monitoring/thanos-querier-77f95c75bb-4mxf7" Jul 13 18:33:37.882234 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.880022 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/572dea7e-ab24-47f1-9e10-e099d9e0316e-metrics-client-ca\") pod \"thanos-querier-77f95c75bb-4mxf7\" (UID: \"572dea7e-ab24-47f1-9e10-e099d9e0316e\") " pod="openshift-monitoring/thanos-querier-77f95c75bb-4mxf7" Jul 13 18:33:37.882234 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.882180 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/572dea7e-ab24-47f1-9e10-e099d9e0316e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-77f95c75bb-4mxf7\" (UID: \"572dea7e-ab24-47f1-9e10-e099d9e0316e\") " pod="openshift-monitoring/thanos-querier-77f95c75bb-4mxf7" Jul 13 18:33:37.882890 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.882863 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/572dea7e-ab24-47f1-9e10-e099d9e0316e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-77f95c75bb-4mxf7\" (UID: \"572dea7e-ab24-47f1-9e10-e099d9e0316e\") " pod="openshift-monitoring/thanos-querier-77f95c75bb-4mxf7" Jul 13 18:33:37.884538 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.884508 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/572dea7e-ab24-47f1-9e10-e099d9e0316e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-77f95c75bb-4mxf7\" (UID: \"572dea7e-ab24-47f1-9e10-e099d9e0316e\") " pod="openshift-monitoring/thanos-querier-77f95c75bb-4mxf7" Jul 13 18:33:37.884657 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.884567 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/572dea7e-ab24-47f1-9e10-e099d9e0316e-secret-thanos-querier-tls\") pod \"thanos-querier-77f95c75bb-4mxf7\" (UID: \"572dea7e-ab24-47f1-9e10-e099d9e0316e\") " pod="openshift-monitoring/thanos-querier-77f95c75bb-4mxf7" Jul 13 18:33:37.885055 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.885008 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/572dea7e-ab24-47f1-9e10-e099d9e0316e-secret-grpc-tls\") pod \"thanos-querier-77f95c75bb-4mxf7\" (UID: \"572dea7e-ab24-47f1-9e10-e099d9e0316e\") " pod="openshift-monitoring/thanos-querier-77f95c75bb-4mxf7" Jul 13 18:33:37.900525 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.899949 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/572dea7e-ab24-47f1-9e10-e099d9e0316e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-77f95c75bb-4mxf7\" (UID: \"572dea7e-ab24-47f1-9e10-e099d9e0316e\") " pod="openshift-monitoring/thanos-querier-77f95c75bb-4mxf7" Jul 13 18:33:37.900783 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.900736 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znvrq\" (UniqueName: \"kubernetes.io/projected/572dea7e-ab24-47f1-9e10-e099d9e0316e-kube-api-access-znvrq\") pod \"thanos-querier-77f95c75bb-4mxf7\" (UID: \"572dea7e-ab24-47f1-9e10-e099d9e0316e\") " pod="openshift-monitoring/thanos-querier-77f95c75bb-4mxf7" Jul 13 18:33:37.939983 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.939904 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:37.976725 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.975279 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-d2cqx"] Jul 13 18:33:37.980577 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.979943 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-77f95c75bb-4mxf7" Jul 13 18:33:37.984418 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.983192 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-d2cqx" Jul 13 18:33:37.987970 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.987948 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-x9q2p\"" Jul 13 18:33:37.992310 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.991708 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 13 18:33:37.997763 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:37.997698 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-d2cqx"] Jul 13 18:33:38.082937 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.082402 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghh72\" (UniqueName: \"kubernetes.io/projected/fb20fd4a-4270-4106-8c3a-330dd18bd732-kube-api-access-ghh72\") pod \"monitoring-plugin-5f68df7f58-d2cqx\" (UID: \"fb20fd4a-4270-4106-8c3a-330dd18bd732\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-d2cqx" Jul 13 18:33:38.082937 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.082512 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fb20fd4a-4270-4106-8c3a-330dd18bd732-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-d2cqx\" (UID: \"fb20fd4a-4270-4106-8c3a-330dd18bd732\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-d2cqx" Jul 13 18:33:38.092298 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.092261 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:38.184831 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.183886 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fb20fd4a-4270-4106-8c3a-330dd18bd732-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-d2cqx\" (UID: \"fb20fd4a-4270-4106-8c3a-330dd18bd732\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-d2cqx" Jul 13 18:33:38.184831 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.183979 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghh72\" (UniqueName: \"kubernetes.io/projected/fb20fd4a-4270-4106-8c3a-330dd18bd732-kube-api-access-ghh72\") pod \"monitoring-plugin-5f68df7f58-d2cqx\" (UID: \"fb20fd4a-4270-4106-8c3a-330dd18bd732\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-d2cqx" Jul 13 18:33:38.187929 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.187657 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fb20fd4a-4270-4106-8c3a-330dd18bd732-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-d2cqx\" (UID: \"fb20fd4a-4270-4106-8c3a-330dd18bd732\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-d2cqx" Jul 13 18:33:38.205386 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.205342 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghh72\" (UniqueName: \"kubernetes.io/projected/fb20fd4a-4270-4106-8c3a-330dd18bd732-kube-api-access-ghh72\") pod \"monitoring-plugin-5f68df7f58-d2cqx\" (UID: \"fb20fd4a-4270-4106-8c3a-330dd18bd732\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-d2cqx" Jul 13 18:33:38.223485 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.223450 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-77f95c75bb-4mxf7"] Jul 13 18:33:38.268156 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.268121 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:38.299321 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.299283 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7b5c78f7b5-j2kwg"] Jul 13 18:33:38.303918 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.303892 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7b5c78f7b5-j2kwg" Jul 13 18:33:38.308973 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.308816 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-c0apbh8f59dvj\"" Jul 13 18:33:38.308973 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.308844 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-69r4g\"" Jul 13 18:33:38.308973 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.308931 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 13 18:33:38.311358 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.311326 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 13 18:33:38.311591 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.311562 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 13 18:33:38.311726 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.311648 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 13 18:33:38.331141 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.331107 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7b5c78f7b5-j2kwg"] Jul 13 18:33:38.368423 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.368156 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-d2cqx" Jul 13 18:33:38.386342 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.386309 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2fe534d4-9744-4412-a2c1-b58d4db2a2a6-metrics-server-audit-profiles\") pod \"metrics-server-7b5c78f7b5-j2kwg\" (UID: \"2fe534d4-9744-4412-a2c1-b58d4db2a2a6\") " pod="openshift-monitoring/metrics-server-7b5c78f7b5-j2kwg" Jul 13 18:33:38.386503 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.386363 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2fe534d4-9744-4412-a2c1-b58d4db2a2a6-secret-metrics-server-client-certs\") pod \"metrics-server-7b5c78f7b5-j2kwg\" (UID: \"2fe534d4-9744-4412-a2c1-b58d4db2a2a6\") " pod="openshift-monitoring/metrics-server-7b5c78f7b5-j2kwg" Jul 13 18:33:38.386503 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.386410 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2fe534d4-9744-4412-a2c1-b58d4db2a2a6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7b5c78f7b5-j2kwg\" (UID: \"2fe534d4-9744-4412-a2c1-b58d4db2a2a6\") " pod="openshift-monitoring/metrics-server-7b5c78f7b5-j2kwg" Jul 13 18:33:38.386503 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.386464 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2fe534d4-9744-4412-a2c1-b58d4db2a2a6-secret-metrics-server-tls\") pod \"metrics-server-7b5c78f7b5-j2kwg\" (UID: \"2fe534d4-9744-4412-a2c1-b58d4db2a2a6\") " pod="openshift-monitoring/metrics-server-7b5c78f7b5-j2kwg" Jul 13 18:33:38.386698 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.386507 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xbk9\" (UniqueName: \"kubernetes.io/projected/2fe534d4-9744-4412-a2c1-b58d4db2a2a6-kube-api-access-9xbk9\") pod \"metrics-server-7b5c78f7b5-j2kwg\" (UID: \"2fe534d4-9744-4412-a2c1-b58d4db2a2a6\") " pod="openshift-monitoring/metrics-server-7b5c78f7b5-j2kwg" Jul 13 18:33:38.386698 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.386536 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2fe534d4-9744-4412-a2c1-b58d4db2a2a6-client-ca-bundle\") pod \"metrics-server-7b5c78f7b5-j2kwg\" (UID: \"2fe534d4-9744-4412-a2c1-b58d4db2a2a6\") " pod="openshift-monitoring/metrics-server-7b5c78f7b5-j2kwg" Jul 13 18:33:38.386698 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.386560 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2fe534d4-9744-4412-a2c1-b58d4db2a2a6-audit-log\") pod \"metrics-server-7b5c78f7b5-j2kwg\" (UID: \"2fe534d4-9744-4412-a2c1-b58d4db2a2a6\") " pod="openshift-monitoring/metrics-server-7b5c78f7b5-j2kwg" Jul 13 18:33:38.487654 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.487598 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2fe534d4-9744-4412-a2c1-b58d4db2a2a6-secret-metrics-server-tls\") pod \"metrics-server-7b5c78f7b5-j2kwg\" (UID: \"2fe534d4-9744-4412-a2c1-b58d4db2a2a6\") " pod="openshift-monitoring/metrics-server-7b5c78f7b5-j2kwg" Jul 13 18:33:38.487830 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.487678 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xbk9\" (UniqueName: \"kubernetes.io/projected/2fe534d4-9744-4412-a2c1-b58d4db2a2a6-kube-api-access-9xbk9\") pod \"metrics-server-7b5c78f7b5-j2kwg\" (UID: \"2fe534d4-9744-4412-a2c1-b58d4db2a2a6\") " pod="openshift-monitoring/metrics-server-7b5c78f7b5-j2kwg" Jul 13 18:33:38.487830 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.487704 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2fe534d4-9744-4412-a2c1-b58d4db2a2a6-client-ca-bundle\") pod \"metrics-server-7b5c78f7b5-j2kwg\" (UID: \"2fe534d4-9744-4412-a2c1-b58d4db2a2a6\") " pod="openshift-monitoring/metrics-server-7b5c78f7b5-j2kwg" Jul 13 18:33:38.487830 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.487726 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2fe534d4-9744-4412-a2c1-b58d4db2a2a6-audit-log\") pod \"metrics-server-7b5c78f7b5-j2kwg\" (UID: \"2fe534d4-9744-4412-a2c1-b58d4db2a2a6\") " pod="openshift-monitoring/metrics-server-7b5c78f7b5-j2kwg" Jul 13 18:33:38.487830 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.487784 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2fe534d4-9744-4412-a2c1-b58d4db2a2a6-metrics-server-audit-profiles\") pod \"metrics-server-7b5c78f7b5-j2kwg\" (UID: \"2fe534d4-9744-4412-a2c1-b58d4db2a2a6\") " pod="openshift-monitoring/metrics-server-7b5c78f7b5-j2kwg" Jul 13 18:33:38.487830 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.487818 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2fe534d4-9744-4412-a2c1-b58d4db2a2a6-secret-metrics-server-client-certs\") pod \"metrics-server-7b5c78f7b5-j2kwg\" (UID: \"2fe534d4-9744-4412-a2c1-b58d4db2a2a6\") " pod="openshift-monitoring/metrics-server-7b5c78f7b5-j2kwg" Jul 13 18:33:38.488025 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.487852 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2fe534d4-9744-4412-a2c1-b58d4db2a2a6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7b5c78f7b5-j2kwg\" (UID: \"2fe534d4-9744-4412-a2c1-b58d4db2a2a6\") " pod="openshift-monitoring/metrics-server-7b5c78f7b5-j2kwg" Jul 13 18:33:38.488811 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.488787 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2fe534d4-9744-4412-a2c1-b58d4db2a2a6-audit-log\") pod \"metrics-server-7b5c78f7b5-j2kwg\" (UID: \"2fe534d4-9744-4412-a2c1-b58d4db2a2a6\") " pod="openshift-monitoring/metrics-server-7b5c78f7b5-j2kwg" Jul 13 18:33:38.488935 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.488918 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2fe534d4-9744-4412-a2c1-b58d4db2a2a6-metrics-server-audit-profiles\") pod \"metrics-server-7b5c78f7b5-j2kwg\" (UID: \"2fe534d4-9744-4412-a2c1-b58d4db2a2a6\") " pod="openshift-monitoring/metrics-server-7b5c78f7b5-j2kwg" Jul 13 18:33:38.489033 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.488996 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2fe534d4-9744-4412-a2c1-b58d4db2a2a6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7b5c78f7b5-j2kwg\" (UID: \"2fe534d4-9744-4412-a2c1-b58d4db2a2a6\") " pod="openshift-monitoring/metrics-server-7b5c78f7b5-j2kwg" Jul 13 18:33:38.491104 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.491076 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2fe534d4-9744-4412-a2c1-b58d4db2a2a6-client-ca-bundle\") pod \"metrics-server-7b5c78f7b5-j2kwg\" (UID: \"2fe534d4-9744-4412-a2c1-b58d4db2a2a6\") " pod="openshift-monitoring/metrics-server-7b5c78f7b5-j2kwg" Jul 13 18:33:38.491212 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.491076 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2fe534d4-9744-4412-a2c1-b58d4db2a2a6-secret-metrics-server-tls\") pod \"metrics-server-7b5c78f7b5-j2kwg\" (UID: \"2fe534d4-9744-4412-a2c1-b58d4db2a2a6\") " pod="openshift-monitoring/metrics-server-7b5c78f7b5-j2kwg" Jul 13 18:33:38.491374 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.491349 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2fe534d4-9744-4412-a2c1-b58d4db2a2a6-secret-metrics-server-client-certs\") pod \"metrics-server-7b5c78f7b5-j2kwg\" (UID: \"2fe534d4-9744-4412-a2c1-b58d4db2a2a6\") " pod="openshift-monitoring/metrics-server-7b5c78f7b5-j2kwg" Jul 13 18:33:38.497837 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.497782 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:38.498954 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.498933 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xbk9\" (UniqueName: \"kubernetes.io/projected/2fe534d4-9744-4412-a2c1-b58d4db2a2a6-kube-api-access-9xbk9\") pod \"metrics-server-7b5c78f7b5-j2kwg\" (UID: \"2fe534d4-9744-4412-a2c1-b58d4db2a2a6\") " pod="openshift-monitoring/metrics-server-7b5c78f7b5-j2kwg" Jul 13 18:33:38.523799 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:33:38.523766 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod572dea7e_ab24_47f1_9e10_e099d9e0316e.slice/crio-6aad6ddab249809171de8766aa507d1d35704511c1d0053e80c513f33cb86040 WatchSource:0}: Error finding container 6aad6ddab249809171de8766aa507d1d35704511c1d0053e80c513f33cb86040: Status 404 returned error can't find the container with id 6aad6ddab249809171de8766aa507d1d35704511c1d0053e80c513f33cb86040 Jul 13 18:33:38.604976 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.604809 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:38.620503 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.614948 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7b5c78f7b5-j2kwg" Jul 13 18:33:38.677089 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.677067 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-d2cqx"] Jul 13 18:33:38.681284 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:33:38.681243 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb20fd4a_4270_4106_8c3a_330dd18bd732.slice/crio-bab6be08ef710a1935bc213444943987a9f14d2237c24086b907b90baeafd772 WatchSource:0}: Error finding container bab6be08ef710a1935bc213444943987a9f14d2237c24086b907b90baeafd772: Status 404 returned error can't find the container with id bab6be08ef710a1935bc213444943987a9f14d2237c24086b907b90baeafd772 Jul 13 18:33:38.692414 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.691765 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-8zgr7" event={"ID":"4fa44170-99b2-4f9a-be6e-415cc519a3f5","Type":"ContainerStarted","Data":"f472eb19b90df399640920f910506aed9a0e97d858a5be4635dfc37c43607308"} Jul 13 18:33:38.692414 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.691839 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-8zgr7" event={"ID":"4fa44170-99b2-4f9a-be6e-415cc519a3f5","Type":"ContainerStarted","Data":"e4b4939f86ba6cc1ddfbbccc34d244c1e391db5321c27e64cce1dc2b9203e2c7"} Jul 13 18:33:38.692414 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.691873 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-8zgr7" event={"ID":"4fa44170-99b2-4f9a-be6e-415cc519a3f5","Type":"ContainerStarted","Data":"165243873d5f2f15c44efee1b343fe4c742c3b77f558869ab24259463131e8a0"} Jul 13 18:33:38.695789 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.695763 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f9014714-972e-429c-88bb-30b690ad1215","Type":"ContainerStarted","Data":"e25541547ffd7f9aa85f2612de213e04772148604643e1c4a706b70226466be9"} Jul 13 18:33:38.698170 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.698054 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-77f95c75bb-4mxf7" event={"ID":"572dea7e-ab24-47f1-9e10-e099d9e0316e","Type":"ContainerStarted","Data":"6aad6ddab249809171de8766aa507d1d35704511c1d0053e80c513f33cb86040"} Jul 13 18:33:38.702732 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.702132 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pg9xz" event={"ID":"cfcad9fb-77bc-460d-9ee0-fbfb5573ba07","Type":"ContainerStarted","Data":"32ebc7e4cc80fc9a9e8ef901f3c8cc19353f2f3bc0f31eccbbf544f54e934647"} Jul 13 18:33:38.702732 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.702558 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:38.704238 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.704218 2582 generic.go:356] "Generic (PLEG): container finished" podID="58ca72a5-ee9b-44f0-9cf2-c503f37d929b" containerID="7d48b8bfd63eae1d772638d7a2abc60ff0336e3dc6614a08454c55e054079fb1" exitCode=0 Jul 13 18:33:38.704573 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.704547 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zd5xm" event={"ID":"58ca72a5-ee9b-44f0-9cf2-c503f37d929b","Type":"ContainerDied","Data":"7d48b8bfd63eae1d772638d7a2abc60ff0336e3dc6614a08454c55e054079fb1"} Jul 13 18:33:38.708373 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.708344 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-vrgqz" event={"ID":"b6d0ba1f-a0cd-490b-9a6a-1c573bdffc5e","Type":"ContainerStarted","Data":"a0190fad3d3b06750544abeb4ae1ac5defb7101d37cbd1fe1bcb5307579f5446"} Jul 13 18:33:38.740962 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.739662 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-8zgr7" podStartSLOduration=2.422331902 podStartE2EDuration="5.739645538s" totalImagesPullingTime="3.317313636s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:33:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:33:38.714827181 +0000 UTC m=+74.998255786" watchObservedRunningTime="2026-07-13 18:33:38.739645538 +0000 UTC m=+75.023074131" Jul 13 18:33:38.740962 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.740741 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-vrgqz" podStartSLOduration=2.890077283 podStartE2EDuration="5.7407308s" totalImagesPullingTime="2.850653517s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:33:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:33:38.739248604 +0000 UTC m=+75.022677233" watchObservedRunningTime="2026-07-13 18:33:38.7407308 +0000 UTC m=+75.024159405" Jul 13 18:33:38.800833 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.799961 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7b5c78f7b5-j2kwg"] Jul 13 18:33:38.804131 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:33:38.804104 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fe534d4_9744_4412_a2c1_b58d4db2a2a6.slice/crio-346be11129b876301c1579e6081511459ac76e73e97cc0d40b74527813a24e7e WatchSource:0}: Error finding container 346be11129b876301c1579e6081511459ac76e73e97cc0d40b74527813a24e7e: Status 404 returned error can't find the container with id 346be11129b876301c1579e6081511459ac76e73e97cc0d40b74527813a24e7e Jul 13 18:33:38.808981 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.807819 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-pg9xz" podStartSLOduration=66.77101807 podStartE2EDuration="1m14.807797992s" totalImagesPullingTime="8.036779922s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:32:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:33:38.805312426 +0000 UTC m=+75.088741053" watchObservedRunningTime="2026-07-13 18:33:38.807797992 +0000 UTC m=+75.091226595" Jul 13 18:33:38.815010 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.814984 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:38.889183 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.889153 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:38.994922 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:38.994842 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:39.092850 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:39.092816 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:39.720733 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:39.720689 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zd5xm" event={"ID":"58ca72a5-ee9b-44f0-9cf2-c503f37d929b","Type":"ContainerStarted","Data":"f7ad61629d9d50fd05390d8517b53dff8a076a4f9ba8349c01022e52dbc12898"} Jul 13 18:33:39.721134 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:39.720803 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zd5xm" event={"ID":"58ca72a5-ee9b-44f0-9cf2-c503f37d929b","Type":"ContainerStarted","Data":"f4d475d62c3c3c3357adcf881229a8e836976bae2960337918afacfa6c8bb9db"} Jul 13 18:33:39.724478 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:39.724403 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7b5c78f7b5-j2kwg" event={"ID":"2fe534d4-9744-4412-a2c1-b58d4db2a2a6","Type":"ContainerStarted","Data":"346be11129b876301c1579e6081511459ac76e73e97cc0d40b74527813a24e7e"} Jul 13 18:33:39.727097 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:39.727069 2582 generic.go:356] "Generic (PLEG): container finished" podID="f9014714-972e-429c-88bb-30b690ad1215" containerID="e25541547ffd7f9aa85f2612de213e04772148604643e1c4a706b70226466be9" exitCode=0 Jul 13 18:33:39.727243 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:39.727158 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f9014714-972e-429c-88bb-30b690ad1215","Type":"ContainerDied","Data":"e25541547ffd7f9aa85f2612de213e04772148604643e1c4a706b70226466be9"} Jul 13 18:33:39.731223 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:39.731195 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-d2cqx" event={"ID":"fb20fd4a-4270-4106-8c3a-330dd18bd732","Type":"ContainerStarted","Data":"bab6be08ef710a1935bc213444943987a9f14d2237c24086b907b90baeafd772"} Jul 13 18:33:39.746729 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:39.746471 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-zd5xm" podStartSLOduration=3.845042789 podStartE2EDuration="6.746449894s" totalImagesPullingTime="2.760716959s" totalInitContainerRuntime="140.690146ms" isStatefulPod=true podCreationTimestamp="2026-07-13 18:33:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:33:39.74433267 +0000 UTC m=+76.027761290" watchObservedRunningTime="2026-07-13 18:33:39.746449894 +0000 UTC m=+76.029878498" Jul 13 18:33:40.012580 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.012445 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 18:33:40.020638 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.020583 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.032099 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.032074 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 13 18:33:40.032227 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.032164 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 13 18:33:40.032346 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.032330 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 13 18:33:40.032720 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.032703 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 13 18:33:40.033944 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.033818 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-k8pkb\"" Jul 13 18:33:40.034062 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.033962 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 13 18:33:40.034062 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.034052 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 13 18:33:40.034281 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.034178 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 13 18:33:40.034281 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.034185 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 13 18:33:40.034534 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.034443 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 13 18:33:40.034700 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.034681 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 13 18:33:40.036149 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.035201 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 13 18:33:40.036149 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.035413 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 13 18:33:40.036149 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.035590 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-dfpqs9oek6f8p\"" Jul 13 18:33:40.039822 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.039801 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 13 18:33:40.051487 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.050714 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 13 18:33:40.055182 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.055157 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 18:33:40.093461 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.093433 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:40.108657 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.107804 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-config\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.108657 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.107849 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.108657 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.107882 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.108657 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.107931 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.108657 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.107959 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.108657 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.107985 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.108657 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.108011 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.108657 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.108043 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.108657 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.108073 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.108657 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.108109 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.108657 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.108135 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.108657 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.108176 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.108657 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.108205 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.108657 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.108230 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.108657 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.108257 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.108657 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.108284 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-web-config\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.109826 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.108311 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.109826 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.108336 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.109826 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.108366 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-config-out\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.109826 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.108411 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzz7r\" (UniqueName: \"kubernetes.io/projected/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-kube-api-access-lzz7r\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.209461 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.209425 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.210142 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.209480 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.212461 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.211297 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.213165 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.213095 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.214851 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.213352 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.214851 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.213401 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.214851 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.213438 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.214851 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.213479 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.214851 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.213507 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.214851 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.213541 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.214851 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.213571 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.214851 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.214238 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.216230 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.215852 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.216381 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.213598 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.221632 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.219279 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.221632 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.219390 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.221632 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.219726 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.221632 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.219793 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.221632 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.219835 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-web-config\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.221632 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.219874 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.221632 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.219909 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.221632 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.219932 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.221632 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.219951 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-config-out\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.221632 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.220004 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzz7r\" (UniqueName: \"kubernetes.io/projected/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-kube-api-access-lzz7r\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.221632 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.220051 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-config\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.221632 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.220081 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.221632 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.220121 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.221632 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.221194 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.222301 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.221862 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.222301 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.221991 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.222984 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.222545 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.224750 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.224728 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.226440 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.226414 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.231687 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.231664 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.231784 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.231711 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.233707 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.233661 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzz7r\" (UniqueName: \"kubernetes.io/projected/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-kube-api-access-lzz7r\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.234975 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.234895 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-web-config\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.235756 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.235712 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.235918 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.235903 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-config\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.237147 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.237125 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-config-out\") pod \"prometheus-k8s-0\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:40.340920 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:40.340826 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:42.351937 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:33:42.351869 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8050c7ee_0e9d_40c6_83f2_77d25bc9a489.slice/crio-ffdad66875774d9bd12ca101a7213f329b8edd663dba12d3cdee4e0fa39173a5 WatchSource:0}: Error finding container ffdad66875774d9bd12ca101a7213f329b8edd663dba12d3cdee4e0fa39173a5: Status 404 returned error can't find the container with id ffdad66875774d9bd12ca101a7213f329b8edd663dba12d3cdee4e0fa39173a5 Jul 13 18:33:42.352394 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:42.352073 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 18:33:42.746395 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:42.746356 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7b5c78f7b5-j2kwg" event={"ID":"2fe534d4-9744-4412-a2c1-b58d4db2a2a6","Type":"ContainerStarted","Data":"c283c5e5dbed4548bfc6608c57a6a148fc80dc975494693234765713ff158be9"} Jul 13 18:33:42.749539 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:42.749508 2582 generic.go:356] "Generic (PLEG): container finished" podID="8050c7ee-0e9d-40c6-83f2-77d25bc9a489" containerID="d4369c51d014bca41d39a2e9fa76824d9f183217ed3680793f6bcd2ae2da4f05" exitCode=0 Jul 13 18:33:42.749709 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:42.749581 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8050c7ee-0e9d-40c6-83f2-77d25bc9a489","Type":"ContainerDied","Data":"d4369c51d014bca41d39a2e9fa76824d9f183217ed3680793f6bcd2ae2da4f05"} Jul 13 18:33:42.749709 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:42.749676 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8050c7ee-0e9d-40c6-83f2-77d25bc9a489","Type":"ContainerStarted","Data":"ffdad66875774d9bd12ca101a7213f329b8edd663dba12d3cdee4e0fa39173a5"} Jul 13 18:33:42.758249 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:42.758226 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f9014714-972e-429c-88bb-30b690ad1215","Type":"ContainerStarted","Data":"6a0d016160eafa3f0df1e5fa81e760b4566f93c22afbf89bbc141236a73ad3c5"} Jul 13 18:33:42.758354 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:42.758291 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f9014714-972e-429c-88bb-30b690ad1215","Type":"ContainerStarted","Data":"6e2235884fe229434b3927cdb32ce26ce706a900da324290dd7638ced9a18d13"} Jul 13 18:33:42.758354 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:42.758321 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f9014714-972e-429c-88bb-30b690ad1215","Type":"ContainerStarted","Data":"8fe7be7547302c0b9cfd74ef0cbba8f61ff9f9e5e8a18e0291da41ce3a8e9f8f"} Jul 13 18:33:42.758472 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:42.758354 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f9014714-972e-429c-88bb-30b690ad1215","Type":"ContainerStarted","Data":"36ac697bf827331bdc45f7ad2575984bc98641276cf9c8f4cb622ebf43e55a78"} Jul 13 18:33:42.758472 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:42.758394 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f9014714-972e-429c-88bb-30b690ad1215","Type":"ContainerStarted","Data":"c8b460f9aed25f01ce659364f60fc50b770ba4a762cbd3d0c501bbb592760a61"} Jul 13 18:33:42.761846 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:42.761821 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-77f95c75bb-4mxf7" event={"ID":"572dea7e-ab24-47f1-9e10-e099d9e0316e","Type":"ContainerStarted","Data":"4a674a766636c838a81df8c48e6d7639b8a7a96cabfe76f6542e5eef304fa322"} Jul 13 18:33:42.761955 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:42.761932 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-77f95c75bb-4mxf7" event={"ID":"572dea7e-ab24-47f1-9e10-e099d9e0316e","Type":"ContainerStarted","Data":"ebfead398bed61191a1d703519a72361320020ede8caa3c217786f175972fe10"} Jul 13 18:33:42.762029 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:42.761995 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-77f95c75bb-4mxf7" event={"ID":"572dea7e-ab24-47f1-9e10-e099d9e0316e","Type":"ContainerStarted","Data":"3823e3e730f5eb829b2a476536d197534aae4df6402a3ed9534d3967bc679775"} Jul 13 18:33:42.763710 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:42.763686 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-d2cqx" event={"ID":"fb20fd4a-4270-4106-8c3a-330dd18bd732","Type":"ContainerStarted","Data":"47ef509de3c87d6f1b879bb836d6de8a4b7987747a355a33961638b4ecf65d8d"} Jul 13 18:33:42.763993 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:42.763973 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-d2cqx" Jul 13 18:33:42.770051 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:42.770032 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-d2cqx" Jul 13 18:33:42.780974 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:42.780921 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7b5c78f7b5-j2kwg" podStartSLOduration=1.445880918 podStartE2EDuration="4.78090336s" totalImagesPullingTime="3.335022442s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:33:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:33:42.780173935 +0000 UTC m=+79.063602557" watchObservedRunningTime="2026-07-13 18:33:42.78090336 +0000 UTC m=+79.064331964" Jul 13 18:33:42.880397 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:42.880341 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-d2cqx" podStartSLOduration=2.421146616 podStartE2EDuration="5.880318324s" totalImagesPullingTime="3.459171708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:33:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:33:42.878400791 +0000 UTC m=+79.161829415" watchObservedRunningTime="2026-07-13 18:33:42.880318324 +0000 UTC m=+79.163746928" Jul 13 18:33:42.958661 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:42.958623 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:43.115625 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:43.115562 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:43.367739 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:43.367654 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:43.431336 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:43.431300 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:43.775649 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:43.775508 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f9014714-972e-429c-88bb-30b690ad1215","Type":"ContainerStarted","Data":"2736c08c69f544b9926ee99dc86908cc01932f0d052c3fd6549b73221125ec00"} Jul 13 18:33:43.780806 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:43.780650 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-77f95c75bb-4mxf7" event={"ID":"572dea7e-ab24-47f1-9e10-e099d9e0316e","Type":"ContainerStarted","Data":"89150c96ffe65978e2419c9e97adbe5f1e0fe07039971fad648fe7b42798d950"} Jul 13 18:33:43.780806 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:43.780719 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-77f95c75bb-4mxf7" event={"ID":"572dea7e-ab24-47f1-9e10-e099d9e0316e","Type":"ContainerStarted","Data":"76ad6830c9e153cadcdc2a55844f8173087fb544d85a4e0c46e1052677881099"} Jul 13 18:33:43.780806 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:43.780765 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-77f95c75bb-4mxf7" event={"ID":"572dea7e-ab24-47f1-9e10-e099d9e0316e","Type":"ContainerStarted","Data":"739f185c6de4cb4058a33f80d0e8cb4933a3131202a563c85eac1e1c947c4502"} Jul 13 18:33:43.781275 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:43.781252 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-77f95c75bb-4mxf7" Jul 13 18:33:43.807985 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:43.807922 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.8707354689999995 podStartE2EDuration="9.807897483s" totalImagesPullingTime="4.913465602s" totalInitContainerRuntime="23.696412ms" isStatefulPod=false podCreationTimestamp="2026-07-13 18:33:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:33:43.805369288 +0000 UTC m=+80.088797928" watchObservedRunningTime="2026-07-13 18:33:43.807897483 +0000 UTC m=+80.091326086" Jul 13 18:33:43.837509 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:43.837382 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-77f95c75bb-4mxf7" podStartSLOduration=2.18633561 podStartE2EDuration="6.837357839s" totalImagesPullingTime="4.651022229s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:33:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:33:43.836118239 +0000 UTC m=+80.119546840" watchObservedRunningTime="2026-07-13 18:33:43.837357839 +0000 UTC m=+80.120786440" Jul 13 18:33:44.374904 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:44.374860 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:44.474956 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:44.474908 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:46.800865 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:46.800802 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8050c7ee-0e9d-40c6-83f2-77d25bc9a489","Type":"ContainerStarted","Data":"eccd5d4e6897f09c8be80c131764138dd27e3d9c0ac31a2982ad143e3c7fa759"} Jul 13 18:33:46.801458 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:46.800899 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8050c7ee-0e9d-40c6-83f2-77d25bc9a489","Type":"ContainerStarted","Data":"6e4f30459a83f48105159942a7d83059acd29726fcecd515fd5a938b8ef8701c"} Jul 13 18:33:46.801458 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:46.800931 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8050c7ee-0e9d-40c6-83f2-77d25bc9a489","Type":"ContainerStarted","Data":"a26f638c65ac847a5a1052dec0b54c8d1a7a562168dd7c6064a92eb61fddfadd"} Jul 13 18:33:46.801458 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:46.800954 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8050c7ee-0e9d-40c6-83f2-77d25bc9a489","Type":"ContainerStarted","Data":"b5a69a141579ef333f1cbcab93e6e86ab9f3664ea1f1d073ea6315e397da6885"} Jul 13 18:33:46.801458 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:46.800988 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8050c7ee-0e9d-40c6-83f2-77d25bc9a489","Type":"ContainerStarted","Data":"5926bd79c577f78a94262909846a2d3038c1ec65a8eaf4c1173bb2a0f4518a40"} Jul 13 18:33:46.801458 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:46.801019 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8050c7ee-0e9d-40c6-83f2-77d25bc9a489","Type":"ContainerStarted","Data":"bd438e630d228d1bdc2fe77d7725a3a8aa5bae94de56b69e6e43aa6b4c7ad3a1"} Jul 13 18:33:46.838672 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:46.838510 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.22830378 podStartE2EDuration="7.83849223s" totalImagesPullingTime="3.57642s" totalInitContainerRuntime="33.76845ms" isStatefulPod=false podCreationTimestamp="2026-07-13 18:33:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:33:46.836465447 +0000 UTC m=+83.119894070" watchObservedRunningTime="2026-07-13 18:33:46.83849223 +0000 UTC m=+83.121920832" Jul 13 18:33:47.179126 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:47.179088 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:47.252406 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:47.252373 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:49.970473 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:49.970426 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:50.070131 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:50.070095 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:50.173062 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:50.173024 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:50.271839 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:50.271758 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:50.341693 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:50.341657 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:33:50.798837 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:50.798811 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-77f95c75bb-4mxf7" Jul 13 18:33:55.402666 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:55.402622 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:55.468099 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:55.468054 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:57.856926 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:57.856880 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-bggrq" event={"ID":"e657c3c1-fb9d-4da0-b88f-6db7f2510f95","Type":"ContainerStarted","Data":"e5e14e2db87c0719d98568b043c6f7bd9524cdb0aed601a3a561a84fafc9e80b"} Jul 13 18:33:57.857372 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:57.857119 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-bggrq" Jul 13 18:33:57.871746 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:57.871720 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:57.875140 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:57.875112 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-bggrq" Jul 13 18:33:57.880960 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:57.880910 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-bggrq" podStartSLOduration=2.560405171 podStartE2EDuration="24.880896562s" totalImagesPullingTime="22.320491391s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:33:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:33:57.878055428 +0000 UTC m=+94.161484057" watchObservedRunningTime="2026-07-13 18:33:57.880896562 +0000 UTC m=+94.164325165" Jul 13 18:33:57.968255 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:57.968224 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:58.166799 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:58.166684 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:58.247951 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:58.247889 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:33:58.615522 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:58.615486 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7b5c78f7b5-j2kwg" Jul 13 18:33:58.615711 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:33:58.615564 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7b5c78f7b5-j2kwg" Jul 13 18:34:05.284740 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:05.284703 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:34:05.410230 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:05.410191 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:34:05.531091 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:05.531039 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:34:05.673630 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:05.673532 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-r24p4" Jul 13 18:34:10.664540 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:10.664495 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:34:10.852962 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:10.852917 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:34:11.177898 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:11.177862 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:34:11.273169 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:11.273133 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:34:11.783972 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:11.783938 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:34:11.862266 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:11.862224 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:34:12.860656 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:12.860625 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:34:12.961508 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:12.961465 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:34:18.620988 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:18.620959 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7b5c78f7b5-j2kwg" Jul 13 18:34:18.624806 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:18.624784 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7b5c78f7b5-j2kwg" Jul 13 18:34:40.341761 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:40.341727 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:34:40.394909 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:40.394884 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:34:41.055496 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:41.055471 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:34:53.958055 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:53.958021 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 18:34:53.958784 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:53.958330 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f9014714-972e-429c-88bb-30b690ad1215" containerName="alertmanager" containerID="cri-o://c8b460f9aed25f01ce659364f60fc50b770ba4a762cbd3d0c501bbb592760a61" gracePeriod=120 Jul 13 18:34:53.958784 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:53.958358 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f9014714-972e-429c-88bb-30b690ad1215" containerName="kube-rbac-proxy" containerID="cri-o://6e2235884fe229434b3927cdb32ce26ce706a900da324290dd7638ced9a18d13" gracePeriod=120 Jul 13 18:34:53.958784 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:53.958401 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f9014714-972e-429c-88bb-30b690ad1215" containerName="prom-label-proxy" containerID="cri-o://2736c08c69f544b9926ee99dc86908cc01932f0d052c3fd6549b73221125ec00" gracePeriod=120 Jul 13 18:34:53.958784 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:53.958452 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f9014714-972e-429c-88bb-30b690ad1215" containerName="config-reloader" containerID="cri-o://36ac697bf827331bdc45f7ad2575984bc98641276cf9c8f4cb622ebf43e55a78" gracePeriod=120 Jul 13 18:34:53.958784 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:53.958401 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f9014714-972e-429c-88bb-30b690ad1215" containerName="kube-rbac-proxy-web" containerID="cri-o://8fe7be7547302c0b9cfd74ef0cbba8f61ff9f9e5e8a18e0291da41ce3a8e9f8f" gracePeriod=120 Jul 13 18:34:53.958784 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:53.958430 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f9014714-972e-429c-88bb-30b690ad1215" containerName="kube-rbac-proxy-metric" containerID="cri-o://6a0d016160eafa3f0df1e5fa81e760b4566f93c22afbf89bbc141236a73ad3c5" gracePeriod=120 Jul 13 18:34:53.995959 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:53.995927 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:34:54.058242 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:54.058217 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:34:54.097846 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:54.097816 2582 generic.go:356] "Generic (PLEG): container finished" podID="f9014714-972e-429c-88bb-30b690ad1215" containerID="2736c08c69f544b9926ee99dc86908cc01932f0d052c3fd6549b73221125ec00" exitCode=0 Jul 13 18:34:54.097846 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:54.097839 2582 generic.go:356] "Generic (PLEG): container finished" podID="f9014714-972e-429c-88bb-30b690ad1215" containerID="6e2235884fe229434b3927cdb32ce26ce706a900da324290dd7638ced9a18d13" exitCode=0 Jul 13 18:34:54.097846 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:54.097845 2582 generic.go:356] "Generic (PLEG): container finished" podID="f9014714-972e-429c-88bb-30b690ad1215" containerID="36ac697bf827331bdc45f7ad2575984bc98641276cf9c8f4cb622ebf43e55a78" exitCode=0 Jul 13 18:34:54.097846 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:54.097851 2582 generic.go:356] "Generic (PLEG): container finished" podID="f9014714-972e-429c-88bb-30b690ad1215" containerID="c8b460f9aed25f01ce659364f60fc50b770ba4a762cbd3d0c501bbb592760a61" exitCode=0 Jul 13 18:34:54.098051 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:54.097885 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f9014714-972e-429c-88bb-30b690ad1215","Type":"ContainerDied","Data":"2736c08c69f544b9926ee99dc86908cc01932f0d052c3fd6549b73221125ec00"} Jul 13 18:34:54.098051 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:54.097971 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f9014714-972e-429c-88bb-30b690ad1215","Type":"ContainerDied","Data":"6e2235884fe229434b3927cdb32ce26ce706a900da324290dd7638ced9a18d13"} Jul 13 18:34:54.098051 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:54.098002 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f9014714-972e-429c-88bb-30b690ad1215","Type":"ContainerDied","Data":"36ac697bf827331bdc45f7ad2575984bc98641276cf9c8f4cb622ebf43e55a78"} Jul 13 18:34:54.098051 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:54.098027 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f9014714-972e-429c-88bb-30b690ad1215","Type":"ContainerDied","Data":"c8b460f9aed25f01ce659364f60fc50b770ba4a762cbd3d0c501bbb592760a61"} Jul 13 18:34:55.105712 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.105676 2582 generic.go:356] "Generic (PLEG): container finished" podID="f9014714-972e-429c-88bb-30b690ad1215" containerID="6a0d016160eafa3f0df1e5fa81e760b4566f93c22afbf89bbc141236a73ad3c5" exitCode=0 Jul 13 18:34:55.105712 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.105703 2582 generic.go:356] "Generic (PLEG): container finished" podID="f9014714-972e-429c-88bb-30b690ad1215" containerID="8fe7be7547302c0b9cfd74ef0cbba8f61ff9f9e5e8a18e0291da41ce3a8e9f8f" exitCode=0 Jul 13 18:34:55.106328 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.105757 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f9014714-972e-429c-88bb-30b690ad1215","Type":"ContainerDied","Data":"6a0d016160eafa3f0df1e5fa81e760b4566f93c22afbf89bbc141236a73ad3c5"} Jul 13 18:34:55.106328 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.105902 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f9014714-972e-429c-88bb-30b690ad1215","Type":"ContainerDied","Data":"8fe7be7547302c0b9cfd74ef0cbba8f61ff9f9e5e8a18e0291da41ce3a8e9f8f"} Jul 13 18:34:55.214698 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.214675 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:55.302632 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.302530 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-config-volume\") pod \"f9014714-972e-429c-88bb-30b690ad1215\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " Jul 13 18:34:55.302632 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.302566 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82ksp\" (UniqueName: \"kubernetes.io/projected/f9014714-972e-429c-88bb-30b690ad1215-kube-api-access-82ksp\") pod \"f9014714-972e-429c-88bb-30b690ad1215\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " Jul 13 18:34:55.302632 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.302597 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-secret-alertmanager-kube-rbac-proxy\") pod \"f9014714-972e-429c-88bb-30b690ad1215\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " Jul 13 18:34:55.302903 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.302657 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-secret-alertmanager-kube-rbac-proxy-web\") pod \"f9014714-972e-429c-88bb-30b690ad1215\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " Jul 13 18:34:55.302903 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.302694 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-secret-alertmanager-main-tls\") pod \"f9014714-972e-429c-88bb-30b690ad1215\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " Jul 13 18:34:55.302903 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.302717 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f9014714-972e-429c-88bb-30b690ad1215-config-out\") pod \"f9014714-972e-429c-88bb-30b690ad1215\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " Jul 13 18:34:55.302903 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.302762 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f9014714-972e-429c-88bb-30b690ad1215-tls-assets\") pod \"f9014714-972e-429c-88bb-30b690ad1215\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " Jul 13 18:34:55.302903 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.302803 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9014714-972e-429c-88bb-30b690ad1215-alertmanager-trusted-ca-bundle\") pod \"f9014714-972e-429c-88bb-30b690ad1215\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " Jul 13 18:34:55.302903 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.302835 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-cluster-tls-config\") pod \"f9014714-972e-429c-88bb-30b690ad1215\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " Jul 13 18:34:55.302903 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.302866 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-web-config\") pod \"f9014714-972e-429c-88bb-30b690ad1215\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " Jul 13 18:34:55.303259 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.302910 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f9014714-972e-429c-88bb-30b690ad1215-metrics-client-ca\") pod \"f9014714-972e-429c-88bb-30b690ad1215\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " Jul 13 18:34:55.303259 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.302953 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f9014714-972e-429c-88bb-30b690ad1215-alertmanager-main-db\") pod \"f9014714-972e-429c-88bb-30b690ad1215\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " Jul 13 18:34:55.303259 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.302984 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-secret-alertmanager-kube-rbac-proxy-metric\") pod \"f9014714-972e-429c-88bb-30b690ad1215\" (UID: \"f9014714-972e-429c-88bb-30b690ad1215\") " Jul 13 18:34:55.304064 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.303976 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9014714-972e-429c-88bb-30b690ad1215-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "f9014714-972e-429c-88bb-30b690ad1215" (UID: "f9014714-972e-429c-88bb-30b690ad1215"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 18:34:55.304351 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.304324 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9014714-972e-429c-88bb-30b690ad1215-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "f9014714-972e-429c-88bb-30b690ad1215" (UID: "f9014714-972e-429c-88bb-30b690ad1215"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 18:34:55.304756 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.304729 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9014714-972e-429c-88bb-30b690ad1215-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "f9014714-972e-429c-88bb-30b690ad1215" (UID: "f9014714-972e-429c-88bb-30b690ad1215"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 18:34:55.305869 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.305839 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "f9014714-972e-429c-88bb-30b690ad1215" (UID: "f9014714-972e-429c-88bb-30b690ad1215"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 18:34:55.306230 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.306204 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "f9014714-972e-429c-88bb-30b690ad1215" (UID: "f9014714-972e-429c-88bb-30b690ad1215"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 18:34:55.306577 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.306531 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "f9014714-972e-429c-88bb-30b690ad1215" (UID: "f9014714-972e-429c-88bb-30b690ad1215"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 18:34:55.306705 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.306542 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9014714-972e-429c-88bb-30b690ad1215-kube-api-access-82ksp" (OuterVolumeSpecName: "kube-api-access-82ksp") pod "f9014714-972e-429c-88bb-30b690ad1215" (UID: "f9014714-972e-429c-88bb-30b690ad1215"). InnerVolumeSpecName "kube-api-access-82ksp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 18:34:55.306956 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.306928 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-config-volume" (OuterVolumeSpecName: "config-volume") pod "f9014714-972e-429c-88bb-30b690ad1215" (UID: "f9014714-972e-429c-88bb-30b690ad1215"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 18:34:55.307195 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.307159 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9014714-972e-429c-88bb-30b690ad1215-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "f9014714-972e-429c-88bb-30b690ad1215" (UID: "f9014714-972e-429c-88bb-30b690ad1215"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 18:34:55.307317 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.307193 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9014714-972e-429c-88bb-30b690ad1215-config-out" (OuterVolumeSpecName: "config-out") pod "f9014714-972e-429c-88bb-30b690ad1215" (UID: "f9014714-972e-429c-88bb-30b690ad1215"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 18:34:55.307693 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.307670 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "f9014714-972e-429c-88bb-30b690ad1215" (UID: "f9014714-972e-429c-88bb-30b690ad1215"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 18:34:55.311339 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.311271 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "f9014714-972e-429c-88bb-30b690ad1215" (UID: "f9014714-972e-429c-88bb-30b690ad1215"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 18:34:55.317795 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.317773 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-web-config" (OuterVolumeSpecName: "web-config") pod "f9014714-972e-429c-88bb-30b690ad1215" (UID: "f9014714-972e-429c-88bb-30b690ad1215"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 18:34:55.404203 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.404175 2582 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f9014714-972e-429c-88bb-30b690ad1215-tls-assets\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:34:55.404203 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.404205 2582 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9014714-972e-429c-88bb-30b690ad1215-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:34:55.404428 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.404219 2582 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-cluster-tls-config\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:34:55.404428 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.404255 2582 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-web-config\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:34:55.404428 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.404267 2582 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f9014714-972e-429c-88bb-30b690ad1215-metrics-client-ca\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:34:55.404428 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.404278 2582 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f9014714-972e-429c-88bb-30b690ad1215-alertmanager-main-db\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:34:55.404428 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.404291 2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:34:55.404428 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.404303 2582 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-config-volume\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:34:55.404428 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.404314 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82ksp\" (UniqueName: \"kubernetes.io/projected/f9014714-972e-429c-88bb-30b690ad1215-kube-api-access-82ksp\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:34:55.404428 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.404325 2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:34:55.404428 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.404338 2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:34:55.404428 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.404351 2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f9014714-972e-429c-88bb-30b690ad1215-secret-alertmanager-main-tls\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:34:55.404428 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:55.404363 2582 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f9014714-972e-429c-88bb-30b690ad1215-config-out\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:34:56.113226 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.113192 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f9014714-972e-429c-88bb-30b690ad1215","Type":"ContainerDied","Data":"eba088ade4fbfe5f08ceb32d81417b78b1bf1c2940a7b4a2afc122d635082aaf"} Jul 13 18:34:56.113226 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.113227 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.113688 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.113273 2582 scope.go:117] "RemoveContainer" containerID="2736c08c69f544b9926ee99dc86908cc01932f0d052c3fd6549b73221125ec00" Jul 13 18:34:56.121638 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.121591 2582 scope.go:117] "RemoveContainer" containerID="6a0d016160eafa3f0df1e5fa81e760b4566f93c22afbf89bbc141236a73ad3c5" Jul 13 18:34:56.128737 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.128719 2582 scope.go:117] "RemoveContainer" containerID="6e2235884fe229434b3927cdb32ce26ce706a900da324290dd7638ced9a18d13" Jul 13 18:34:56.135374 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.135358 2582 scope.go:117] "RemoveContainer" containerID="8fe7be7547302c0b9cfd74ef0cbba8f61ff9f9e5e8a18e0291da41ce3a8e9f8f" Jul 13 18:34:56.141859 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.141841 2582 scope.go:117] "RemoveContainer" containerID="36ac697bf827331bdc45f7ad2575984bc98641276cf9c8f4cb622ebf43e55a78" Jul 13 18:34:56.148421 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.148403 2582 scope.go:117] "RemoveContainer" containerID="c8b460f9aed25f01ce659364f60fc50b770ba4a762cbd3d0c501bbb592760a61" Jul 13 18:34:56.155696 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.155681 2582 scope.go:117] "RemoveContainer" containerID="e25541547ffd7f9aa85f2612de213e04772148604643e1c4a706b70226466be9" Jul 13 18:34:56.177199 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.177173 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 18:34:56.190053 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.190029 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 18:34:56.249411 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.249384 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 18:34:56.249729 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.249715 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9014714-972e-429c-88bb-30b690ad1215" containerName="kube-rbac-proxy-web" Jul 13 18:34:56.249778 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.249730 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9014714-972e-429c-88bb-30b690ad1215" containerName="kube-rbac-proxy-web" Jul 13 18:34:56.249778 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.249740 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9014714-972e-429c-88bb-30b690ad1215" containerName="alertmanager" Jul 13 18:34:56.249778 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.249745 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9014714-972e-429c-88bb-30b690ad1215" containerName="alertmanager" Jul 13 18:34:56.249778 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.249754 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9014714-972e-429c-88bb-30b690ad1215" containerName="kube-rbac-proxy-metric" Jul 13 18:34:56.249778 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.249761 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9014714-972e-429c-88bb-30b690ad1215" containerName="kube-rbac-proxy-metric" Jul 13 18:34:56.249778 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.249770 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9014714-972e-429c-88bb-30b690ad1215" containerName="init-config-reloader" Jul 13 18:34:56.249778 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.249776 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9014714-972e-429c-88bb-30b690ad1215" containerName="init-config-reloader" Jul 13 18:34:56.249983 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.249785 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9014714-972e-429c-88bb-30b690ad1215" containerName="kube-rbac-proxy" Jul 13 18:34:56.249983 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.249790 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9014714-972e-429c-88bb-30b690ad1215" containerName="kube-rbac-proxy" Jul 13 18:34:56.249983 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.249797 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9014714-972e-429c-88bb-30b690ad1215" containerName="prom-label-proxy" Jul 13 18:34:56.249983 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.249802 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9014714-972e-429c-88bb-30b690ad1215" containerName="prom-label-proxy" Jul 13 18:34:56.249983 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.249812 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9014714-972e-429c-88bb-30b690ad1215" containerName="config-reloader" Jul 13 18:34:56.249983 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.249817 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9014714-972e-429c-88bb-30b690ad1215" containerName="config-reloader" Jul 13 18:34:56.249983 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.249866 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9014714-972e-429c-88bb-30b690ad1215" containerName="prom-label-proxy" Jul 13 18:34:56.249983 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.249875 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9014714-972e-429c-88bb-30b690ad1215" containerName="alertmanager" Jul 13 18:34:56.249983 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.249881 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9014714-972e-429c-88bb-30b690ad1215" containerName="kube-rbac-proxy-web" Jul 13 18:34:56.249983 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.249888 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9014714-972e-429c-88bb-30b690ad1215" containerName="kube-rbac-proxy-metric" Jul 13 18:34:56.249983 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.249893 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9014714-972e-429c-88bb-30b690ad1215" containerName="kube-rbac-proxy" Jul 13 18:34:56.249983 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.249899 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9014714-972e-429c-88bb-30b690ad1215" containerName="config-reloader" Jul 13 18:34:56.253043 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.253025 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.257085 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.257064 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 13 18:34:56.257085 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.257064 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 13 18:34:56.257247 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.257122 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-bmbw9\"" Jul 13 18:34:56.257352 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.257331 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 13 18:34:56.257441 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.257426 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 13 18:34:56.257559 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.257543 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 13 18:34:56.257669 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.257654 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 13 18:34:56.259932 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.259835 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 13 18:34:56.261823 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.261806 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 13 18:34:56.262882 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.262866 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 13 18:34:56.273241 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.273185 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 18:34:56.281364 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.281343 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9014714-972e-429c-88bb-30b690ad1215" path="/var/lib/kubelet/pods/f9014714-972e-429c-88bb-30b690ad1215/volumes" Jul 13 18:34:56.311996 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.311970 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/975ed946-1a3d-46ea-82c6-f2f35d35721e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.312192 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.312013 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/975ed946-1a3d-46ea-82c6-f2f35d35721e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.312192 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.312036 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/975ed946-1a3d-46ea-82c6-f2f35d35721e-web-config\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.312192 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.312058 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/975ed946-1a3d-46ea-82c6-f2f35d35721e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.312192 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.312085 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/975ed946-1a3d-46ea-82c6-f2f35d35721e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.312192 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.312101 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/975ed946-1a3d-46ea-82c6-f2f35d35721e-config-volume\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.312192 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.312121 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/975ed946-1a3d-46ea-82c6-f2f35d35721e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.312192 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.312178 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/975ed946-1a3d-46ea-82c6-f2f35d35721e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.312544 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.312199 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/975ed946-1a3d-46ea-82c6-f2f35d35721e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.312544 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.312218 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sb557\" (UniqueName: \"kubernetes.io/projected/975ed946-1a3d-46ea-82c6-f2f35d35721e-kube-api-access-sb557\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.312544 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.312260 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/975ed946-1a3d-46ea-82c6-f2f35d35721e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.312544 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.312283 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/975ed946-1a3d-46ea-82c6-f2f35d35721e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.312544 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.312304 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/975ed946-1a3d-46ea-82c6-f2f35d35721e-config-out\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.413425 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.413394 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/975ed946-1a3d-46ea-82c6-f2f35d35721e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.413579 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.413434 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/975ed946-1a3d-46ea-82c6-f2f35d35721e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.413579 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.413453 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/975ed946-1a3d-46ea-82c6-f2f35d35721e-web-config\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.413579 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.413475 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/975ed946-1a3d-46ea-82c6-f2f35d35721e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.413579 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.413506 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/975ed946-1a3d-46ea-82c6-f2f35d35721e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.413579 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.413531 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/975ed946-1a3d-46ea-82c6-f2f35d35721e-config-volume\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.413579 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.413561 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/975ed946-1a3d-46ea-82c6-f2f35d35721e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.413897 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.413591 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/975ed946-1a3d-46ea-82c6-f2f35d35721e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.413897 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.413642 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/975ed946-1a3d-46ea-82c6-f2f35d35721e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.413897 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.413666 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sb557\" (UniqueName: \"kubernetes.io/projected/975ed946-1a3d-46ea-82c6-f2f35d35721e-kube-api-access-sb557\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.413897 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.413701 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/975ed946-1a3d-46ea-82c6-f2f35d35721e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.413897 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.413736 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/975ed946-1a3d-46ea-82c6-f2f35d35721e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.413897 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.413770 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/975ed946-1a3d-46ea-82c6-f2f35d35721e-config-out\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.413897 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.413883 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/975ed946-1a3d-46ea-82c6-f2f35d35721e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.415154 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.415124 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/975ed946-1a3d-46ea-82c6-f2f35d35721e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.417006 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.416671 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/975ed946-1a3d-46ea-82c6-f2f35d35721e-config-volume\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.417006 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.416679 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/975ed946-1a3d-46ea-82c6-f2f35d35721e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.417006 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.416723 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/975ed946-1a3d-46ea-82c6-f2f35d35721e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.417006 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.416947 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/975ed946-1a3d-46ea-82c6-f2f35d35721e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.417275 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.417249 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/975ed946-1a3d-46ea-82c6-f2f35d35721e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.417580 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.417551 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/975ed946-1a3d-46ea-82c6-f2f35d35721e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.417713 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.417586 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/975ed946-1a3d-46ea-82c6-f2f35d35721e-config-out\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.417945 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.417927 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/975ed946-1a3d-46ea-82c6-f2f35d35721e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.418103 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.418083 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/975ed946-1a3d-46ea-82c6-f2f35d35721e-web-config\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.418898 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.418879 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/975ed946-1a3d-46ea-82c6-f2f35d35721e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.437549 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.437522 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sb557\" (UniqueName: \"kubernetes.io/projected/975ed946-1a3d-46ea-82c6-f2f35d35721e-kube-api-access-sb557\") pod \"alertmanager-main-0\" (UID: \"975ed946-1a3d-46ea-82c6-f2f35d35721e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.562772 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.562679 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:34:56.706869 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:56.705544 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 18:34:57.119034 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:57.119002 2582 generic.go:356] "Generic (PLEG): container finished" podID="975ed946-1a3d-46ea-82c6-f2f35d35721e" containerID="402d28726bb12523803e2b6979022854c01bbc66398456ed5168ee5031ec7721" exitCode=0 Jul 13 18:34:57.119374 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:57.119050 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"975ed946-1a3d-46ea-82c6-f2f35d35721e","Type":"ContainerDied","Data":"402d28726bb12523803e2b6979022854c01bbc66398456ed5168ee5031ec7721"} Jul 13 18:34:57.119374 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:57.119096 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"975ed946-1a3d-46ea-82c6-f2f35d35721e","Type":"ContainerStarted","Data":"abf4da63a6393727969db7437d583f4fd9b7e350e498946a9341bddc8e8d512a"} Jul 13 18:34:58.100644 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.100590 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-856878f6c6-mclbd"] Jul 13 18:34:58.103133 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.103111 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-856878f6c6-mclbd" Jul 13 18:34:58.106413 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.106387 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 13 18:34:58.106531 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.106433 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 13 18:34:58.106812 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.106799 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 13 18:34:58.107770 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.107743 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-pdrw2\"" Jul 13 18:34:58.107888 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.107835 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 13 18:34:58.108170 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.108148 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 13 18:34:58.112712 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.112694 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 13 18:34:58.125793 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.125765 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b524ba2c-fcaa-437a-9cd5-64512e35cf7e-serving-certs-ca-bundle\") pod \"telemeter-client-856878f6c6-mclbd\" (UID: \"b524ba2c-fcaa-437a-9cd5-64512e35cf7e\") " pod="openshift-monitoring/telemeter-client-856878f6c6-mclbd" Jul 13 18:34:58.126094 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.125810 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b524ba2c-fcaa-437a-9cd5-64512e35cf7e-federate-client-tls\") pod \"telemeter-client-856878f6c6-mclbd\" (UID: \"b524ba2c-fcaa-437a-9cd5-64512e35cf7e\") " pod="openshift-monitoring/telemeter-client-856878f6c6-mclbd" Jul 13 18:34:58.126094 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.125855 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b524ba2c-fcaa-437a-9cd5-64512e35cf7e-secret-telemeter-client\") pod \"telemeter-client-856878f6c6-mclbd\" (UID: \"b524ba2c-fcaa-437a-9cd5-64512e35cf7e\") " pod="openshift-monitoring/telemeter-client-856878f6c6-mclbd" Jul 13 18:34:58.126094 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.125877 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b524ba2c-fcaa-437a-9cd5-64512e35cf7e-telemeter-client-tls\") pod \"telemeter-client-856878f6c6-mclbd\" (UID: \"b524ba2c-fcaa-437a-9cd5-64512e35cf7e\") " pod="openshift-monitoring/telemeter-client-856878f6c6-mclbd" Jul 13 18:34:58.126094 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.125918 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b524ba2c-fcaa-437a-9cd5-64512e35cf7e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-856878f6c6-mclbd\" (UID: \"b524ba2c-fcaa-437a-9cd5-64512e35cf7e\") " pod="openshift-monitoring/telemeter-client-856878f6c6-mclbd" Jul 13 18:34:58.126094 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.126047 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b524ba2c-fcaa-437a-9cd5-64512e35cf7e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-856878f6c6-mclbd\" (UID: \"b524ba2c-fcaa-437a-9cd5-64512e35cf7e\") " pod="openshift-monitoring/telemeter-client-856878f6c6-mclbd" Jul 13 18:34:58.126308 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.126105 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b524ba2c-fcaa-437a-9cd5-64512e35cf7e-metrics-client-ca\") pod \"telemeter-client-856878f6c6-mclbd\" (UID: \"b524ba2c-fcaa-437a-9cd5-64512e35cf7e\") " pod="openshift-monitoring/telemeter-client-856878f6c6-mclbd" Jul 13 18:34:58.126308 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.126143 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vhwk\" (UniqueName: \"kubernetes.io/projected/b524ba2c-fcaa-437a-9cd5-64512e35cf7e-kube-api-access-6vhwk\") pod \"telemeter-client-856878f6c6-mclbd\" (UID: \"b524ba2c-fcaa-437a-9cd5-64512e35cf7e\") " pod="openshift-monitoring/telemeter-client-856878f6c6-mclbd" Jul 13 18:34:58.126936 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.126913 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"975ed946-1a3d-46ea-82c6-f2f35d35721e","Type":"ContainerStarted","Data":"21592566706a670572a33c8c623b4697f854fa9b0734f8e7f618cacbb80d50f4"} Jul 13 18:34:58.127035 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.126971 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"975ed946-1a3d-46ea-82c6-f2f35d35721e","Type":"ContainerStarted","Data":"736c7b5b47eb68b2db2bc804984b4fcfde41bcbc51cff07cea001a5b3a0e23c4"} Jul 13 18:34:58.127035 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.126998 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"975ed946-1a3d-46ea-82c6-f2f35d35721e","Type":"ContainerStarted","Data":"a8d63b4045bfe730707da3d9fe46d37a0c647ee763e7201c4caaefef0b6ff5b2"} Jul 13 18:34:58.127035 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.127029 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"975ed946-1a3d-46ea-82c6-f2f35d35721e","Type":"ContainerStarted","Data":"e492b6892e024a9e7a7cce9a369648db54f94374f572fc51407c6edf336050ac"} Jul 13 18:34:58.127190 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.127054 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"975ed946-1a3d-46ea-82c6-f2f35d35721e","Type":"ContainerStarted","Data":"aec70d2abeb63539a1fbd99b5eb73f0903523f7bb290aecd27f894162f3a13cb"} Jul 13 18:34:58.127190 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.127077 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"975ed946-1a3d-46ea-82c6-f2f35d35721e","Type":"ContainerStarted","Data":"d0c182fe32e5dcb573663ba873c2d66285704259335277f19f2fbbdd1bbb9c61"} Jul 13 18:34:58.131084 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.131065 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-856878f6c6-mclbd"] Jul 13 18:34:58.191928 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.191873 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.176034588 podStartE2EDuration="2.191852382s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.817794ms" isStatefulPod=false podCreationTimestamp="2026-07-13 18:34:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:34:58.19125283 +0000 UTC m=+154.474681495" watchObservedRunningTime="2026-07-13 18:34:58.191852382 +0000 UTC m=+154.475280985" Jul 13 18:34:58.227361 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.227328 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b524ba2c-fcaa-437a-9cd5-64512e35cf7e-metrics-client-ca\") pod \"telemeter-client-856878f6c6-mclbd\" (UID: \"b524ba2c-fcaa-437a-9cd5-64512e35cf7e\") " pod="openshift-monitoring/telemeter-client-856878f6c6-mclbd" Jul 13 18:34:58.227562 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.227378 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vhwk\" (UniqueName: \"kubernetes.io/projected/b524ba2c-fcaa-437a-9cd5-64512e35cf7e-kube-api-access-6vhwk\") pod \"telemeter-client-856878f6c6-mclbd\" (UID: \"b524ba2c-fcaa-437a-9cd5-64512e35cf7e\") " pod="openshift-monitoring/telemeter-client-856878f6c6-mclbd" Jul 13 18:34:58.227562 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.227424 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b524ba2c-fcaa-437a-9cd5-64512e35cf7e-serving-certs-ca-bundle\") pod \"telemeter-client-856878f6c6-mclbd\" (UID: \"b524ba2c-fcaa-437a-9cd5-64512e35cf7e\") " pod="openshift-monitoring/telemeter-client-856878f6c6-mclbd" Jul 13 18:34:58.227562 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.227468 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b524ba2c-fcaa-437a-9cd5-64512e35cf7e-federate-client-tls\") pod \"telemeter-client-856878f6c6-mclbd\" (UID: \"b524ba2c-fcaa-437a-9cd5-64512e35cf7e\") " pod="openshift-monitoring/telemeter-client-856878f6c6-mclbd" Jul 13 18:34:58.227562 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.227530 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b524ba2c-fcaa-437a-9cd5-64512e35cf7e-secret-telemeter-client\") pod \"telemeter-client-856878f6c6-mclbd\" (UID: \"b524ba2c-fcaa-437a-9cd5-64512e35cf7e\") " pod="openshift-monitoring/telemeter-client-856878f6c6-mclbd" Jul 13 18:34:58.227562 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.227555 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b524ba2c-fcaa-437a-9cd5-64512e35cf7e-telemeter-client-tls\") pod \"telemeter-client-856878f6c6-mclbd\" (UID: \"b524ba2c-fcaa-437a-9cd5-64512e35cf7e\") " pod="openshift-monitoring/telemeter-client-856878f6c6-mclbd" Jul 13 18:34:58.227836 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.227577 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b524ba2c-fcaa-437a-9cd5-64512e35cf7e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-856878f6c6-mclbd\" (UID: \"b524ba2c-fcaa-437a-9cd5-64512e35cf7e\") " pod="openshift-monitoring/telemeter-client-856878f6c6-mclbd" Jul 13 18:34:58.227836 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.227698 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b524ba2c-fcaa-437a-9cd5-64512e35cf7e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-856878f6c6-mclbd\" (UID: \"b524ba2c-fcaa-437a-9cd5-64512e35cf7e\") " pod="openshift-monitoring/telemeter-client-856878f6c6-mclbd" Jul 13 18:34:58.228209 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.228181 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b524ba2c-fcaa-437a-9cd5-64512e35cf7e-metrics-client-ca\") pod \"telemeter-client-856878f6c6-mclbd\" (UID: \"b524ba2c-fcaa-437a-9cd5-64512e35cf7e\") " pod="openshift-monitoring/telemeter-client-856878f6c6-mclbd" Jul 13 18:34:58.228811 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.228783 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b524ba2c-fcaa-437a-9cd5-64512e35cf7e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-856878f6c6-mclbd\" (UID: \"b524ba2c-fcaa-437a-9cd5-64512e35cf7e\") " pod="openshift-monitoring/telemeter-client-856878f6c6-mclbd" Jul 13 18:34:58.228926 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.228796 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b524ba2c-fcaa-437a-9cd5-64512e35cf7e-serving-certs-ca-bundle\") pod \"telemeter-client-856878f6c6-mclbd\" (UID: \"b524ba2c-fcaa-437a-9cd5-64512e35cf7e\") " pod="openshift-monitoring/telemeter-client-856878f6c6-mclbd" Jul 13 18:34:58.231092 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.231063 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b524ba2c-fcaa-437a-9cd5-64512e35cf7e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-856878f6c6-mclbd\" (UID: \"b524ba2c-fcaa-437a-9cd5-64512e35cf7e\") " pod="openshift-monitoring/telemeter-client-856878f6c6-mclbd" Jul 13 18:34:58.231705 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.231513 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b524ba2c-fcaa-437a-9cd5-64512e35cf7e-telemeter-client-tls\") pod \"telemeter-client-856878f6c6-mclbd\" (UID: \"b524ba2c-fcaa-437a-9cd5-64512e35cf7e\") " pod="openshift-monitoring/telemeter-client-856878f6c6-mclbd" Jul 13 18:34:58.231705 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.231663 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b524ba2c-fcaa-437a-9cd5-64512e35cf7e-federate-client-tls\") pod \"telemeter-client-856878f6c6-mclbd\" (UID: \"b524ba2c-fcaa-437a-9cd5-64512e35cf7e\") " pod="openshift-monitoring/telemeter-client-856878f6c6-mclbd" Jul 13 18:34:58.231951 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.231930 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b524ba2c-fcaa-437a-9cd5-64512e35cf7e-secret-telemeter-client\") pod \"telemeter-client-856878f6c6-mclbd\" (UID: \"b524ba2c-fcaa-437a-9cd5-64512e35cf7e\") " pod="openshift-monitoring/telemeter-client-856878f6c6-mclbd" Jul 13 18:34:58.243919 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.243896 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vhwk\" (UniqueName: \"kubernetes.io/projected/b524ba2c-fcaa-437a-9cd5-64512e35cf7e-kube-api-access-6vhwk\") pod \"telemeter-client-856878f6c6-mclbd\" (UID: \"b524ba2c-fcaa-437a-9cd5-64512e35cf7e\") " pod="openshift-monitoring/telemeter-client-856878f6c6-mclbd" Jul 13 18:34:58.367411 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.367318 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 18:34:58.367743 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.367710 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8050c7ee-0e9d-40c6-83f2-77d25bc9a489" containerName="prometheus" containerID="cri-o://bd438e630d228d1bdc2fe77d7725a3a8aa5bae94de56b69e6e43aa6b4c7ad3a1" gracePeriod=600 Jul 13 18:34:58.368115 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.368085 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8050c7ee-0e9d-40c6-83f2-77d25bc9a489" containerName="kube-rbac-proxy-thanos" containerID="cri-o://eccd5d4e6897f09c8be80c131764138dd27e3d9c0ac31a2982ad143e3c7fa759" gracePeriod=600 Jul 13 18:34:58.368225 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.368197 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8050c7ee-0e9d-40c6-83f2-77d25bc9a489" containerName="kube-rbac-proxy" containerID="cri-o://6e4f30459a83f48105159942a7d83059acd29726fcecd515fd5a938b8ef8701c" gracePeriod=600 Jul 13 18:34:58.368312 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.368249 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8050c7ee-0e9d-40c6-83f2-77d25bc9a489" containerName="thanos-sidecar" containerID="cri-o://b5a69a141579ef333f1cbcab93e6e86ab9f3664ea1f1d073ea6315e397da6885" gracePeriod=600 Jul 13 18:34:58.368312 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.368282 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8050c7ee-0e9d-40c6-83f2-77d25bc9a489" containerName="config-reloader" containerID="cri-o://5926bd79c577f78a94262909846a2d3038c1ec65a8eaf4c1173bb2a0f4518a40" gracePeriod=600 Jul 13 18:34:58.368312 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.368281 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8050c7ee-0e9d-40c6-83f2-77d25bc9a489" containerName="kube-rbac-proxy-web" containerID="cri-o://a26f638c65ac847a5a1052dec0b54c8d1a7a562168dd7c6064a92eb61fddfadd" gracePeriod=600 Jul 13 18:34:58.413938 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.413916 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-856878f6c6-mclbd" Jul 13 18:34:58.573187 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:58.572533 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-856878f6c6-mclbd"] Jul 13 18:34:58.575064 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:34:58.575038 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb524ba2c_fcaa_437a_9cd5_64512e35cf7e.slice/crio-f1c0e15afd75d039021108f0248050acc626276a7b2f404913632af1c5baa670 WatchSource:0}: Error finding container f1c0e15afd75d039021108f0248050acc626276a7b2f404913632af1c5baa670: Status 404 returned error can't find the container with id f1c0e15afd75d039021108f0248050acc626276a7b2f404913632af1c5baa670 Jul 13 18:34:59.136006 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.135973 2582 generic.go:356] "Generic (PLEG): container finished" podID="8050c7ee-0e9d-40c6-83f2-77d25bc9a489" containerID="eccd5d4e6897f09c8be80c131764138dd27e3d9c0ac31a2982ad143e3c7fa759" exitCode=0 Jul 13 18:34:59.136006 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.136001 2582 generic.go:356] "Generic (PLEG): container finished" podID="8050c7ee-0e9d-40c6-83f2-77d25bc9a489" containerID="6e4f30459a83f48105159942a7d83059acd29726fcecd515fd5a938b8ef8701c" exitCode=0 Jul 13 18:34:59.136006 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.136007 2582 generic.go:356] "Generic (PLEG): container finished" podID="8050c7ee-0e9d-40c6-83f2-77d25bc9a489" containerID="b5a69a141579ef333f1cbcab93e6e86ab9f3664ea1f1d073ea6315e397da6885" exitCode=0 Jul 13 18:34:59.136006 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.136012 2582 generic.go:356] "Generic (PLEG): container finished" podID="8050c7ee-0e9d-40c6-83f2-77d25bc9a489" containerID="5926bd79c577f78a94262909846a2d3038c1ec65a8eaf4c1173bb2a0f4518a40" exitCode=0 Jul 13 18:34:59.136006 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.136017 2582 generic.go:356] "Generic (PLEG): container finished" podID="8050c7ee-0e9d-40c6-83f2-77d25bc9a489" containerID="bd438e630d228d1bdc2fe77d7725a3a8aa5bae94de56b69e6e43aa6b4c7ad3a1" exitCode=0 Jul 13 18:34:59.136597 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.136045 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8050c7ee-0e9d-40c6-83f2-77d25bc9a489","Type":"ContainerDied","Data":"eccd5d4e6897f09c8be80c131764138dd27e3d9c0ac31a2982ad143e3c7fa759"} Jul 13 18:34:59.136597 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.136132 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8050c7ee-0e9d-40c6-83f2-77d25bc9a489","Type":"ContainerDied","Data":"6e4f30459a83f48105159942a7d83059acd29726fcecd515fd5a938b8ef8701c"} Jul 13 18:34:59.136597 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.136180 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8050c7ee-0e9d-40c6-83f2-77d25bc9a489","Type":"ContainerDied","Data":"b5a69a141579ef333f1cbcab93e6e86ab9f3664ea1f1d073ea6315e397da6885"} Jul 13 18:34:59.136597 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.136218 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8050c7ee-0e9d-40c6-83f2-77d25bc9a489","Type":"ContainerDied","Data":"5926bd79c577f78a94262909846a2d3038c1ec65a8eaf4c1173bb2a0f4518a40"} Jul 13 18:34:59.136597 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.136261 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8050c7ee-0e9d-40c6-83f2-77d25bc9a489","Type":"ContainerDied","Data":"bd438e630d228d1bdc2fe77d7725a3a8aa5bae94de56b69e6e43aa6b4c7ad3a1"} Jul 13 18:34:59.137426 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.137401 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-856878f6c6-mclbd" event={"ID":"b524ba2c-fcaa-437a-9cd5-64512e35cf7e","Type":"ContainerStarted","Data":"f1c0e15afd75d039021108f0248050acc626276a7b2f404913632af1c5baa670"} Jul 13 18:34:59.636887 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.636864 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:34:59.743318 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.743286 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-tls-assets\") pod \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " Jul 13 18:34:59.743505 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.743339 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-prometheus-k8s-rulefiles-0\") pod \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " Jul 13 18:34:59.743505 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.743419 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-web-config\") pod \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " Jul 13 18:34:59.743505 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.743469 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-thanos-prometheus-http-client-file\") pod \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " Jul 13 18:34:59.743716 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.743511 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-prometheus-trusted-ca-bundle\") pod \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " Jul 13 18:34:59.743716 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.743566 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-prometheus-k8s-rulefiles-2\") pod \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " Jul 13 18:34:59.743716 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.743617 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lzz7r\" (UniqueName: \"kubernetes.io/projected/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-kube-api-access-lzz7r\") pod \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " Jul 13 18:34:59.743716 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.743646 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-secret-metrics-client-certs\") pod \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " Jul 13 18:34:59.743716 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.743685 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-config\") pod \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " Jul 13 18:34:59.743951 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.743737 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-config-out\") pod \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " Jul 13 18:34:59.743951 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.743762 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-prometheus-k8s-db\") pod \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " Jul 13 18:34:59.743951 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.743787 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-prometheus-k8s-rulefiles-1\") pod \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " Jul 13 18:34:59.743951 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.743818 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " Jul 13 18:34:59.743951 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.743855 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-secret-grpc-tls\") pod \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " Jul 13 18:34:59.743951 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.743887 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " Jul 13 18:34:59.743951 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.743935 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-secret-kube-rbac-proxy\") pod \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " Jul 13 18:34:59.744273 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.743960 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-secret-prometheus-k8s-tls\") pod \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " Jul 13 18:34:59.744273 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.744016 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "8050c7ee-0e9d-40c6-83f2-77d25bc9a489" (UID: "8050c7ee-0e9d-40c6-83f2-77d25bc9a489"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 18:34:59.744273 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.744064 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-configmap-metrics-client-ca\") pod \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " Jul 13 18:34:59.744273 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.744188 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-configmap-kubelet-serving-ca-bundle\") pod \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " Jul 13 18:34:59.744273 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.744188 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "8050c7ee-0e9d-40c6-83f2-77d25bc9a489" (UID: "8050c7ee-0e9d-40c6-83f2-77d25bc9a489"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 18:34:59.744273 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.744223 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-configmap-serving-certs-ca-bundle\") pod \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\" (UID: \"8050c7ee-0e9d-40c6-83f2-77d25bc9a489\") " Jul 13 18:34:59.744559 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.744515 2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-prometheus-trusted-ca-bundle\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:34:59.744559 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.744535 2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:34:59.746285 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.745336 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "8050c7ee-0e9d-40c6-83f2-77d25bc9a489" (UID: "8050c7ee-0e9d-40c6-83f2-77d25bc9a489"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 18:34:59.746285 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.745298 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "8050c7ee-0e9d-40c6-83f2-77d25bc9a489" (UID: "8050c7ee-0e9d-40c6-83f2-77d25bc9a489"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 18:34:59.746285 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.745427 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "8050c7ee-0e9d-40c6-83f2-77d25bc9a489" (UID: "8050c7ee-0e9d-40c6-83f2-77d25bc9a489"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 18:34:59.746285 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.745859 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "8050c7ee-0e9d-40c6-83f2-77d25bc9a489" (UID: "8050c7ee-0e9d-40c6-83f2-77d25bc9a489"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 18:34:59.746285 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.745908 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "8050c7ee-0e9d-40c6-83f2-77d25bc9a489" (UID: "8050c7ee-0e9d-40c6-83f2-77d25bc9a489"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 18:34:59.746285 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.745924 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "8050c7ee-0e9d-40c6-83f2-77d25bc9a489" (UID: "8050c7ee-0e9d-40c6-83f2-77d25bc9a489"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 18:34:59.748058 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.748035 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "8050c7ee-0e9d-40c6-83f2-77d25bc9a489" (UID: "8050c7ee-0e9d-40c6-83f2-77d25bc9a489"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 18:34:59.748460 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.748416 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "8050c7ee-0e9d-40c6-83f2-77d25bc9a489" (UID: "8050c7ee-0e9d-40c6-83f2-77d25bc9a489"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 18:34:59.748561 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.748472 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "8050c7ee-0e9d-40c6-83f2-77d25bc9a489" (UID: "8050c7ee-0e9d-40c6-83f2-77d25bc9a489"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 18:34:59.748756 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.748722 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "8050c7ee-0e9d-40c6-83f2-77d25bc9a489" (UID: "8050c7ee-0e9d-40c6-83f2-77d25bc9a489"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 18:34:59.748998 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.748955 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "8050c7ee-0e9d-40c6-83f2-77d25bc9a489" (UID: "8050c7ee-0e9d-40c6-83f2-77d25bc9a489"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 18:34:59.749497 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.749468 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "8050c7ee-0e9d-40c6-83f2-77d25bc9a489" (UID: "8050c7ee-0e9d-40c6-83f2-77d25bc9a489"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 18:34:59.750129 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.750079 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "8050c7ee-0e9d-40c6-83f2-77d25bc9a489" (UID: "8050c7ee-0e9d-40c6-83f2-77d25bc9a489"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 18:34:59.750231 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.750128 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-config-out" (OuterVolumeSpecName: "config-out") pod "8050c7ee-0e9d-40c6-83f2-77d25bc9a489" (UID: "8050c7ee-0e9d-40c6-83f2-77d25bc9a489"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 18:34:59.750231 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.750128 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-config" (OuterVolumeSpecName: "config") pod "8050c7ee-0e9d-40c6-83f2-77d25bc9a489" (UID: "8050c7ee-0e9d-40c6-83f2-77d25bc9a489"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 18:34:59.750231 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.750200 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-kube-api-access-lzz7r" (OuterVolumeSpecName: "kube-api-access-lzz7r") pod "8050c7ee-0e9d-40c6-83f2-77d25bc9a489" (UID: "8050c7ee-0e9d-40c6-83f2-77d25bc9a489"). InnerVolumeSpecName "kube-api-access-lzz7r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 18:34:59.750878 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.750850 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "8050c7ee-0e9d-40c6-83f2-77d25bc9a489" (UID: "8050c7ee-0e9d-40c6-83f2-77d25bc9a489"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 18:34:59.761570 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.761535 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-web-config" (OuterVolumeSpecName: "web-config") pod "8050c7ee-0e9d-40c6-83f2-77d25bc9a489" (UID: "8050c7ee-0e9d-40c6-83f2-77d25bc9a489"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 18:34:59.845018 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.844986 2582 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-tls-assets\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:34:59.845018 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.845010 2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:34:59.845018 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.845019 2582 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-web-config\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:34:59.845223 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.845030 2582 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-thanos-prometheus-http-client-file\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:34:59.845223 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.845039 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzz7r\" (UniqueName: \"kubernetes.io/projected/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-kube-api-access-lzz7r\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:34:59.845223 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.845048 2582 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-secret-metrics-client-certs\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:34:59.845223 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.845056 2582 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-config\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:34:59.845223 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.845064 2582 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-config-out\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:34:59.845223 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.845072 2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-prometheus-k8s-db\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:34:59.845223 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.845080 2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:34:59.845223 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.845088 2582 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:34:59.845223 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.845097 2582 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-secret-grpc-tls\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:34:59.845223 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.845105 2582 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:34:59.845223 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.845114 2582 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-secret-kube-rbac-proxy\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:34:59.845223 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.845123 2582 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-secret-prometheus-k8s-tls\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:34:59.845223 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.845131 2582 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-configmap-metrics-client-ca\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:34:59.845223 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.845139 2582 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:34:59.845223 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:34:59.845147 2582 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8050c7ee-0e9d-40c6-83f2-77d25bc9a489-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:35:00.144723 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.144686 2582 generic.go:356] "Generic (PLEG): container finished" podID="8050c7ee-0e9d-40c6-83f2-77d25bc9a489" containerID="a26f638c65ac847a5a1052dec0b54c8d1a7a562168dd7c6064a92eb61fddfadd" exitCode=0 Jul 13 18:35:00.145085 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.144734 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8050c7ee-0e9d-40c6-83f2-77d25bc9a489","Type":"ContainerDied","Data":"a26f638c65ac847a5a1052dec0b54c8d1a7a562168dd7c6064a92eb61fddfadd"} Jul 13 18:35:00.145085 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.144777 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.145085 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.144812 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8050c7ee-0e9d-40c6-83f2-77d25bc9a489","Type":"ContainerDied","Data":"ffdad66875774d9bd12ca101a7213f329b8edd663dba12d3cdee4e0fa39173a5"} Jul 13 18:35:00.145085 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.144859 2582 scope.go:117] "RemoveContainer" containerID="eccd5d4e6897f09c8be80c131764138dd27e3d9c0ac31a2982ad143e3c7fa759" Jul 13 18:35:00.153449 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.153431 2582 scope.go:117] "RemoveContainer" containerID="6e4f30459a83f48105159942a7d83059acd29726fcecd515fd5a938b8ef8701c" Jul 13 18:35:00.160770 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.160753 2582 scope.go:117] "RemoveContainer" containerID="a26f638c65ac847a5a1052dec0b54c8d1a7a562168dd7c6064a92eb61fddfadd" Jul 13 18:35:00.167434 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.167417 2582 scope.go:117] "RemoveContainer" containerID="b5a69a141579ef333f1cbcab93e6e86ab9f3664ea1f1d073ea6315e397da6885" Jul 13 18:35:00.174668 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.174649 2582 scope.go:117] "RemoveContainer" containerID="5926bd79c577f78a94262909846a2d3038c1ec65a8eaf4c1173bb2a0f4518a40" Jul 13 18:35:00.174731 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.174718 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 18:35:00.181491 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.181444 2582 scope.go:117] "RemoveContainer" containerID="bd438e630d228d1bdc2fe77d7725a3a8aa5bae94de56b69e6e43aa6b4c7ad3a1" Jul 13 18:35:00.184095 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.184076 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 18:35:00.190086 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.190058 2582 scope.go:117] "RemoveContainer" containerID="d4369c51d014bca41d39a2e9fa76824d9f183217ed3680793f6bcd2ae2da4f05" Jul 13 18:35:00.197773 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.197755 2582 scope.go:117] "RemoveContainer" containerID="eccd5d4e6897f09c8be80c131764138dd27e3d9c0ac31a2982ad143e3c7fa759" Jul 13 18:35:00.198110 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:35:00.198088 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eccd5d4e6897f09c8be80c131764138dd27e3d9c0ac31a2982ad143e3c7fa759\": container with ID starting with eccd5d4e6897f09c8be80c131764138dd27e3d9c0ac31a2982ad143e3c7fa759 not found: ID does not exist" containerID="eccd5d4e6897f09c8be80c131764138dd27e3d9c0ac31a2982ad143e3c7fa759" Jul 13 18:35:00.198201 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.198117 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eccd5d4e6897f09c8be80c131764138dd27e3d9c0ac31a2982ad143e3c7fa759"} err="failed to get container status \"eccd5d4e6897f09c8be80c131764138dd27e3d9c0ac31a2982ad143e3c7fa759\": rpc error: code = NotFound desc = could not find container \"eccd5d4e6897f09c8be80c131764138dd27e3d9c0ac31a2982ad143e3c7fa759\": container with ID starting with eccd5d4e6897f09c8be80c131764138dd27e3d9c0ac31a2982ad143e3c7fa759 not found: ID does not exist" Jul 13 18:35:00.198201 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.198156 2582 scope.go:117] "RemoveContainer" containerID="6e4f30459a83f48105159942a7d83059acd29726fcecd515fd5a938b8ef8701c" Jul 13 18:35:00.198438 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:35:00.198423 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e4f30459a83f48105159942a7d83059acd29726fcecd515fd5a938b8ef8701c\": container with ID starting with 6e4f30459a83f48105159942a7d83059acd29726fcecd515fd5a938b8ef8701c not found: ID does not exist" containerID="6e4f30459a83f48105159942a7d83059acd29726fcecd515fd5a938b8ef8701c" Jul 13 18:35:00.198479 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.198443 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e4f30459a83f48105159942a7d83059acd29726fcecd515fd5a938b8ef8701c"} err="failed to get container status \"6e4f30459a83f48105159942a7d83059acd29726fcecd515fd5a938b8ef8701c\": rpc error: code = NotFound desc = could not find container \"6e4f30459a83f48105159942a7d83059acd29726fcecd515fd5a938b8ef8701c\": container with ID starting with 6e4f30459a83f48105159942a7d83059acd29726fcecd515fd5a938b8ef8701c not found: ID does not exist" Jul 13 18:35:00.198479 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.198458 2582 scope.go:117] "RemoveContainer" containerID="a26f638c65ac847a5a1052dec0b54c8d1a7a562168dd7c6064a92eb61fddfadd" Jul 13 18:35:00.198978 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:35:00.198951 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a26f638c65ac847a5a1052dec0b54c8d1a7a562168dd7c6064a92eb61fddfadd\": container with ID starting with a26f638c65ac847a5a1052dec0b54c8d1a7a562168dd7c6064a92eb61fddfadd not found: ID does not exist" containerID="a26f638c65ac847a5a1052dec0b54c8d1a7a562168dd7c6064a92eb61fddfadd" Jul 13 18:35:00.198978 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.198971 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a26f638c65ac847a5a1052dec0b54c8d1a7a562168dd7c6064a92eb61fddfadd"} err="failed to get container status \"a26f638c65ac847a5a1052dec0b54c8d1a7a562168dd7c6064a92eb61fddfadd\": rpc error: code = NotFound desc = could not find container \"a26f638c65ac847a5a1052dec0b54c8d1a7a562168dd7c6064a92eb61fddfadd\": container with ID starting with a26f638c65ac847a5a1052dec0b54c8d1a7a562168dd7c6064a92eb61fddfadd not found: ID does not exist" Jul 13 18:35:00.199140 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.198987 2582 scope.go:117] "RemoveContainer" containerID="b5a69a141579ef333f1cbcab93e6e86ab9f3664ea1f1d073ea6315e397da6885" Jul 13 18:35:00.199204 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:35:00.199189 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5a69a141579ef333f1cbcab93e6e86ab9f3664ea1f1d073ea6315e397da6885\": container with ID starting with b5a69a141579ef333f1cbcab93e6e86ab9f3664ea1f1d073ea6315e397da6885 not found: ID does not exist" containerID="b5a69a141579ef333f1cbcab93e6e86ab9f3664ea1f1d073ea6315e397da6885" Jul 13 18:35:00.199252 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.199207 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5a69a141579ef333f1cbcab93e6e86ab9f3664ea1f1d073ea6315e397da6885"} err="failed to get container status \"b5a69a141579ef333f1cbcab93e6e86ab9f3664ea1f1d073ea6315e397da6885\": rpc error: code = NotFound desc = could not find container \"b5a69a141579ef333f1cbcab93e6e86ab9f3664ea1f1d073ea6315e397da6885\": container with ID starting with b5a69a141579ef333f1cbcab93e6e86ab9f3664ea1f1d073ea6315e397da6885 not found: ID does not exist" Jul 13 18:35:00.199252 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.199219 2582 scope.go:117] "RemoveContainer" containerID="5926bd79c577f78a94262909846a2d3038c1ec65a8eaf4c1173bb2a0f4518a40" Jul 13 18:35:00.199478 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:35:00.199455 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5926bd79c577f78a94262909846a2d3038c1ec65a8eaf4c1173bb2a0f4518a40\": container with ID starting with 5926bd79c577f78a94262909846a2d3038c1ec65a8eaf4c1173bb2a0f4518a40 not found: ID does not exist" containerID="5926bd79c577f78a94262909846a2d3038c1ec65a8eaf4c1173bb2a0f4518a40" Jul 13 18:35:00.199571 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.199478 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5926bd79c577f78a94262909846a2d3038c1ec65a8eaf4c1173bb2a0f4518a40"} err="failed to get container status \"5926bd79c577f78a94262909846a2d3038c1ec65a8eaf4c1173bb2a0f4518a40\": rpc error: code = NotFound desc = could not find container \"5926bd79c577f78a94262909846a2d3038c1ec65a8eaf4c1173bb2a0f4518a40\": container with ID starting with 5926bd79c577f78a94262909846a2d3038c1ec65a8eaf4c1173bb2a0f4518a40 not found: ID does not exist" Jul 13 18:35:00.199571 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.199491 2582 scope.go:117] "RemoveContainer" containerID="bd438e630d228d1bdc2fe77d7725a3a8aa5bae94de56b69e6e43aa6b4c7ad3a1" Jul 13 18:35:00.199838 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:35:00.199805 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd438e630d228d1bdc2fe77d7725a3a8aa5bae94de56b69e6e43aa6b4c7ad3a1\": container with ID starting with bd438e630d228d1bdc2fe77d7725a3a8aa5bae94de56b69e6e43aa6b4c7ad3a1 not found: ID does not exist" containerID="bd438e630d228d1bdc2fe77d7725a3a8aa5bae94de56b69e6e43aa6b4c7ad3a1" Jul 13 18:35:00.199918 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.199846 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd438e630d228d1bdc2fe77d7725a3a8aa5bae94de56b69e6e43aa6b4c7ad3a1"} err="failed to get container status \"bd438e630d228d1bdc2fe77d7725a3a8aa5bae94de56b69e6e43aa6b4c7ad3a1\": rpc error: code = NotFound desc = could not find container \"bd438e630d228d1bdc2fe77d7725a3a8aa5bae94de56b69e6e43aa6b4c7ad3a1\": container with ID starting with bd438e630d228d1bdc2fe77d7725a3a8aa5bae94de56b69e6e43aa6b4c7ad3a1 not found: ID does not exist" Jul 13 18:35:00.199918 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.199863 2582 scope.go:117] "RemoveContainer" containerID="d4369c51d014bca41d39a2e9fa76824d9f183217ed3680793f6bcd2ae2da4f05" Jul 13 18:35:00.200095 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:35:00.200076 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4369c51d014bca41d39a2e9fa76824d9f183217ed3680793f6bcd2ae2da4f05\": container with ID starting with d4369c51d014bca41d39a2e9fa76824d9f183217ed3680793f6bcd2ae2da4f05 not found: ID does not exist" containerID="d4369c51d014bca41d39a2e9fa76824d9f183217ed3680793f6bcd2ae2da4f05" Jul 13 18:35:00.200159 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.200100 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4369c51d014bca41d39a2e9fa76824d9f183217ed3680793f6bcd2ae2da4f05"} err="failed to get container status \"d4369c51d014bca41d39a2e9fa76824d9f183217ed3680793f6bcd2ae2da4f05\": rpc error: code = NotFound desc = could not find container \"d4369c51d014bca41d39a2e9fa76824d9f183217ed3680793f6bcd2ae2da4f05\": container with ID starting with d4369c51d014bca41d39a2e9fa76824d9f183217ed3680793f6bcd2ae2da4f05 not found: ID does not exist" Jul 13 18:35:00.230169 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.230138 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 18:35:00.230456 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.230442 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8050c7ee-0e9d-40c6-83f2-77d25bc9a489" containerName="config-reloader" Jul 13 18:35:00.230498 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.230462 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8050c7ee-0e9d-40c6-83f2-77d25bc9a489" containerName="config-reloader" Jul 13 18:35:00.230498 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.230471 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8050c7ee-0e9d-40c6-83f2-77d25bc9a489" containerName="kube-rbac-proxy-thanos" Jul 13 18:35:00.230498 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.230476 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8050c7ee-0e9d-40c6-83f2-77d25bc9a489" containerName="kube-rbac-proxy-thanos" Jul 13 18:35:00.230498 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.230490 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8050c7ee-0e9d-40c6-83f2-77d25bc9a489" containerName="kube-rbac-proxy" Jul 13 18:35:00.230498 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.230495 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8050c7ee-0e9d-40c6-83f2-77d25bc9a489" containerName="kube-rbac-proxy" Jul 13 18:35:00.230698 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.230506 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8050c7ee-0e9d-40c6-83f2-77d25bc9a489" containerName="thanos-sidecar" Jul 13 18:35:00.230698 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.230511 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8050c7ee-0e9d-40c6-83f2-77d25bc9a489" containerName="thanos-sidecar" Jul 13 18:35:00.230698 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.230516 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8050c7ee-0e9d-40c6-83f2-77d25bc9a489" containerName="prometheus" Jul 13 18:35:00.230698 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.230521 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8050c7ee-0e9d-40c6-83f2-77d25bc9a489" containerName="prometheus" Jul 13 18:35:00.230698 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.230529 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8050c7ee-0e9d-40c6-83f2-77d25bc9a489" containerName="init-config-reloader" Jul 13 18:35:00.230698 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.230534 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8050c7ee-0e9d-40c6-83f2-77d25bc9a489" containerName="init-config-reloader" Jul 13 18:35:00.230698 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.230539 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8050c7ee-0e9d-40c6-83f2-77d25bc9a489" containerName="kube-rbac-proxy-web" Jul 13 18:35:00.230698 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.230545 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8050c7ee-0e9d-40c6-83f2-77d25bc9a489" containerName="kube-rbac-proxy-web" Jul 13 18:35:00.230698 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.230594 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8050c7ee-0e9d-40c6-83f2-77d25bc9a489" containerName="kube-rbac-proxy-web" Jul 13 18:35:00.230698 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.230625 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8050c7ee-0e9d-40c6-83f2-77d25bc9a489" containerName="thanos-sidecar" Jul 13 18:35:00.230698 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.230633 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8050c7ee-0e9d-40c6-83f2-77d25bc9a489" containerName="kube-rbac-proxy" Jul 13 18:35:00.230698 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.230645 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8050c7ee-0e9d-40c6-83f2-77d25bc9a489" containerName="prometheus" Jul 13 18:35:00.230698 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.230653 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8050c7ee-0e9d-40c6-83f2-77d25bc9a489" containerName="kube-rbac-proxy-thanos" Jul 13 18:35:00.230698 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.230659 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8050c7ee-0e9d-40c6-83f2-77d25bc9a489" containerName="config-reloader" Jul 13 18:35:00.234774 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.234754 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.238951 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.238933 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 13 18:35:00.238951 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.238942 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 13 18:35:00.239273 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.239256 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-k8pkb\"" Jul 13 18:35:00.239367 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.239354 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 13 18:35:00.239656 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.239640 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 13 18:35:00.239698 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.239641 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 13 18:35:00.239923 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.239910 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 13 18:35:00.240097 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.240081 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 13 18:35:00.240143 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.240119 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 13 18:35:00.240190 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.240150 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 13 18:35:00.240270 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.240257 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 13 18:35:00.240391 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.240343 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 13 18:35:00.240391 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.240347 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-dfpqs9oek6f8p\"" Jul 13 18:35:00.240587 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.240512 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 13 18:35:00.249262 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.249233 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 13 18:35:00.251759 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.251736 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 13 18:35:00.262959 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.262936 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 18:35:00.280177 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.280116 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8050c7ee-0e9d-40c6-83f2-77d25bc9a489" path="/var/lib/kubelet/pods/8050c7ee-0e9d-40c6-83f2-77d25bc9a489/volumes" Jul 13 18:35:00.348578 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.348542 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8f9f41cf-286b-4bc6-9523-b364128375aa-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.348774 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.348588 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8f9f41cf-286b-4bc6-9523-b364128375aa-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.348774 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.348717 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8f9f41cf-286b-4bc6-9523-b364128375aa-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.348774 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.348763 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f9f41cf-286b-4bc6-9523-b364128375aa-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.348950 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.348795 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8f9f41cf-286b-4bc6-9523-b364128375aa-config-out\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.348950 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.348889 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8f9f41cf-286b-4bc6-9523-b364128375aa-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.349083 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.348947 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f9f41cf-286b-4bc6-9523-b364128375aa-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.349083 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.349030 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26tf8\" (UniqueName: \"kubernetes.io/projected/8f9f41cf-286b-4bc6-9523-b364128375aa-kube-api-access-26tf8\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.349083 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.349062 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8f9f41cf-286b-4bc6-9523-b364128375aa-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.349222 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.349091 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8f9f41cf-286b-4bc6-9523-b364128375aa-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.349222 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.349155 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8f9f41cf-286b-4bc6-9523-b364128375aa-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.349222 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.349187 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f9f41cf-286b-4bc6-9523-b364128375aa-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.349222 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.349217 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8f9f41cf-286b-4bc6-9523-b364128375aa-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.349392 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.349269 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8f9f41cf-286b-4bc6-9523-b364128375aa-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.349392 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.349296 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8f9f41cf-286b-4bc6-9523-b364128375aa-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.349392 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.349321 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8f9f41cf-286b-4bc6-9523-b364128375aa-web-config\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.349392 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.349381 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8f9f41cf-286b-4bc6-9523-b364128375aa-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.349523 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.349452 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8f9f41cf-286b-4bc6-9523-b364128375aa-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.349523 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.349477 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8f9f41cf-286b-4bc6-9523-b364128375aa-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.349523 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.349498 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8f9f41cf-286b-4bc6-9523-b364128375aa-config\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.450831 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.450792 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8f9f41cf-286b-4bc6-9523-b364128375aa-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.450831 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.450833 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f9f41cf-286b-4bc6-9523-b364128375aa-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.451066 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.450863 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8f9f41cf-286b-4bc6-9523-b364128375aa-config-out\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.451066 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.450902 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8f9f41cf-286b-4bc6-9523-b364128375aa-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.451066 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.450934 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f9f41cf-286b-4bc6-9523-b364128375aa-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.451066 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.450966 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26tf8\" (UniqueName: \"kubernetes.io/projected/8f9f41cf-286b-4bc6-9523-b364128375aa-kube-api-access-26tf8\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.451066 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.451001 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8f9f41cf-286b-4bc6-9523-b364128375aa-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.451066 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.451030 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8f9f41cf-286b-4bc6-9523-b364128375aa-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.451066 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.451056 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8f9f41cf-286b-4bc6-9523-b364128375aa-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.451379 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.451078 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f9f41cf-286b-4bc6-9523-b364128375aa-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.451379 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.451100 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8f9f41cf-286b-4bc6-9523-b364128375aa-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.451379 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.451126 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8f9f41cf-286b-4bc6-9523-b364128375aa-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.451379 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.451150 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8f9f41cf-286b-4bc6-9523-b364128375aa-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.451379 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.451176 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8f9f41cf-286b-4bc6-9523-b364128375aa-web-config\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.451379 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.451205 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8f9f41cf-286b-4bc6-9523-b364128375aa-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.451379 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.451245 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8f9f41cf-286b-4bc6-9523-b364128375aa-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.451379 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.451279 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8f9f41cf-286b-4bc6-9523-b364128375aa-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.451379 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.451315 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8f9f41cf-286b-4bc6-9523-b364128375aa-config\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.451379 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.451379 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8f9f41cf-286b-4bc6-9523-b364128375aa-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.452045 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.451413 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8f9f41cf-286b-4bc6-9523-b364128375aa-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.452045 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.451792 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f9f41cf-286b-4bc6-9523-b364128375aa-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.452045 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.451953 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f9f41cf-286b-4bc6-9523-b364128375aa-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.452388 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.452261 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8f9f41cf-286b-4bc6-9523-b364128375aa-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.454129 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.454104 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8f9f41cf-286b-4bc6-9523-b364128375aa-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.456017 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.454711 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8f9f41cf-286b-4bc6-9523-b364128375aa-config-out\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.456017 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.455508 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8f9f41cf-286b-4bc6-9523-b364128375aa-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.456017 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.455740 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8f9f41cf-286b-4bc6-9523-b364128375aa-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.456017 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.455980 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8f9f41cf-286b-4bc6-9523-b364128375aa-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.456017 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.455997 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f9f41cf-286b-4bc6-9523-b364128375aa-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.456347 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.456227 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8f9f41cf-286b-4bc6-9523-b364128375aa-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.456578 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.456550 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/8f9f41cf-286b-4bc6-9523-b364128375aa-config\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.456686 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.456654 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8f9f41cf-286b-4bc6-9523-b364128375aa-web-config\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.457281 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.457256 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8f9f41cf-286b-4bc6-9523-b364128375aa-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.457387 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.457333 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8f9f41cf-286b-4bc6-9523-b364128375aa-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.457763 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.457739 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8f9f41cf-286b-4bc6-9523-b364128375aa-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.458396 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.458379 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8f9f41cf-286b-4bc6-9523-b364128375aa-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.458508 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.458489 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8f9f41cf-286b-4bc6-9523-b364128375aa-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.459149 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.459106 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8f9f41cf-286b-4bc6-9523-b364128375aa-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.459379 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.459361 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8f9f41cf-286b-4bc6-9523-b364128375aa-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.461510 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.461491 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26tf8\" (UniqueName: \"kubernetes.io/projected/8f9f41cf-286b-4bc6-9523-b364128375aa-kube-api-access-26tf8\") pod \"prometheus-k8s-0\" (UID: \"8f9f41cf-286b-4bc6-9523-b364128375aa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.544948 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.544863 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:00.883787 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:00.883762 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 18:35:00.885777 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:35:00.885753 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f9f41cf_286b_4bc6_9523_b364128375aa.slice/crio-564a919dbcf6380a15e4d9ddd4329d30dfc18e90835edcbe743d75833aaa5193 WatchSource:0}: Error finding container 564a919dbcf6380a15e4d9ddd4329d30dfc18e90835edcbe743d75833aaa5193: Status 404 returned error can't find the container with id 564a919dbcf6380a15e4d9ddd4329d30dfc18e90835edcbe743d75833aaa5193 Jul 13 18:35:01.151137 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:01.151048 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-856878f6c6-mclbd" event={"ID":"b524ba2c-fcaa-437a-9cd5-64512e35cf7e","Type":"ContainerStarted","Data":"68e80e5f1643871fdd7f83ba59c61aabcb335f25320838826dff28c7015a90ca"} Jul 13 18:35:01.151137 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:01.151105 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-856878f6c6-mclbd" event={"ID":"b524ba2c-fcaa-437a-9cd5-64512e35cf7e","Type":"ContainerStarted","Data":"1fe9e960f30bbcce6fdb95b2e21011669c56d5ec34850f2572bb3c15fc8d8b00"} Jul 13 18:35:01.151137 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:01.151123 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-856878f6c6-mclbd" event={"ID":"b524ba2c-fcaa-437a-9cd5-64512e35cf7e","Type":"ContainerStarted","Data":"da0d282a1e0253fdef4691f67c450f09e2f181e15dfe5093ce88768c457b5475"} Jul 13 18:35:01.152693 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:01.152668 2582 generic.go:356] "Generic (PLEG): container finished" podID="8f9f41cf-286b-4bc6-9523-b364128375aa" containerID="bb8f7c016fae31f88d5a6b2c7073649a35fa60c026ab252fd5a2793bafdc9150" exitCode=0 Jul 13 18:35:01.152810 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:01.152759 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8f9f41cf-286b-4bc6-9523-b364128375aa","Type":"ContainerDied","Data":"bb8f7c016fae31f88d5a6b2c7073649a35fa60c026ab252fd5a2793bafdc9150"} Jul 13 18:35:01.152854 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:01.152830 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8f9f41cf-286b-4bc6-9523-b364128375aa","Type":"ContainerStarted","Data":"564a919dbcf6380a15e4d9ddd4329d30dfc18e90835edcbe743d75833aaa5193"} Jul 13 18:35:01.179112 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:01.179057 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-856878f6c6-mclbd" podStartSLOduration=0.986145459 podStartE2EDuration="3.179036535s" totalImagesPullingTime="2.192891076s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:34:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:35:01.176523844 +0000 UTC m=+157.459952482" watchObservedRunningTime="2026-07-13 18:35:01.179036535 +0000 UTC m=+157.462465140" Jul 13 18:35:02.026355 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:02.026320 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:35:02.123302 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:02.123267 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:35:02.161984 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:02.161950 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8f9f41cf-286b-4bc6-9523-b364128375aa","Type":"ContainerStarted","Data":"ddb94024c001927d9ed473b1db0df6e9b55b93727fab4547ba82b168c0a77af2"} Jul 13 18:35:02.162338 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:02.162044 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8f9f41cf-286b-4bc6-9523-b364128375aa","Type":"ContainerStarted","Data":"1eed5d45d5e4a7d8249d02e8ef8c32547188e9afe0ff85600e127149881172c0"} Jul 13 18:35:02.162338 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:02.162080 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8f9f41cf-286b-4bc6-9523-b364128375aa","Type":"ContainerStarted","Data":"0439a302ecec61a947173c3251c71838d69fb12903f89ff15ef9dc8208de99a0"} Jul 13 18:35:02.162338 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:02.162106 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8f9f41cf-286b-4bc6-9523-b364128375aa","Type":"ContainerStarted","Data":"7dab709e0945366f2990b8dae8cbc0ee0e0ba1fc048b022d4ce45916232c391b"} Jul 13 18:35:02.162338 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:02.162131 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8f9f41cf-286b-4bc6-9523-b364128375aa","Type":"ContainerStarted","Data":"a7cc962db54600159364d286d30bfab07f0d64380af23b7c3e9dd99445b0fa09"} Jul 13 18:35:02.162338 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:02.162156 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8f9f41cf-286b-4bc6-9523-b364128375aa","Type":"ContainerStarted","Data":"9c10343cc3dbf13428369aa0d27bca2159af58db703076a25e45d92e76fbe260"} Jul 13 18:35:02.202085 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:02.202015 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.181779445 podStartE2EDuration="2.201992023s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.212578ms" isStatefulPod=false podCreationTimestamp="2026-07-13 18:35:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:35:02.201304089 +0000 UTC m=+158.484732740" watchObservedRunningTime="2026-07-13 18:35:02.201992023 +0000 UTC m=+158.485420625" Jul 13 18:35:02.701279 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:02.701240 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:35:02.829131 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:02.829101 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:35:05.545223 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:05.545180 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:35:13.380000 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:13.379966 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:35:13.480742 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:13.480705 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:35:13.559153 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:13.559116 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:35:54.409469 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:35:54.409429 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:36:00.545968 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:36:00.545920 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:36:00.562114 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:36:00.562086 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:36:01.429488 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:36:01.429463 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:36:08.222096 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:36:08.222057 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:36:08.283030 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:36:08.282996 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:36:09.150215 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:36:09.150173 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:36:09.218905 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:36:09.218869 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:36:09.482570 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:36:09.482542 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:36:09.584345 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:36:09.584292 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:36:10.082507 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:36:10.082451 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:36:10.140792 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:36:10.140765 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:36:20.202779 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:36:20.202745 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:36:20.272902 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:36:20.272871 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:36:20.576724 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:36:20.576633 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:36:20.681016 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:36:20.680983 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:36:20.798968 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:36:20.798935 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:36:27.896201 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:36:27.896147 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:36:28.158147 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:36:28.158049 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:36:30.834885 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:36:30.834851 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:36:33.780998 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:36:33.780959 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:36:34.661297 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:36:34.661259 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:36:36.260256 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:36:36.260222 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:36:49.968420 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:36:49.968328 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:37:12.547856 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:12.547818 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:37:12.649962 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:12.649916 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:37:24.096497 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:24.096467 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8l7w6_b14ef695-c589-4fca-bd12-2e951f04e466/ovn-acl-logging/0.log" Jul 13 18:37:24.098678 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:24.098659 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8l7w6_b14ef695-c589-4fca-bd12-2e951f04e466/ovn-acl-logging/0.log" Jul 13 18:37:24.099873 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:24.099841 2582 kubelet.go:1648] "Image garbage collection succeeded" Jul 13 18:37:39.563919 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:39.563882 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:37:40.920232 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:40.920196 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:37:43.108083 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:43.108027 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:37:43.207611 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:43.207579 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:37:44.637714 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:44.637674 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:37:44.806299 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:44.806262 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:37:44.841562 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:44.841525 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-6qjfs"] Jul 13 18:37:44.843939 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:44.843919 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-6qjfs" Jul 13 18:37:44.846794 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:44.846778 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 13 18:37:44.847361 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:44.847346 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 13 18:37:44.861237 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:44.861209 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-6qjfs"] Jul 13 18:37:44.920168 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:44.920086 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0c30f0fe-fe4c-4f03-a531-15bafc720ec8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-6qjfs\" (UID: \"0c30f0fe-fe4c-4f03-a531-15bafc720ec8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6qjfs" Jul 13 18:37:44.920168 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:44.920119 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmfk4\" (UniqueName: \"kubernetes.io/projected/0c30f0fe-fe4c-4f03-a531-15bafc720ec8-kube-api-access-qmfk4\") pod \"cert-manager-webhook-6486ffd997-6qjfs\" (UID: \"0c30f0fe-fe4c-4f03-a531-15bafc720ec8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6qjfs" Jul 13 18:37:45.020539 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:45.020503 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0c30f0fe-fe4c-4f03-a531-15bafc720ec8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-6qjfs\" (UID: \"0c30f0fe-fe4c-4f03-a531-15bafc720ec8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6qjfs" Jul 13 18:37:45.020539 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:45.020539 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmfk4\" (UniqueName: \"kubernetes.io/projected/0c30f0fe-fe4c-4f03-a531-15bafc720ec8-kube-api-access-qmfk4\") pod \"cert-manager-webhook-6486ffd997-6qjfs\" (UID: \"0c30f0fe-fe4c-4f03-a531-15bafc720ec8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6qjfs" Jul 13 18:37:45.030483 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:45.030449 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0c30f0fe-fe4c-4f03-a531-15bafc720ec8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-6qjfs\" (UID: \"0c30f0fe-fe4c-4f03-a531-15bafc720ec8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6qjfs" Jul 13 18:37:45.030599 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:45.030558 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmfk4\" (UniqueName: \"kubernetes.io/projected/0c30f0fe-fe4c-4f03-a531-15bafc720ec8-kube-api-access-qmfk4\") pod \"cert-manager-webhook-6486ffd997-6qjfs\" (UID: \"0c30f0fe-fe4c-4f03-a531-15bafc720ec8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6qjfs" Jul 13 18:37:45.162365 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:45.162332 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-6qjfs" Jul 13 18:37:45.186792 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:45.186756 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:37:45.308577 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:45.308553 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-6qjfs"] Jul 13 18:37:45.310975 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:37:45.310950 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0c30f0fe_fe4c_4f03_a531_15bafc720ec8.slice/crio-d9a11385ef444ee7947500fe90d2943a63433eb9b88f4d9df8a6d447c927d9e1 WatchSource:0}: Error finding container d9a11385ef444ee7947500fe90d2943a63433eb9b88f4d9df8a6d447c927d9e1: Status 404 returned error can't find the container with id d9a11385ef444ee7947500fe90d2943a63433eb9b88f4d9df8a6d447c927d9e1 Jul 13 18:37:45.312697 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:45.312675 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 18:37:45.401470 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:45.401431 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:37:45.829931 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:45.829877 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-6qjfs" event={"ID":"0c30f0fe-fe4c-4f03-a531-15bafc720ec8","Type":"ContainerStarted","Data":"d9a11385ef444ee7947500fe90d2943a63433eb9b88f4d9df8a6d447c927d9e1"} Jul 13 18:37:47.009394 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:47.009326 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:37:47.229127 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:47.229096 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:37:48.846764 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:48.846730 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-6qjfs" event={"ID":"0c30f0fe-fe4c-4f03-a531-15bafc720ec8","Type":"ContainerStarted","Data":"63cf45b220cae762bf8f25c26ba2145af6179a6b3a2253e4a4ff01471b4c5a1c"} Jul 13 18:37:48.847198 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:48.846859 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-6qjfs" Jul 13 18:37:48.866234 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:48.866181 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-6qjfs" podStartSLOduration=1.511613391 podStartE2EDuration="4.866166407s" totalImagesPullingTime="3.354553016s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:37:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:37:48.864580259 +0000 UTC m=+325.148008862" watchObservedRunningTime="2026-07-13 18:37:48.866166407 +0000 UTC m=+325.149595008" Jul 13 18:37:49.916959 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:49.916926 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:37:49.984522 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:49.984488 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:37:54.809375 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:54.809336 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:37:54.852516 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:54.852488 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-6qjfs" Jul 13 18:37:57.479630 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:57.479577 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:37:57.579435 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:37:57.579391 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:38:00.337306 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:00.337275 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:38:00.436762 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:00.436716 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:38:02.592123 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:02.592087 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9nwgn"] Jul 13 18:38:02.594558 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:02.594542 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9nwgn" Jul 13 18:38:02.597522 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:02.597504 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 13 18:38:02.598923 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:02.598906 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 13 18:38:02.599029 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:02.598917 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-gnvg8\"" Jul 13 18:38:02.604293 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:02.604270 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9nwgn"] Jul 13 18:38:02.675929 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:02.675890 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/908fd811-9587-407d-92c6-c1d37f9f5163-tmp\") pod \"openshift-lws-operator-fd8ccff4c-9nwgn\" (UID: \"908fd811-9587-407d-92c6-c1d37f9f5163\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9nwgn" Jul 13 18:38:02.676101 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:02.675949 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nt9w7\" (UniqueName: \"kubernetes.io/projected/908fd811-9587-407d-92c6-c1d37f9f5163-kube-api-access-nt9w7\") pod \"openshift-lws-operator-fd8ccff4c-9nwgn\" (UID: \"908fd811-9587-407d-92c6-c1d37f9f5163\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9nwgn" Jul 13 18:38:02.776768 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:02.776732 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nt9w7\" (UniqueName: \"kubernetes.io/projected/908fd811-9587-407d-92c6-c1d37f9f5163-kube-api-access-nt9w7\") pod \"openshift-lws-operator-fd8ccff4c-9nwgn\" (UID: \"908fd811-9587-407d-92c6-c1d37f9f5163\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9nwgn" Jul 13 18:38:02.776931 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:02.776814 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/908fd811-9587-407d-92c6-c1d37f9f5163-tmp\") pod \"openshift-lws-operator-fd8ccff4c-9nwgn\" (UID: \"908fd811-9587-407d-92c6-c1d37f9f5163\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9nwgn" Jul 13 18:38:02.777122 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:02.777107 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/908fd811-9587-407d-92c6-c1d37f9f5163-tmp\") pod \"openshift-lws-operator-fd8ccff4c-9nwgn\" (UID: \"908fd811-9587-407d-92c6-c1d37f9f5163\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9nwgn" Jul 13 18:38:02.786489 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:02.786462 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nt9w7\" (UniqueName: \"kubernetes.io/projected/908fd811-9587-407d-92c6-c1d37f9f5163-kube-api-access-nt9w7\") pod \"openshift-lws-operator-fd8ccff4c-9nwgn\" (UID: \"908fd811-9587-407d-92c6-c1d37f9f5163\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9nwgn" Jul 13 18:38:02.903332 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:02.903245 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9nwgn" Jul 13 18:38:03.040504 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:03.040480 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9nwgn"] Jul 13 18:38:03.043091 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:38:03.043061 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod908fd811_9587_407d_92c6_c1d37f9f5163.slice/crio-030a3f2375e513e1425b1c7a0ea1c04f6c70fe39a2c228081a032dbba954bde9 WatchSource:0}: Error finding container 030a3f2375e513e1425b1c7a0ea1c04f6c70fe39a2c228081a032dbba954bde9: Status 404 returned error can't find the container with id 030a3f2375e513e1425b1c7a0ea1c04f6c70fe39a2c228081a032dbba954bde9 Jul 13 18:38:03.908121 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:03.908082 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9nwgn" event={"ID":"908fd811-9587-407d-92c6-c1d37f9f5163","Type":"ContainerStarted","Data":"030a3f2375e513e1425b1c7a0ea1c04f6c70fe39a2c228081a032dbba954bde9"} Jul 13 18:38:07.927728 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:07.927695 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9nwgn" event={"ID":"908fd811-9587-407d-92c6-c1d37f9f5163","Type":"ContainerStarted","Data":"a5f50bee9cce14ccaa9c463b16c7df75c14240e12f7a93735fa89a8ef0e177c8"} Jul 13 18:38:07.946321 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:07.946260 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9nwgn" podStartSLOduration=1.14827083 podStartE2EDuration="5.946239615s" totalImagesPullingTime="4.797968785s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:38:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:38:07.944840449 +0000 UTC m=+344.228269122" watchObservedRunningTime="2026-07-13 18:38:07.946239615 +0000 UTC m=+344.229668217" Jul 13 18:38:08.722937 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:08.722905 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:38:08.823290 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:08.823259 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:38:10.934342 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:10.934309 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:38:21.160271 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:21.160183 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:38:21.207503 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:21.207408 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7f7b869d75-hgcsz"] Jul 13 18:38:21.214203 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:21.214175 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7f7b869d75-hgcsz" Jul 13 18:38:21.216964 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:21.216937 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 13 18:38:21.217092 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:21.216982 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 13 18:38:21.217148 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:21.217113 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 13 18:38:21.217210 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:21.217186 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-jppj5\"" Jul 13 18:38:21.225473 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:21.225450 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7f7b869d75-hgcsz"] Jul 13 18:38:21.234471 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:21.234448 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:38:21.339055 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:21.339016 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/06386055-ce8e-465f-950a-ab247881f51b-cert\") pod \"lws-controller-manager-7f7b869d75-hgcsz\" (UID: \"06386055-ce8e-465f-950a-ab247881f51b\") " pod="openshift-lws-operator/lws-controller-manager-7f7b869d75-hgcsz" Jul 13 18:38:21.339231 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:21.339119 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/06386055-ce8e-465f-950a-ab247881f51b-metrics-cert\") pod \"lws-controller-manager-7f7b869d75-hgcsz\" (UID: \"06386055-ce8e-465f-950a-ab247881f51b\") " pod="openshift-lws-operator/lws-controller-manager-7f7b869d75-hgcsz" Jul 13 18:38:21.339231 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:21.339141 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvtrr\" (UniqueName: \"kubernetes.io/projected/06386055-ce8e-465f-950a-ab247881f51b-kube-api-access-vvtrr\") pod \"lws-controller-manager-7f7b869d75-hgcsz\" (UID: \"06386055-ce8e-465f-950a-ab247881f51b\") " pod="openshift-lws-operator/lws-controller-manager-7f7b869d75-hgcsz" Jul 13 18:38:21.339309 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:21.339216 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/06386055-ce8e-465f-950a-ab247881f51b-manager-config\") pod \"lws-controller-manager-7f7b869d75-hgcsz\" (UID: \"06386055-ce8e-465f-950a-ab247881f51b\") " pod="openshift-lws-operator/lws-controller-manager-7f7b869d75-hgcsz" Jul 13 18:38:21.440061 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:21.439976 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/06386055-ce8e-465f-950a-ab247881f51b-cert\") pod \"lws-controller-manager-7f7b869d75-hgcsz\" (UID: \"06386055-ce8e-465f-950a-ab247881f51b\") " pod="openshift-lws-operator/lws-controller-manager-7f7b869d75-hgcsz" Jul 13 18:38:21.440061 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:21.440049 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/06386055-ce8e-465f-950a-ab247881f51b-metrics-cert\") pod \"lws-controller-manager-7f7b869d75-hgcsz\" (UID: \"06386055-ce8e-465f-950a-ab247881f51b\") " pod="openshift-lws-operator/lws-controller-manager-7f7b869d75-hgcsz" Jul 13 18:38:21.440280 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:21.440075 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvtrr\" (UniqueName: \"kubernetes.io/projected/06386055-ce8e-465f-950a-ab247881f51b-kube-api-access-vvtrr\") pod \"lws-controller-manager-7f7b869d75-hgcsz\" (UID: \"06386055-ce8e-465f-950a-ab247881f51b\") " pod="openshift-lws-operator/lws-controller-manager-7f7b869d75-hgcsz" Jul 13 18:38:21.440280 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:21.440112 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/06386055-ce8e-465f-950a-ab247881f51b-manager-config\") pod \"lws-controller-manager-7f7b869d75-hgcsz\" (UID: \"06386055-ce8e-465f-950a-ab247881f51b\") " pod="openshift-lws-operator/lws-controller-manager-7f7b869d75-hgcsz" Jul 13 18:38:21.440816 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:21.440785 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/06386055-ce8e-465f-950a-ab247881f51b-manager-config\") pod \"lws-controller-manager-7f7b869d75-hgcsz\" (UID: \"06386055-ce8e-465f-950a-ab247881f51b\") " pod="openshift-lws-operator/lws-controller-manager-7f7b869d75-hgcsz" Jul 13 18:38:21.442547 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:21.442520 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/06386055-ce8e-465f-950a-ab247881f51b-cert\") pod \"lws-controller-manager-7f7b869d75-hgcsz\" (UID: \"06386055-ce8e-465f-950a-ab247881f51b\") " pod="openshift-lws-operator/lws-controller-manager-7f7b869d75-hgcsz" Jul 13 18:38:21.442666 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:21.442563 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/06386055-ce8e-465f-950a-ab247881f51b-metrics-cert\") pod \"lws-controller-manager-7f7b869d75-hgcsz\" (UID: \"06386055-ce8e-465f-950a-ab247881f51b\") " pod="openshift-lws-operator/lws-controller-manager-7f7b869d75-hgcsz" Jul 13 18:38:21.450222 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:21.450199 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvtrr\" (UniqueName: \"kubernetes.io/projected/06386055-ce8e-465f-950a-ab247881f51b-kube-api-access-vvtrr\") pod \"lws-controller-manager-7f7b869d75-hgcsz\" (UID: \"06386055-ce8e-465f-950a-ab247881f51b\") " pod="openshift-lws-operator/lws-controller-manager-7f7b869d75-hgcsz" Jul 13 18:38:21.524366 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:21.524331 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7f7b869d75-hgcsz" Jul 13 18:38:21.665293 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:21.665259 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7f7b869d75-hgcsz"] Jul 13 18:38:21.668382 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:38:21.668354 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06386055_ce8e_465f_950a_ab247881f51b.slice/crio-95ff1e93c247fabca18200edcd799e20c47da2f966974254767268c6000405e1 WatchSource:0}: Error finding container 95ff1e93c247fabca18200edcd799e20c47da2f966974254767268c6000405e1: Status 404 returned error can't find the container with id 95ff1e93c247fabca18200edcd799e20c47da2f966974254767268c6000405e1 Jul 13 18:38:21.985452 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:21.985415 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7f7b869d75-hgcsz" event={"ID":"06386055-ce8e-465f-950a-ab247881f51b","Type":"ContainerStarted","Data":"95ff1e93c247fabca18200edcd799e20c47da2f966974254767268c6000405e1"} Jul 13 18:38:24.000836 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:24.000748 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7f7b869d75-hgcsz" event={"ID":"06386055-ce8e-465f-950a-ab247881f51b","Type":"ContainerStarted","Data":"8d45d1981fa4695038362c477768044b0b74cbc8b021190bb1a8684640adb7af"} Jul 13 18:38:24.001194 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:24.000855 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7f7b869d75-hgcsz" Jul 13 18:38:24.026409 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:24.026361 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7f7b869d75-hgcsz" podStartSLOduration=0.955789686 podStartE2EDuration="3.02634617s" totalImagesPullingTime="2.070556484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:38:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:38:24.025669564 +0000 UTC m=+360.309098186" watchObservedRunningTime="2026-07-13 18:38:24.02634617 +0000 UTC m=+360.309774772" Jul 13 18:38:28.681434 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:28.681399 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:38:28.788120 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:28.788076 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:38:29.296638 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:29.296591 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:38:29.666040 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:29.665948 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:38:29.779921 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:29.779889 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:38:29.808323 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:29.808293 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b4789cfbf-tpzzr"] Jul 13 18:38:29.810883 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:29.810868 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-tpzzr" Jul 13 18:38:29.813964 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:29.813934 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 13 18:38:29.814083 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:29.814035 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-vdvqz\"" Jul 13 18:38:29.814250 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:29.814205 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 13 18:38:29.814336 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:29.814253 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 13 18:38:29.814336 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:29.814295 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 13 18:38:29.831061 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:29.831037 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b4789cfbf-tpzzr"] Jul 13 18:38:29.919804 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:29.919718 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e9fbcb61-a250-427c-9ba9-4d5ca449c193-manifests\") pod \"opendatahub-operator-controller-manager-b4789cfbf-tpzzr\" (UID: \"e9fbcb61-a250-427c-9ba9-4d5ca449c193\") " pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-tpzzr" Jul 13 18:38:29.919804 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:29.919757 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e9fbcb61-a250-427c-9ba9-4d5ca449c193-webhook-cert\") pod \"opendatahub-operator-controller-manager-b4789cfbf-tpzzr\" (UID: \"e9fbcb61-a250-427c-9ba9-4d5ca449c193\") " pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-tpzzr" Jul 13 18:38:29.919997 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:29.919866 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v45bj\" (UniqueName: \"kubernetes.io/projected/e9fbcb61-a250-427c-9ba9-4d5ca449c193-kube-api-access-v45bj\") pod \"opendatahub-operator-controller-manager-b4789cfbf-tpzzr\" (UID: \"e9fbcb61-a250-427c-9ba9-4d5ca449c193\") " pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-tpzzr" Jul 13 18:38:29.919997 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:29.919919 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e9fbcb61-a250-427c-9ba9-4d5ca449c193-tmp\") pod \"opendatahub-operator-controller-manager-b4789cfbf-tpzzr\" (UID: \"e9fbcb61-a250-427c-9ba9-4d5ca449c193\") " pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-tpzzr" Jul 13 18:38:29.919997 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:29.919949 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e9fbcb61-a250-427c-9ba9-4d5ca449c193-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b4789cfbf-tpzzr\" (UID: \"e9fbcb61-a250-427c-9ba9-4d5ca449c193\") " pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-tpzzr" Jul 13 18:38:29.940010 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:29.939959 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:38:30.021229 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:30.021195 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v45bj\" (UniqueName: \"kubernetes.io/projected/e9fbcb61-a250-427c-9ba9-4d5ca449c193-kube-api-access-v45bj\") pod \"opendatahub-operator-controller-manager-b4789cfbf-tpzzr\" (UID: \"e9fbcb61-a250-427c-9ba9-4d5ca449c193\") " pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-tpzzr" Jul 13 18:38:30.021424 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:30.021241 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e9fbcb61-a250-427c-9ba9-4d5ca449c193-tmp\") pod \"opendatahub-operator-controller-manager-b4789cfbf-tpzzr\" (UID: \"e9fbcb61-a250-427c-9ba9-4d5ca449c193\") " pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-tpzzr" Jul 13 18:38:30.021424 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:30.021263 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e9fbcb61-a250-427c-9ba9-4d5ca449c193-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b4789cfbf-tpzzr\" (UID: \"e9fbcb61-a250-427c-9ba9-4d5ca449c193\") " pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-tpzzr" Jul 13 18:38:30.021532 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:30.021443 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e9fbcb61-a250-427c-9ba9-4d5ca449c193-manifests\") pod \"opendatahub-operator-controller-manager-b4789cfbf-tpzzr\" (UID: \"e9fbcb61-a250-427c-9ba9-4d5ca449c193\") " pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-tpzzr" Jul 13 18:38:30.021532 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:30.021490 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e9fbcb61-a250-427c-9ba9-4d5ca449c193-webhook-cert\") pod \"opendatahub-operator-controller-manager-b4789cfbf-tpzzr\" (UID: \"e9fbcb61-a250-427c-9ba9-4d5ca449c193\") " pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-tpzzr" Jul 13 18:38:30.021792 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:30.021775 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e9fbcb61-a250-427c-9ba9-4d5ca449c193-manifests\") pod \"opendatahub-operator-controller-manager-b4789cfbf-tpzzr\" (UID: \"e9fbcb61-a250-427c-9ba9-4d5ca449c193\") " pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-tpzzr" Jul 13 18:38:30.023744 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:30.023718 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e9fbcb61-a250-427c-9ba9-4d5ca449c193-tmp\") pod \"opendatahub-operator-controller-manager-b4789cfbf-tpzzr\" (UID: \"e9fbcb61-a250-427c-9ba9-4d5ca449c193\") " pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-tpzzr" Jul 13 18:38:30.023957 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:30.023940 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e9fbcb61-a250-427c-9ba9-4d5ca449c193-webhook-cert\") pod \"opendatahub-operator-controller-manager-b4789cfbf-tpzzr\" (UID: \"e9fbcb61-a250-427c-9ba9-4d5ca449c193\") " pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-tpzzr" Jul 13 18:38:30.024019 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:30.024000 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e9fbcb61-a250-427c-9ba9-4d5ca449c193-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b4789cfbf-tpzzr\" (UID: \"e9fbcb61-a250-427c-9ba9-4d5ca449c193\") " pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-tpzzr" Jul 13 18:38:30.030137 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:30.030117 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v45bj\" (UniqueName: \"kubernetes.io/projected/e9fbcb61-a250-427c-9ba9-4d5ca449c193-kube-api-access-v45bj\") pod \"opendatahub-operator-controller-manager-b4789cfbf-tpzzr\" (UID: \"e9fbcb61-a250-427c-9ba9-4d5ca449c193\") " pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-tpzzr" Jul 13 18:38:30.035763 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:30.035735 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:38:30.121492 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:30.121459 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-tpzzr" Jul 13 18:38:30.137133 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:30.137103 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:38:30.237377 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:30.237340 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:38:30.270345 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:30.270312 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b4789cfbf-tpzzr"] Jul 13 18:38:30.273654 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:38:30.273628 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9fbcb61_a250_427c_9ba9_4d5ca449c193.slice/crio-445fd5e65a0d51a1554cc5ac439ab7b50787dbf8fe4ec6dc26d723cffd19d365 WatchSource:0}: Error finding container 445fd5e65a0d51a1554cc5ac439ab7b50787dbf8fe4ec6dc26d723cffd19d365: Status 404 returned error can't find the container with id 445fd5e65a0d51a1554cc5ac439ab7b50787dbf8fe4ec6dc26d723cffd19d365 Jul 13 18:38:30.356099 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:30.356067 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:38:31.034122 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:31.034082 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-tpzzr" event={"ID":"e9fbcb61-a250-427c-9ba9-4d5ca449c193","Type":"ContainerStarted","Data":"445fd5e65a0d51a1554cc5ac439ab7b50787dbf8fe4ec6dc26d723cffd19d365"} Jul 13 18:38:34.055349 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:34.055310 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-tpzzr" event={"ID":"e9fbcb61-a250-427c-9ba9-4d5ca449c193","Type":"ContainerStarted","Data":"4afb1d93b42bde8001bbb5ba2d78a88bab9bf759f9aae97c87ecbade50648b9c"} Jul 13 18:38:35.007373 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:35.007345 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7f7b869d75-hgcsz" Jul 13 18:38:35.064333 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:35.064298 2582 generic.go:356] "Generic (PLEG): container finished" podID="e9fbcb61-a250-427c-9ba9-4d5ca449c193" containerID="4afb1d93b42bde8001bbb5ba2d78a88bab9bf759f9aae97c87ecbade50648b9c" exitCode=0 Jul 13 18:38:35.064775 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:35.064337 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-tpzzr" event={"ID":"e9fbcb61-a250-427c-9ba9-4d5ca449c193","Type":"ContainerDied","Data":"4afb1d93b42bde8001bbb5ba2d78a88bab9bf759f9aae97c87ecbade50648b9c"} Jul 13 18:38:36.071093 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:36.071058 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-tpzzr" event={"ID":"e9fbcb61-a250-427c-9ba9-4d5ca449c193","Type":"ContainerStarted","Data":"d8006e3e783e83bf7b5469494eead27e70460a3fcb3fd7e16d104a4b833a3094"} Jul 13 18:38:36.071431 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:36.071160 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-tpzzr" Jul 13 18:38:36.094108 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:36.094059 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-tpzzr" podStartSLOduration=2.634846661 podStartE2EDuration="7.094043624s" totalImagesPullingTime="3.427414494s" totalInitContainerRuntime="1.031782469s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:38:29 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:38:36.092636096 +0000 UTC m=+372.376064721" watchObservedRunningTime="2026-07-13 18:38:36.094043624 +0000 UTC m=+372.377472225" Jul 13 18:38:47.079075 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:47.079043 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-tpzzr" Jul 13 18:38:48.884542 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:48.884493 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:38:49.052310 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:49.052276 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:38:50.179064 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:50.179030 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:38:50.321128 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:50.321091 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:38:50.519205 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:50.519170 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:38:50.618543 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:50.618501 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:38:51.028278 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:51.028246 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:38:51.103076 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:38:51.103039 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:39:00.180681 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:00.180642 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:39:00.231012 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:00.230981 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-66f8768797-j4bdx"] Jul 13 18:39:00.233656 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:00.233635 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-66f8768797-j4bdx" Jul 13 18:39:00.239617 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:00.239575 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 13 18:39:00.240837 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:00.240811 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 13 18:39:00.240941 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:00.240856 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 13 18:39:00.240941 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:00.240891 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 13 18:39:00.241080 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:00.241063 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-5qx9d\"" Jul 13 18:39:00.258615 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:00.258580 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-66f8768797-j4bdx"] Jul 13 18:39:00.342025 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:00.341978 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:39:00.391400 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:00.391371 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtm9h\" (UniqueName: \"kubernetes.io/projected/cc526317-542a-4c45-81c4-fac064ee4661-kube-api-access-vtm9h\") pod \"kube-auth-proxy-66f8768797-j4bdx\" (UID: \"cc526317-542a-4c45-81c4-fac064ee4661\") " pod="openshift-ingress/kube-auth-proxy-66f8768797-j4bdx" Jul 13 18:39:00.391561 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:00.391438 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cc526317-542a-4c45-81c4-fac064ee4661-tmp\") pod \"kube-auth-proxy-66f8768797-j4bdx\" (UID: \"cc526317-542a-4c45-81c4-fac064ee4661\") " pod="openshift-ingress/kube-auth-proxy-66f8768797-j4bdx" Jul 13 18:39:00.391561 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:00.391461 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc526317-542a-4c45-81c4-fac064ee4661-tls-certs\") pod \"kube-auth-proxy-66f8768797-j4bdx\" (UID: \"cc526317-542a-4c45-81c4-fac064ee4661\") " pod="openshift-ingress/kube-auth-proxy-66f8768797-j4bdx" Jul 13 18:39:00.492214 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:00.492178 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtm9h\" (UniqueName: \"kubernetes.io/projected/cc526317-542a-4c45-81c4-fac064ee4661-kube-api-access-vtm9h\") pod \"kube-auth-proxy-66f8768797-j4bdx\" (UID: \"cc526317-542a-4c45-81c4-fac064ee4661\") " pod="openshift-ingress/kube-auth-proxy-66f8768797-j4bdx" Jul 13 18:39:00.492409 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:00.492268 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cc526317-542a-4c45-81c4-fac064ee4661-tmp\") pod \"kube-auth-proxy-66f8768797-j4bdx\" (UID: \"cc526317-542a-4c45-81c4-fac064ee4661\") " pod="openshift-ingress/kube-auth-proxy-66f8768797-j4bdx" Jul 13 18:39:00.492409 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:00.492301 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc526317-542a-4c45-81c4-fac064ee4661-tls-certs\") pod \"kube-auth-proxy-66f8768797-j4bdx\" (UID: \"cc526317-542a-4c45-81c4-fac064ee4661\") " pod="openshift-ingress/kube-auth-proxy-66f8768797-j4bdx" Jul 13 18:39:00.494871 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:00.494847 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cc526317-542a-4c45-81c4-fac064ee4661-tmp\") pod \"kube-auth-proxy-66f8768797-j4bdx\" (UID: \"cc526317-542a-4c45-81c4-fac064ee4661\") " pod="openshift-ingress/kube-auth-proxy-66f8768797-j4bdx" Jul 13 18:39:00.495038 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:00.495015 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc526317-542a-4c45-81c4-fac064ee4661-tls-certs\") pod \"kube-auth-proxy-66f8768797-j4bdx\" (UID: \"cc526317-542a-4c45-81c4-fac064ee4661\") " pod="openshift-ingress/kube-auth-proxy-66f8768797-j4bdx" Jul 13 18:39:00.507705 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:00.507682 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtm9h\" (UniqueName: \"kubernetes.io/projected/cc526317-542a-4c45-81c4-fac064ee4661-kube-api-access-vtm9h\") pod \"kube-auth-proxy-66f8768797-j4bdx\" (UID: \"cc526317-542a-4c45-81c4-fac064ee4661\") " pod="openshift-ingress/kube-auth-proxy-66f8768797-j4bdx" Jul 13 18:39:00.519983 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:00.519960 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:39:00.543088 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:00.543063 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-66f8768797-j4bdx" Jul 13 18:39:00.661532 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:00.661504 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:39:00.697319 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:00.697297 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-66f8768797-j4bdx"] Jul 13 18:39:00.699806 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:39:00.699783 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc526317_542a_4c45_81c4_fac064ee4661.slice/crio-0c4553cb58ca0740a6a67567b2470f8537600caf9341bb9c6d1ff585d57ae878 WatchSource:0}: Error finding container 0c4553cb58ca0740a6a67567b2470f8537600caf9341bb9c6d1ff585d57ae878: Status 404 returned error can't find the container with id 0c4553cb58ca0740a6a67567b2470f8537600caf9341bb9c6d1ff585d57ae878 Jul 13 18:39:01.099524 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:01.099475 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:39:01.193154 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:01.193121 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-66f8768797-j4bdx" event={"ID":"cc526317-542a-4c45-81c4-fac064ee4661","Type":"ContainerStarted","Data":"0c4553cb58ca0740a6a67567b2470f8537600caf9341bb9c6d1ff585d57ae878"} Jul 13 18:39:01.851467 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:01.851433 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:39:01.977219 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:01.977177 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:39:02.847819 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:02.847787 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:39:03.311449 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:03.311415 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:39:03.455260 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:03.455224 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:39:04.209347 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:04.209302 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-66f8768797-j4bdx" event={"ID":"cc526317-542a-4c45-81c4-fac064ee4661","Type":"ContainerStarted","Data":"920b31361304197fbb119bdfcf29f54a42b6b94b97c297b327657181c5129b19"} Jul 13 18:39:04.248613 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:04.248557 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-66f8768797-j4bdx" podStartSLOduration=1.406497975 podStartE2EDuration="4.248536953s" totalImagesPullingTime="2.842038978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:39:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:39:04.246891378 +0000 UTC m=+400.530320000" watchObservedRunningTime="2026-07-13 18:39:04.248536953 +0000 UTC m=+400.531965555" Jul 13 18:39:05.132953 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:05.132912 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:39:06.529368 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:06.529329 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:39:06.633968 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:06.633935 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:39:09.439970 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:09.439942 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:39:09.516088 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:09.516056 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:39:09.820087 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:09.820045 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:39:09.982178 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:09.982148 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:39:10.080089 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:10.080004 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:39:50.395469 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:50.395389 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:39:50.502888 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:50.502854 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:39:50.603360 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:50.603324 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:39:51.316528 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:51.316487 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:39:51.521721 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:39:51.521674 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:40:10.090305 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:10.090265 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:40:10.192852 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:10.192813 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:40:10.291827 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:10.291797 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:40:10.838312 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:10.838283 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:40:10.895851 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:10.895825 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:40:16.175495 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:16.175462 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:40:19.621217 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:19.621174 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:40:20.380767 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:20.380735 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:40:20.659725 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:20.659632 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:40:20.762103 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:20.762071 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:40:20.861367 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:20.861323 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:40:20.964933 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:20.964899 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:40:23.554340 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:23.554307 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:40:23.984922 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:23.984890 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:40:26.509835 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:26.509791 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:40:27.606325 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:27.606289 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:40:29.179902 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:29.179865 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:40:30.417809 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:30.417776 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:40:30.923021 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:30.922992 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-5n7cf"] Jul 13 18:40:30.926594 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:30.926576 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5n7cf" Jul 13 18:40:30.931292 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:30.931270 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 13 18:40:30.931401 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:30.931295 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 13 18:40:30.932456 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:30.932438 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-2g4b7\"" Jul 13 18:40:30.932568 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:30.932446 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 13 18:40:30.943222 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:30.943201 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-5n7cf"] Jul 13 18:40:31.117683 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:31.117648 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxthv\" (UniqueName: \"kubernetes.io/projected/ad280742-6869-4e53-8348-b4f3e578d870-kube-api-access-jxthv\") pod \"dns-operator-controller-manager-864dd78c4c-5n7cf\" (UID: \"ad280742-6869-4e53-8348-b4f3e578d870\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5n7cf" Jul 13 18:40:31.218553 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:31.218520 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxthv\" (UniqueName: \"kubernetes.io/projected/ad280742-6869-4e53-8348-b4f3e578d870-kube-api-access-jxthv\") pod \"dns-operator-controller-manager-864dd78c4c-5n7cf\" (UID: \"ad280742-6869-4e53-8348-b4f3e578d870\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5n7cf" Jul 13 18:40:31.229006 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:31.228982 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxthv\" (UniqueName: \"kubernetes.io/projected/ad280742-6869-4e53-8348-b4f3e578d870-kube-api-access-jxthv\") pod \"dns-operator-controller-manager-864dd78c4c-5n7cf\" (UID: \"ad280742-6869-4e53-8348-b4f3e578d870\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5n7cf" Jul 13 18:40:31.238727 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:31.238703 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5n7cf" Jul 13 18:40:31.389192 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:31.389105 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-5n7cf"] Jul 13 18:40:31.393469 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:40:31.393435 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad280742_6869_4e53_8348_b4f3e578d870.slice/crio-bffcda8776fdd65e3fbc09b37522168528cada76b74a7be9c7a593704098b720 WatchSource:0}: Error finding container bffcda8776fdd65e3fbc09b37522168528cada76b74a7be9c7a593704098b720: Status 404 returned error can't find the container with id bffcda8776fdd65e3fbc09b37522168528cada76b74a7be9c7a593704098b720 Jul 13 18:40:31.602040 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:31.601952 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5n7cf" event={"ID":"ad280742-6869-4e53-8348-b4f3e578d870","Type":"ContainerStarted","Data":"bffcda8776fdd65e3fbc09b37522168528cada76b74a7be9c7a593704098b720"} Jul 13 18:40:31.896225 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:31.896141 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:40:32.384930 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:32.384869 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:40:32.487109 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:32.487079 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:40:35.621593 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:35.621560 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5n7cf" event={"ID":"ad280742-6869-4e53-8348-b4f3e578d870","Type":"ContainerStarted","Data":"d9b6e118773528fc5179d21faa9ee2b684e0a8f238c0c87d7e0d917e86190c1b"} Jul 13 18:40:35.621954 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:35.621627 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5n7cf" Jul 13 18:40:35.664953 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:35.664908 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5n7cf" podStartSLOduration=2.055350602 podStartE2EDuration="5.664894397s" totalImagesPullingTime="3.609543795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:40:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:40:35.663978077 +0000 UTC m=+491.947406678" watchObservedRunningTime="2026-07-13 18:40:35.664894397 +0000 UTC m=+491.948322998" Jul 13 18:40:36.396316 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:36.396279 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:40:36.521234 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:36.521200 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:40:36.916021 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:36.915990 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:40:37.031932 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:37.031905 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:40:37.989700 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:37.989656 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-cnls6"] Jul 13 18:40:37.994888 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:37.994864 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cnls6" Jul 13 18:40:37.997904 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:37.997868 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 13 18:40:37.998473 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:37.998454 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 13 18:40:37.998473 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:37.998471 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kkqv9\"" Jul 13 18:40:38.000467 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:38.000444 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:40:38.007279 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:38.007258 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-cnls6"] Jul 13 18:40:38.072827 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:38.072801 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2f436467-347d-4bf3-9194-7bf0a7acab6e-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-cnls6\" (UID: \"2f436467-347d-4bf3-9194-7bf0a7acab6e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cnls6" Jul 13 18:40:38.073010 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:38.072842 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crtdr\" (UniqueName: \"kubernetes.io/projected/2f436467-347d-4bf3-9194-7bf0a7acab6e-kube-api-access-crtdr\") pod \"kuadrant-console-plugin-66b5774985-cnls6\" (UID: \"2f436467-347d-4bf3-9194-7bf0a7acab6e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cnls6" Jul 13 18:40:38.073010 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:38.072976 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/2f436467-347d-4bf3-9194-7bf0a7acab6e-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-cnls6\" (UID: \"2f436467-347d-4bf3-9194-7bf0a7acab6e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cnls6" Jul 13 18:40:38.174353 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:38.174329 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/2f436467-347d-4bf3-9194-7bf0a7acab6e-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-cnls6\" (UID: \"2f436467-347d-4bf3-9194-7bf0a7acab6e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cnls6" Jul 13 18:40:38.174511 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:38.174379 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2f436467-347d-4bf3-9194-7bf0a7acab6e-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-cnls6\" (UID: \"2f436467-347d-4bf3-9194-7bf0a7acab6e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cnls6" Jul 13 18:40:38.174511 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:40:38.174470 2582 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 13 18:40:38.174656 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:38.174505 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crtdr\" (UniqueName: \"kubernetes.io/projected/2f436467-347d-4bf3-9194-7bf0a7acab6e-kube-api-access-crtdr\") pod \"kuadrant-console-plugin-66b5774985-cnls6\" (UID: \"2f436467-347d-4bf3-9194-7bf0a7acab6e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cnls6" Jul 13 18:40:38.174656 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:40:38.174532 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f436467-347d-4bf3-9194-7bf0a7acab6e-plugin-serving-cert podName:2f436467-347d-4bf3-9194-7bf0a7acab6e nodeName:}" failed. No retries permitted until 2026-07-13 18:40:38.674515963 +0000 UTC m=+494.957944545 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/2f436467-347d-4bf3-9194-7bf0a7acab6e-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-cnls6" (UID: "2f436467-347d-4bf3-9194-7bf0a7acab6e") : secret "plugin-serving-cert" not found Jul 13 18:40:38.174964 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:38.174947 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2f436467-347d-4bf3-9194-7bf0a7acab6e-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-cnls6\" (UID: \"2f436467-347d-4bf3-9194-7bf0a7acab6e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cnls6" Jul 13 18:40:38.188321 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:38.188301 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crtdr\" (UniqueName: \"kubernetes.io/projected/2f436467-347d-4bf3-9194-7bf0a7acab6e-kube-api-access-crtdr\") pod \"kuadrant-console-plugin-66b5774985-cnls6\" (UID: \"2f436467-347d-4bf3-9194-7bf0a7acab6e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cnls6" Jul 13 18:40:38.456042 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:38.456007 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:40:38.679655 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:38.679622 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/2f436467-347d-4bf3-9194-7bf0a7acab6e-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-cnls6\" (UID: \"2f436467-347d-4bf3-9194-7bf0a7acab6e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cnls6" Jul 13 18:40:38.682067 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:38.682039 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/2f436467-347d-4bf3-9194-7bf0a7acab6e-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-cnls6\" (UID: \"2f436467-347d-4bf3-9194-7bf0a7acab6e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cnls6" Jul 13 18:40:38.904398 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:38.904323 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cnls6" Jul 13 18:40:39.038663 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:39.038638 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-cnls6"] Jul 13 18:40:39.040749 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:40:39.040720 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2f436467_347d_4bf3_9194_7bf0a7acab6e.slice/crio-1da9482c83f039870d0470ba6d518967ca58d835c4d2d9f56d43437c8b43a0a3 WatchSource:0}: Error finding container 1da9482c83f039870d0470ba6d518967ca58d835c4d2d9f56d43437c8b43a0a3: Status 404 returned error can't find the container with id 1da9482c83f039870d0470ba6d518967ca58d835c4d2d9f56d43437c8b43a0a3 Jul 13 18:40:39.640568 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:39.640532 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cnls6" event={"ID":"2f436467-347d-4bf3-9194-7bf0a7acab6e","Type":"ContainerStarted","Data":"1da9482c83f039870d0470ba6d518967ca58d835c4d2d9f56d43437c8b43a0a3"} Jul 13 18:40:40.612233 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:40.612195 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:40:40.714033 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:40.713990 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:40:46.628579 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:46.628547 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5n7cf" Jul 13 18:40:47.424760 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:47.424718 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:40:47.566665 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:47.566557 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:40:48.243687 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:48.243650 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mq79h"] Jul 13 18:40:48.247520 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:48.247497 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mq79h" Jul 13 18:40:48.251619 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:48.251575 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-hv92m\"" Jul 13 18:40:48.266843 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:48.266813 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mq79h"] Jul 13 18:40:48.374747 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:48.374717 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdsdl\" (UniqueName: \"kubernetes.io/projected/ff654a92-d36e-458b-aa26-d6b67a36ba9c-kube-api-access-hdsdl\") pod \"kuadrant-operator-controller-manager-55467d9f7-mq79h\" (UID: \"ff654a92-d36e-458b-aa26-d6b67a36ba9c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mq79h" Jul 13 18:40:48.374898 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:48.374776 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff654a92-d36e-458b-aa26-d6b67a36ba9c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-mq79h\" (UID: \"ff654a92-d36e-458b-aa26-d6b67a36ba9c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mq79h" Jul 13 18:40:48.476044 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:48.476017 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdsdl\" (UniqueName: \"kubernetes.io/projected/ff654a92-d36e-458b-aa26-d6b67a36ba9c-kube-api-access-hdsdl\") pod \"kuadrant-operator-controller-manager-55467d9f7-mq79h\" (UID: \"ff654a92-d36e-458b-aa26-d6b67a36ba9c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mq79h" Jul 13 18:40:48.476200 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:48.476063 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff654a92-d36e-458b-aa26-d6b67a36ba9c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-mq79h\" (UID: \"ff654a92-d36e-458b-aa26-d6b67a36ba9c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mq79h" Jul 13 18:40:48.476459 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:48.476441 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff654a92-d36e-458b-aa26-d6b67a36ba9c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-mq79h\" (UID: \"ff654a92-d36e-458b-aa26-d6b67a36ba9c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mq79h" Jul 13 18:40:48.491878 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:48.491851 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdsdl\" (UniqueName: \"kubernetes.io/projected/ff654a92-d36e-458b-aa26-d6b67a36ba9c-kube-api-access-hdsdl\") pod \"kuadrant-operator-controller-manager-55467d9f7-mq79h\" (UID: \"ff654a92-d36e-458b-aa26-d6b67a36ba9c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mq79h" Jul 13 18:40:48.525550 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:48.525480 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mq79h"] Jul 13 18:40:48.525838 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:48.525823 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mq79h" Jul 13 18:40:48.531727 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:48.531668 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mq79h"] Jul 13 18:40:48.572719 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:48.572661 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jmnvw"] Jul 13 18:40:48.577635 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:48.577589 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jmnvw" Jul 13 18:40:48.603199 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:48.603176 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jmnvw"] Jul 13 18:40:48.678160 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:48.678130 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7g8x\" (UniqueName: \"kubernetes.io/projected/18077f97-2643-4cb8-aea6-adc682810628-kube-api-access-f7g8x\") pod \"kuadrant-operator-controller-manager-55467d9f7-jmnvw\" (UID: \"18077f97-2643-4cb8-aea6-adc682810628\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jmnvw" Jul 13 18:40:48.678311 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:48.678181 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/18077f97-2643-4cb8-aea6-adc682810628-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-jmnvw\" (UID: \"18077f97-2643-4cb8-aea6-adc682810628\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jmnvw" Jul 13 18:40:48.779092 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:48.778998 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/18077f97-2643-4cb8-aea6-adc682810628-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-jmnvw\" (UID: \"18077f97-2643-4cb8-aea6-adc682810628\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jmnvw" Jul 13 18:40:48.779243 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:48.779105 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7g8x\" (UniqueName: \"kubernetes.io/projected/18077f97-2643-4cb8-aea6-adc682810628-kube-api-access-f7g8x\") pod \"kuadrant-operator-controller-manager-55467d9f7-jmnvw\" (UID: \"18077f97-2643-4cb8-aea6-adc682810628\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jmnvw" Jul 13 18:40:48.779386 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:48.779367 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/18077f97-2643-4cb8-aea6-adc682810628-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-jmnvw\" (UID: \"18077f97-2643-4cb8-aea6-adc682810628\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jmnvw" Jul 13 18:40:48.791836 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:48.791810 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7g8x\" (UniqueName: \"kubernetes.io/projected/18077f97-2643-4cb8-aea6-adc682810628-kube-api-access-f7g8x\") pod \"kuadrant-operator-controller-manager-55467d9f7-jmnvw\" (UID: \"18077f97-2643-4cb8-aea6-adc682810628\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jmnvw" Jul 13 18:40:48.892857 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:48.892821 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jmnvw" Jul 13 18:40:49.048458 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:49.048428 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jmnvw"] Jul 13 18:40:51.462666 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:51.462591 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:40:51.566324 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:51.566293 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:40:55.251015 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:55.250974 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:40:55.319828 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:55.319795 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:40:55.448359 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:55.448320 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:40:55.553739 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:40:55.553652 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:41:00.991177 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:00.991120 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:41:01.091503 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:01.091465 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:41:03.094734 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:41:03.094697 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18077f97_2643_4cb8_aea6_adc682810628.slice/crio-6d47fd372115a1f5e02706f38b20e5dacfcb5b0586b6baa3a59bf944840b44a4 WatchSource:0}: Error finding container 6d47fd372115a1f5e02706f38b20e5dacfcb5b0586b6baa3a59bf944840b44a4: Status 404 returned error can't find the container with id 6d47fd372115a1f5e02706f38b20e5dacfcb5b0586b6baa3a59bf944840b44a4 Jul 13 18:41:03.097027 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:41:03.096990 2582 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-mq79h_kuadrant-system_ff654a92-d36e-458b-aa26-d6b67a36ba9c_0(54d8df0edfcc331c38e0765deb1e3dc271a1ed10a3610354cbaa4d50c78025ab): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-mq79h to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" Jul 13 18:41:03.097127 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:41:03.097052 2582 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-mq79h_kuadrant-system_ff654a92-d36e-458b-aa26-d6b67a36ba9c_0(54d8df0edfcc331c38e0765deb1e3dc271a1ed10a3610354cbaa4d50c78025ab): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-mq79h to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mq79h" Jul 13 18:41:03.788791 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:03.788755 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cnls6" event={"ID":"2f436467-347d-4bf3-9194-7bf0a7acab6e","Type":"ContainerStarted","Data":"6575754baff8f6f4d1a2efc17b895a7e675a48ffe8c3346af84a66feef2189f1"} Jul 13 18:41:03.790142 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:03.790120 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mq79h" Jul 13 18:41:03.790262 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:03.790120 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jmnvw" event={"ID":"18077f97-2643-4cb8-aea6-adc682810628","Type":"ContainerStarted","Data":"6d47fd372115a1f5e02706f38b20e5dacfcb5b0586b6baa3a59bf944840b44a4"} Jul 13 18:41:03.805619 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:03.805562 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cnls6" podStartSLOduration=2.66046199 podStartE2EDuration="26.805543811s" totalImagesPullingTime="24.145081821s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:40:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:41:03.805376841 +0000 UTC m=+520.088805443" watchObservedRunningTime="2026-07-13 18:41:03.805543811 +0000 UTC m=+520.088972413" Jul 13 18:41:03.806986 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:03.806963 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mq79h" Jul 13 18:41:03.807698 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:41:03.807675 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-mq79h\" is forbidden: User \"system:node:ip-10-0-129-191.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-191.ec2.internal' and this object" podUID="ff654a92-d36e-458b-aa26-d6b67a36ba9c" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mq79h" Jul 13 18:41:03.809727 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:41:03.809708 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-mq79h\" is forbidden: User \"system:node:ip-10-0-129-191.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-191.ec2.internal' and this object" podUID="ff654a92-d36e-458b-aa26-d6b67a36ba9c" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mq79h" Jul 13 18:41:03.923977 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:03.923945 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff654a92-d36e-458b-aa26-d6b67a36ba9c-extensions-socket-volume\") pod \"ff654a92-d36e-458b-aa26-d6b67a36ba9c\" (UID: \"ff654a92-d36e-458b-aa26-d6b67a36ba9c\") " Jul 13 18:41:03.924124 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:03.923997 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hdsdl\" (UniqueName: \"kubernetes.io/projected/ff654a92-d36e-458b-aa26-d6b67a36ba9c-kube-api-access-hdsdl\") pod \"ff654a92-d36e-458b-aa26-d6b67a36ba9c\" (UID: \"ff654a92-d36e-458b-aa26-d6b67a36ba9c\") " Jul 13 18:41:03.924205 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:03.924180 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff654a92-d36e-458b-aa26-d6b67a36ba9c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ff654a92-d36e-458b-aa26-d6b67a36ba9c" (UID: "ff654a92-d36e-458b-aa26-d6b67a36ba9c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 18:41:03.924346 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:03.924321 2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff654a92-d36e-458b-aa26-d6b67a36ba9c-extensions-socket-volume\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:41:03.926039 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:03.926018 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff654a92-d36e-458b-aa26-d6b67a36ba9c-kube-api-access-hdsdl" (OuterVolumeSpecName: "kube-api-access-hdsdl") pod "ff654a92-d36e-458b-aa26-d6b67a36ba9c" (UID: "ff654a92-d36e-458b-aa26-d6b67a36ba9c"). InnerVolumeSpecName "kube-api-access-hdsdl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 18:41:04.024919 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:04.024884 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdsdl\" (UniqueName: \"kubernetes.io/projected/ff654a92-d36e-458b-aa26-d6b67a36ba9c-kube-api-access-hdsdl\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:41:04.280216 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:41:04.280179 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-mq79h\" is forbidden: User \"system:node:ip-10-0-129-191.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-191.ec2.internal' and this object" podUID="ff654a92-d36e-458b-aa26-d6b67a36ba9c" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mq79h" Jul 13 18:41:04.283074 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:04.283052 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff654a92-d36e-458b-aa26-d6b67a36ba9c" path="/var/lib/kubelet/pods/ff654a92-d36e-458b-aa26-d6b67a36ba9c/volumes" Jul 13 18:41:04.797829 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:04.797797 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mq79h" Jul 13 18:41:04.800520 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:41:04.800488 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-mq79h\" is forbidden: User \"system:node:ip-10-0-129-191.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-191.ec2.internal' and this object" podUID="ff654a92-d36e-458b-aa26-d6b67a36ba9c" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mq79h" Jul 13 18:41:04.802882 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:41:04.802861 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-mq79h\" is forbidden: User \"system:node:ip-10-0-129-191.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-191.ec2.internal' and this object" podUID="ff654a92-d36e-458b-aa26-d6b67a36ba9c" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mq79h" Jul 13 18:41:05.965615 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:05.965568 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:41:06.063435 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:06.063384 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:41:08.385888 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:08.385830 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:41:08.447656 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:08.447592 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:41:10.829839 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:10.829803 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jmnvw" event={"ID":"18077f97-2643-4cb8-aea6-adc682810628","Type":"ContainerStarted","Data":"a56002f47dbf5cb9dc3f3b1501d48f2be11fdbeb2da3b8db6b1481c2d52a76e6"} Jul 13 18:41:10.830298 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:10.829887 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jmnvw" Jul 13 18:41:10.850344 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:10.850279 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jmnvw" podStartSLOduration=15.763592586 podStartE2EDuration="22.850260701s" totalImagesPullingTime="7.086668115s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:40:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:41:10.848789244 +0000 UTC m=+527.132217867" watchObservedRunningTime="2026-07-13 18:41:10.850260701 +0000 UTC m=+527.133689305" Jul 13 18:41:21.836723 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:21.836642 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jmnvw" Jul 13 18:41:33.452990 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:33.452961 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:41:33.859904 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:33.859825 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jmnvw"] Jul 13 18:41:33.860084 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:33.860012 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jmnvw" podUID="18077f97-2643-4cb8-aea6-adc682810628" containerName="manager" containerID="cri-o://a56002f47dbf5cb9dc3f3b1501d48f2be11fdbeb2da3b8db6b1481c2d52a76e6" gracePeriod=10 Jul 13 18:41:34.641663 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:34.641636 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jmnvw" Jul 13 18:41:34.698349 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:34.698320 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7g8x\" (UniqueName: \"kubernetes.io/projected/18077f97-2643-4cb8-aea6-adc682810628-kube-api-access-f7g8x\") pod \"18077f97-2643-4cb8-aea6-adc682810628\" (UID: \"18077f97-2643-4cb8-aea6-adc682810628\") " Jul 13 18:41:34.698519 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:34.698399 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/18077f97-2643-4cb8-aea6-adc682810628-extensions-socket-volume\") pod \"18077f97-2643-4cb8-aea6-adc682810628\" (UID: \"18077f97-2643-4cb8-aea6-adc682810628\") " Jul 13 18:41:34.698860 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:34.698829 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18077f97-2643-4cb8-aea6-adc682810628-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "18077f97-2643-4cb8-aea6-adc682810628" (UID: "18077f97-2643-4cb8-aea6-adc682810628"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 18:41:34.700821 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:34.700792 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18077f97-2643-4cb8-aea6-adc682810628-kube-api-access-f7g8x" (OuterVolumeSpecName: "kube-api-access-f7g8x") pod "18077f97-2643-4cb8-aea6-adc682810628" (UID: "18077f97-2643-4cb8-aea6-adc682810628"). InnerVolumeSpecName "kube-api-access-f7g8x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 18:41:34.799764 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:34.799701 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7g8x\" (UniqueName: \"kubernetes.io/projected/18077f97-2643-4cb8-aea6-adc682810628-kube-api-access-f7g8x\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:41:34.799764 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:34.799726 2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/18077f97-2643-4cb8-aea6-adc682810628-extensions-socket-volume\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:41:34.941079 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:34.941042 2582 generic.go:356] "Generic (PLEG): container finished" podID="18077f97-2643-4cb8-aea6-adc682810628" containerID="a56002f47dbf5cb9dc3f3b1501d48f2be11fdbeb2da3b8db6b1481c2d52a76e6" exitCode=0 Jul 13 18:41:34.941256 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:34.941095 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jmnvw" Jul 13 18:41:34.941256 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:34.941126 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jmnvw" event={"ID":"18077f97-2643-4cb8-aea6-adc682810628","Type":"ContainerDied","Data":"a56002f47dbf5cb9dc3f3b1501d48f2be11fdbeb2da3b8db6b1481c2d52a76e6"} Jul 13 18:41:34.941256 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:34.941214 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jmnvw" event={"ID":"18077f97-2643-4cb8-aea6-adc682810628","Type":"ContainerDied","Data":"6d47fd372115a1f5e02706f38b20e5dacfcb5b0586b6baa3a59bf944840b44a4"} Jul 13 18:41:34.941256 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:34.941242 2582 scope.go:117] "RemoveContainer" containerID="a56002f47dbf5cb9dc3f3b1501d48f2be11fdbeb2da3b8db6b1481c2d52a76e6" Jul 13 18:41:34.950941 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:34.950922 2582 scope.go:117] "RemoveContainer" containerID="a56002f47dbf5cb9dc3f3b1501d48f2be11fdbeb2da3b8db6b1481c2d52a76e6" Jul 13 18:41:34.951195 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:41:34.951173 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a56002f47dbf5cb9dc3f3b1501d48f2be11fdbeb2da3b8db6b1481c2d52a76e6\": container with ID starting with a56002f47dbf5cb9dc3f3b1501d48f2be11fdbeb2da3b8db6b1481c2d52a76e6 not found: ID does not exist" containerID="a56002f47dbf5cb9dc3f3b1501d48f2be11fdbeb2da3b8db6b1481c2d52a76e6" Jul 13 18:41:34.951296 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:34.951202 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a56002f47dbf5cb9dc3f3b1501d48f2be11fdbeb2da3b8db6b1481c2d52a76e6"} err="failed to get container status \"a56002f47dbf5cb9dc3f3b1501d48f2be11fdbeb2da3b8db6b1481c2d52a76e6\": rpc error: code = NotFound desc = could not find container \"a56002f47dbf5cb9dc3f3b1501d48f2be11fdbeb2da3b8db6b1481c2d52a76e6\": container with ID starting with a56002f47dbf5cb9dc3f3b1501d48f2be11fdbeb2da3b8db6b1481c2d52a76e6 not found: ID does not exist" Jul 13 18:41:34.994029 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:34.994000 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jmnvw"] Jul 13 18:41:34.996543 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:34.996524 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jmnvw"] Jul 13 18:41:36.280842 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:36.280810 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18077f97-2643-4cb8-aea6-adc682810628" path="/var/lib/kubelet/pods/18077f97-2643-4cb8-aea6-adc682810628/volumes" Jul 13 18:41:38.346123 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:38.346092 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:41:38.895941 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:38.895892 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:41:39.390776 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:39.390747 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:41:41.132053 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:41.132025 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:41:41.212266 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:41.212226 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:41:42.138990 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:42.138952 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-77kdg"] Jul 13 18:41:42.139540 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:42.139519 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18077f97-2643-4cb8-aea6-adc682810628" containerName="manager" Jul 13 18:41:42.139649 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:42.139542 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="18077f97-2643-4cb8-aea6-adc682810628" containerName="manager" Jul 13 18:41:42.139709 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:42.139660 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="18077f97-2643-4cb8-aea6-adc682810628" containerName="manager" Jul 13 18:41:42.144576 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:42.144550 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:41:42.156569 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:42.156546 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-77kdg"] Jul 13 18:41:42.156700 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:42.156669 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-77kdg" Jul 13 18:41:42.159646 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:42.159629 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 13 18:41:42.234502 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:42.234474 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-77kdg"] Jul 13 18:41:42.244857 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:42.244825 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:41:42.271336 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:42.271311 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/afb40f22-2b1e-411f-82cb-d072df19fed8-config-file\") pod \"limitador-limitador-64c94fd64f-77kdg\" (UID: \"afb40f22-2b1e-411f-82cb-d072df19fed8\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-77kdg" Jul 13 18:41:42.271433 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:42.271398 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c95f8\" (UniqueName: \"kubernetes.io/projected/afb40f22-2b1e-411f-82cb-d072df19fed8-kube-api-access-c95f8\") pod \"limitador-limitador-64c94fd64f-77kdg\" (UID: \"afb40f22-2b1e-411f-82cb-d072df19fed8\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-77kdg" Jul 13 18:41:42.313943 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:42.313919 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:41:42.372344 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:42.372320 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/afb40f22-2b1e-411f-82cb-d072df19fed8-config-file\") pod \"limitador-limitador-64c94fd64f-77kdg\" (UID: \"afb40f22-2b1e-411f-82cb-d072df19fed8\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-77kdg" Jul 13 18:41:42.372466 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:42.372431 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c95f8\" (UniqueName: \"kubernetes.io/projected/afb40f22-2b1e-411f-82cb-d072df19fed8-kube-api-access-c95f8\") pod \"limitador-limitador-64c94fd64f-77kdg\" (UID: \"afb40f22-2b1e-411f-82cb-d072df19fed8\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-77kdg" Jul 13 18:41:42.372940 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:42.372921 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/afb40f22-2b1e-411f-82cb-d072df19fed8-config-file\") pod \"limitador-limitador-64c94fd64f-77kdg\" (UID: \"afb40f22-2b1e-411f-82cb-d072df19fed8\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-77kdg" Jul 13 18:41:42.381781 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:42.381756 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c95f8\" (UniqueName: \"kubernetes.io/projected/afb40f22-2b1e-411f-82cb-d072df19fed8-kube-api-access-c95f8\") pod \"limitador-limitador-64c94fd64f-77kdg\" (UID: \"afb40f22-2b1e-411f-82cb-d072df19fed8\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-77kdg" Jul 13 18:41:42.466736 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:42.466706 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-77kdg" Jul 13 18:41:42.598257 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:42.598232 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-77kdg"] Jul 13 18:41:42.600360 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:41:42.600332 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafb40f22_2b1e_411f_82cb_d072df19fed8.slice/crio-300d1d4e048eff6ae42e4ebdace63aca799fbc0e32148b5c6fcfbe771dbf00bf WatchSource:0}: Error finding container 300d1d4e048eff6ae42e4ebdace63aca799fbc0e32148b5c6fcfbe771dbf00bf: Status 404 returned error can't find the container with id 300d1d4e048eff6ae42e4ebdace63aca799fbc0e32148b5c6fcfbe771dbf00bf Jul 13 18:41:42.971874 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:42.971839 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:41:42.985127 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:42.985099 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-77kdg" event={"ID":"afb40f22-2b1e-411f-82cb-d072df19fed8","Type":"ContainerStarted","Data":"300d1d4e048eff6ae42e4ebdace63aca799fbc0e32148b5c6fcfbe771dbf00bf"} Jul 13 18:41:43.072482 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:43.072418 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:41:43.139370 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:43.139333 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-w94ml"] Jul 13 18:41:43.142490 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:43.142471 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-w94ml" Jul 13 18:41:43.145048 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:43.145024 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-wb6lw\"" Jul 13 18:41:43.150989 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:43.150969 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-w94ml"] Jul 13 18:41:43.279781 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:43.279710 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wdk4\" (UniqueName: \"kubernetes.io/projected/710e57b0-4774-4494-b2f7-2af1aa5a361c-kube-api-access-8wdk4\") pod \"authorino-75d556bd77-w94ml\" (UID: \"710e57b0-4774-4494-b2f7-2af1aa5a361c\") " pod="kuadrant-system/authorino-75d556bd77-w94ml" Jul 13 18:41:43.380666 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:43.380637 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wdk4\" (UniqueName: \"kubernetes.io/projected/710e57b0-4774-4494-b2f7-2af1aa5a361c-kube-api-access-8wdk4\") pod \"authorino-75d556bd77-w94ml\" (UID: \"710e57b0-4774-4494-b2f7-2af1aa5a361c\") " pod="kuadrant-system/authorino-75d556bd77-w94ml" Jul 13 18:41:43.389717 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:43.389683 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wdk4\" (UniqueName: \"kubernetes.io/projected/710e57b0-4774-4494-b2f7-2af1aa5a361c-kube-api-access-8wdk4\") pod \"authorino-75d556bd77-w94ml\" (UID: \"710e57b0-4774-4494-b2f7-2af1aa5a361c\") " pod="kuadrant-system/authorino-75d556bd77-w94ml" Jul 13 18:41:43.452758 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:43.452730 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-w94ml" Jul 13 18:41:43.623133 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:43.623097 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-w94ml"] Jul 13 18:41:43.626978 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:41:43.626947 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod710e57b0_4774_4494_b2f7_2af1aa5a361c.slice/crio-473598efc52c6b69b2f751ca439d763d4a73fad7c55aa5ebcaebff240d742836 WatchSource:0}: Error finding container 473598efc52c6b69b2f751ca439d763d4a73fad7c55aa5ebcaebff240d742836: Status 404 returned error can't find the container with id 473598efc52c6b69b2f751ca439d763d4a73fad7c55aa5ebcaebff240d742836 Jul 13 18:41:43.992071 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:43.992032 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-w94ml" event={"ID":"710e57b0-4774-4494-b2f7-2af1aa5a361c","Type":"ContainerStarted","Data":"473598efc52c6b69b2f751ca439d763d4a73fad7c55aa5ebcaebff240d742836"} Jul 13 18:41:49.027171 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:49.027132 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-77kdg" event={"ID":"afb40f22-2b1e-411f-82cb-d072df19fed8","Type":"ContainerStarted","Data":"9fe3fea55a13adf89a64166391dfa5ab4ec1c410634abe1561eceda982c37693"} Jul 13 18:41:49.027627 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:49.027194 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-77kdg" Jul 13 18:41:49.028503 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:49.028480 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-w94ml" event={"ID":"710e57b0-4774-4494-b2f7-2af1aa5a361c","Type":"ContainerStarted","Data":"93d307ee182263495c1db186034b58c9271d367db81866e5f19b27db3ca250ea"} Jul 13 18:41:49.046048 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:49.045997 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-77kdg" podStartSLOduration=1.6095369609999999 podStartE2EDuration="7.045984223s" totalImagesPullingTime="5.436447262s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:41:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:41:49.044940143 +0000 UTC m=+565.328368747" watchObservedRunningTime="2026-07-13 18:41:49.045984223 +0000 UTC m=+565.329412857" Jul 13 18:41:49.060111 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:49.060068 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-w94ml" podStartSLOduration=1.649124197 podStartE2EDuration="6.060056159s" totalImagesPullingTime="4.410931962s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:41:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:41:49.058727294 +0000 UTC m=+565.342155897" watchObservedRunningTime="2026-07-13 18:41:49.060056159 +0000 UTC m=+565.343484761" Jul 13 18:41:57.958223 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:57.958183 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-77kdg"] Jul 13 18:41:57.958599 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:57.958410 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-77kdg" podUID="afb40f22-2b1e-411f-82cb-d072df19fed8" containerName="limitador" containerID="cri-o://9fe3fea55a13adf89a64166391dfa5ab4ec1c410634abe1561eceda982c37693" gracePeriod=30 Jul 13 18:41:57.960722 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:57.960692 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-77kdg" Jul 13 18:41:58.511715 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:58.511682 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-77kdg" Jul 13 18:41:58.623524 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:58.623455 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/afb40f22-2b1e-411f-82cb-d072df19fed8-config-file\") pod \"afb40f22-2b1e-411f-82cb-d072df19fed8\" (UID: \"afb40f22-2b1e-411f-82cb-d072df19fed8\") " Jul 13 18:41:58.623524 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:58.623486 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c95f8\" (UniqueName: \"kubernetes.io/projected/afb40f22-2b1e-411f-82cb-d072df19fed8-kube-api-access-c95f8\") pod \"afb40f22-2b1e-411f-82cb-d072df19fed8\" (UID: \"afb40f22-2b1e-411f-82cb-d072df19fed8\") " Jul 13 18:41:58.623856 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:58.623831 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afb40f22-2b1e-411f-82cb-d072df19fed8-config-file" (OuterVolumeSpecName: "config-file") pod "afb40f22-2b1e-411f-82cb-d072df19fed8" (UID: "afb40f22-2b1e-411f-82cb-d072df19fed8"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 18:41:58.625935 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:58.625905 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afb40f22-2b1e-411f-82cb-d072df19fed8-kube-api-access-c95f8" (OuterVolumeSpecName: "kube-api-access-c95f8") pod "afb40f22-2b1e-411f-82cb-d072df19fed8" (UID: "afb40f22-2b1e-411f-82cb-d072df19fed8"). InnerVolumeSpecName "kube-api-access-c95f8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 18:41:58.724505 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:58.724464 2582 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/afb40f22-2b1e-411f-82cb-d072df19fed8-config-file\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:41:58.724505 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:58.724502 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c95f8\" (UniqueName: \"kubernetes.io/projected/afb40f22-2b1e-411f-82cb-d072df19fed8-kube-api-access-c95f8\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:41:59.080138 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:59.080108 2582 generic.go:356] "Generic (PLEG): container finished" podID="afb40f22-2b1e-411f-82cb-d072df19fed8" containerID="9fe3fea55a13adf89a64166391dfa5ab4ec1c410634abe1561eceda982c37693" exitCode=0 Jul 13 18:41:59.080500 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:59.080150 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-77kdg" Jul 13 18:41:59.080500 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:59.080196 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-77kdg" event={"ID":"afb40f22-2b1e-411f-82cb-d072df19fed8","Type":"ContainerDied","Data":"9fe3fea55a13adf89a64166391dfa5ab4ec1c410634abe1561eceda982c37693"} Jul 13 18:41:59.080500 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:59.080247 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-77kdg" event={"ID":"afb40f22-2b1e-411f-82cb-d072df19fed8","Type":"ContainerDied","Data":"300d1d4e048eff6ae42e4ebdace63aca799fbc0e32148b5c6fcfbe771dbf00bf"} Jul 13 18:41:59.080500 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:59.080274 2582 scope.go:117] "RemoveContainer" containerID="9fe3fea55a13adf89a64166391dfa5ab4ec1c410634abe1561eceda982c37693" Jul 13 18:41:59.089394 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:59.089377 2582 scope.go:117] "RemoveContainer" containerID="9fe3fea55a13adf89a64166391dfa5ab4ec1c410634abe1561eceda982c37693" Jul 13 18:41:59.089696 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:41:59.089675 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fe3fea55a13adf89a64166391dfa5ab4ec1c410634abe1561eceda982c37693\": container with ID starting with 9fe3fea55a13adf89a64166391dfa5ab4ec1c410634abe1561eceda982c37693 not found: ID does not exist" containerID="9fe3fea55a13adf89a64166391dfa5ab4ec1c410634abe1561eceda982c37693" Jul 13 18:41:59.089764 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:59.089708 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fe3fea55a13adf89a64166391dfa5ab4ec1c410634abe1561eceda982c37693"} err="failed to get container status \"9fe3fea55a13adf89a64166391dfa5ab4ec1c410634abe1561eceda982c37693\": rpc error: code = NotFound desc = could not find container \"9fe3fea55a13adf89a64166391dfa5ab4ec1c410634abe1561eceda982c37693\": container with ID starting with 9fe3fea55a13adf89a64166391dfa5ab4ec1c410634abe1561eceda982c37693 not found: ID does not exist" Jul 13 18:41:59.105961 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:59.104084 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-77kdg"] Jul 13 18:41:59.110642 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:41:59.110617 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-77kdg"] Jul 13 18:42:00.281552 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:00.281518 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afb40f22-2b1e-411f-82cb-d072df19fed8" path="/var/lib/kubelet/pods/afb40f22-2b1e-411f-82cb-d072df19fed8/volumes" Jul 13 18:42:01.385694 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:01.385656 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:42:01.490590 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:01.490543 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:42:01.901971 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:01.901942 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-srgb8"] Jul 13 18:42:01.902353 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:01.902340 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afb40f22-2b1e-411f-82cb-d072df19fed8" containerName="limitador" Jul 13 18:42:01.902397 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:01.902355 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="afb40f22-2b1e-411f-82cb-d072df19fed8" containerName="limitador" Jul 13 18:42:01.902441 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:01.902432 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="afb40f22-2b1e-411f-82cb-d072df19fed8" containerName="limitador" Jul 13 18:42:01.906742 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:01.906725 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-srgb8" Jul 13 18:42:01.911903 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:01.911879 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 13 18:42:01.912027 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:01.911901 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 13 18:42:01.912973 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:01.912951 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-5hb2w\"" Jul 13 18:42:01.913118 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:01.912956 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 18:42:01.921109 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:01.921083 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-srgb8"] Jul 13 18:42:01.927927 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:01.927885 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:42:01.949359 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:01.949329 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/0474b7cd-3f14-4f74-bdad-60283629dc07-data\") pod \"postgres-69fb4594cf-srgb8\" (UID: \"0474b7cd-3f14-4f74-bdad-60283629dc07\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-srgb8" Jul 13 18:42:01.949524 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:01.949378 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sd8z\" (UniqueName: \"kubernetes.io/projected/0474b7cd-3f14-4f74-bdad-60283629dc07-kube-api-access-4sd8z\") pod \"postgres-69fb4594cf-srgb8\" (UID: \"0474b7cd-3f14-4f74-bdad-60283629dc07\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-srgb8" Jul 13 18:42:02.050932 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:02.050876 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/0474b7cd-3f14-4f74-bdad-60283629dc07-data\") pod \"postgres-69fb4594cf-srgb8\" (UID: \"0474b7cd-3f14-4f74-bdad-60283629dc07\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-srgb8" Jul 13 18:42:02.051089 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:02.050952 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sd8z\" (UniqueName: \"kubernetes.io/projected/0474b7cd-3f14-4f74-bdad-60283629dc07-kube-api-access-4sd8z\") pod \"postgres-69fb4594cf-srgb8\" (UID: \"0474b7cd-3f14-4f74-bdad-60283629dc07\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-srgb8" Jul 13 18:42:02.051289 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:02.051269 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/0474b7cd-3f14-4f74-bdad-60283629dc07-data\") pod \"postgres-69fb4594cf-srgb8\" (UID: \"0474b7cd-3f14-4f74-bdad-60283629dc07\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-srgb8" Jul 13 18:42:02.061904 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:02.061882 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sd8z\" (UniqueName: \"kubernetes.io/projected/0474b7cd-3f14-4f74-bdad-60283629dc07-kube-api-access-4sd8z\") pod \"postgres-69fb4594cf-srgb8\" (UID: \"0474b7cd-3f14-4f74-bdad-60283629dc07\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-srgb8" Jul 13 18:42:02.216981 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:02.216950 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-srgb8" Jul 13 18:42:02.354119 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:02.354093 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-srgb8"] Jul 13 18:42:02.356252 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:42:02.356227 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0474b7cd_3f14_4f74_bdad_60283629dc07.slice/crio-df8de56d7a5990f4aaa2dcfd39ed57d5b1391a737f662b06f581a3a7ecbf4ee3 WatchSource:0}: Error finding container df8de56d7a5990f4aaa2dcfd39ed57d5b1391a737f662b06f581a3a7ecbf4ee3: Status 404 returned error can't find the container with id df8de56d7a5990f4aaa2dcfd39ed57d5b1391a737f662b06f581a3a7ecbf4ee3 Jul 13 18:42:03.102214 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:03.102178 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-srgb8" event={"ID":"0474b7cd-3f14-4f74-bdad-60283629dc07","Type":"ContainerStarted","Data":"df8de56d7a5990f4aaa2dcfd39ed57d5b1391a737f662b06f581a3a7ecbf4ee3"} Jul 13 18:42:08.133578 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:08.133500 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-srgb8" event={"ID":"0474b7cd-3f14-4f74-bdad-60283629dc07","Type":"ContainerStarted","Data":"99cb3279d3386f6a7a118f2122b1bb50372848ed5653c389cba6fdf8ae7bddb9"} Jul 13 18:42:08.133578 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:08.133562 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-srgb8" Jul 13 18:42:08.154317 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:08.154256 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-srgb8" podStartSLOduration=1.688895806 podStartE2EDuration="7.154233349s" totalImagesPullingTime="5.465337543s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:42:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:42:08.151862998 +0000 UTC m=+584.435291602" watchObservedRunningTime="2026-07-13 18:42:08.154233349 +0000 UTC m=+584.437661952" Jul 13 18:42:14.167716 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:14.167688 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-srgb8" Jul 13 18:42:14.510364 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:14.510335 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:42:14.761479 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:14.761404 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-psxhf"] Jul 13 18:42:14.767909 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:14.767893 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-psxhf" Jul 13 18:42:14.773051 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:14.773027 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-psxhf"] Jul 13 18:42:14.872020 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:14.871995 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8m7s4\" (UniqueName: \"kubernetes.io/projected/5ba5cdee-af13-45d4-97f0-dab425529041-kube-api-access-8m7s4\") pod \"authorino-5d4b67c446-psxhf\" (UID: \"5ba5cdee-af13-45d4-97f0-dab425529041\") " pod="kuadrant-system/authorino-5d4b67c446-psxhf" Jul 13 18:42:14.973177 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:14.973143 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8m7s4\" (UniqueName: \"kubernetes.io/projected/5ba5cdee-af13-45d4-97f0-dab425529041-kube-api-access-8m7s4\") pod \"authorino-5d4b67c446-psxhf\" (UID: \"5ba5cdee-af13-45d4-97f0-dab425529041\") " pod="kuadrant-system/authorino-5d4b67c446-psxhf" Jul 13 18:42:14.981541 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:14.981517 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8m7s4\" (UniqueName: \"kubernetes.io/projected/5ba5cdee-af13-45d4-97f0-dab425529041-kube-api-access-8m7s4\") pod \"authorino-5d4b67c446-psxhf\" (UID: \"5ba5cdee-af13-45d4-97f0-dab425529041\") " pod="kuadrant-system/authorino-5d4b67c446-psxhf" Jul 13 18:42:15.019075 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:15.019011 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-psxhf"] Jul 13 18:42:15.019284 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:15.019272 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-psxhf" Jul 13 18:42:15.046314 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:15.046289 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-cd6bd8b54-c8v5g"] Jul 13 18:42:15.051689 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:15.051670 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cd6bd8b54-c8v5g" Jul 13 18:42:15.054350 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:15.054326 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-cd6bd8b54-c8v5g"] Jul 13 18:42:15.158987 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:15.158964 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-psxhf"] Jul 13 18:42:15.161628 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:42:15.161581 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5ba5cdee_af13_45d4_97f0_dab425529041.slice/crio-f8c8ab395d1cebb4172a16d749f8e3aceb38d72243e39df975b59b17c2dd2eec WatchSource:0}: Error finding container f8c8ab395d1cebb4172a16d749f8e3aceb38d72243e39df975b59b17c2dd2eec: Status 404 returned error can't find the container with id f8c8ab395d1cebb4172a16d749f8e3aceb38d72243e39df975b59b17c2dd2eec Jul 13 18:42:15.175069 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:15.175046 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlpwx\" (UniqueName: \"kubernetes.io/projected/1b010be2-87a5-46ab-9b8b-98135c7c83fe-kube-api-access-tlpwx\") pod \"authorino-cd6bd8b54-c8v5g\" (UID: \"1b010be2-87a5-46ab-9b8b-98135c7c83fe\") " pod="kuadrant-system/authorino-cd6bd8b54-c8v5g" Jul 13 18:42:15.275726 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:15.275643 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlpwx\" (UniqueName: \"kubernetes.io/projected/1b010be2-87a5-46ab-9b8b-98135c7c83fe-kube-api-access-tlpwx\") pod \"authorino-cd6bd8b54-c8v5g\" (UID: \"1b010be2-87a5-46ab-9b8b-98135c7c83fe\") " pod="kuadrant-system/authorino-cd6bd8b54-c8v5g" Jul 13 18:42:15.285768 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:15.285744 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlpwx\" (UniqueName: \"kubernetes.io/projected/1b010be2-87a5-46ab-9b8b-98135c7c83fe-kube-api-access-tlpwx\") pod \"authorino-cd6bd8b54-c8v5g\" (UID: \"1b010be2-87a5-46ab-9b8b-98135c7c83fe\") " pod="kuadrant-system/authorino-cd6bd8b54-c8v5g" Jul 13 18:42:15.286619 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:15.286576 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:42:15.362731 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:15.362679 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cd6bd8b54-c8v5g" Jul 13 18:42:15.495220 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:15.495196 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-cd6bd8b54-c8v5g"] Jul 13 18:42:15.497675 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:42:15.497647 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1b010be2_87a5_46ab_9b8b_98135c7c83fe.slice/crio-36fce77221e4835fcc888d4bdede1d79149dde2b2ddf0f3d0872824594257afe WatchSource:0}: Error finding container 36fce77221e4835fcc888d4bdede1d79149dde2b2ddf0f3d0872824594257afe: Status 404 returned error can't find the container with id 36fce77221e4835fcc888d4bdede1d79149dde2b2ddf0f3d0872824594257afe Jul 13 18:42:16.173547 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:16.173515 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-psxhf" event={"ID":"5ba5cdee-af13-45d4-97f0-dab425529041","Type":"ContainerStarted","Data":"77c4d620043d4872ff834d12ac50c255c18b778f9f466f008ea92ea263961a8e"} Jul 13 18:42:16.173775 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:16.173557 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-psxhf" event={"ID":"5ba5cdee-af13-45d4-97f0-dab425529041","Type":"ContainerStarted","Data":"f8c8ab395d1cebb4172a16d749f8e3aceb38d72243e39df975b59b17c2dd2eec"} Jul 13 18:42:16.173775 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:16.173557 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-psxhf" podUID="5ba5cdee-af13-45d4-97f0-dab425529041" containerName="authorino" containerID="cri-o://77c4d620043d4872ff834d12ac50c255c18b778f9f466f008ea92ea263961a8e" gracePeriod=30 Jul 13 18:42:16.174882 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:16.174853 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-cd6bd8b54-c8v5g" event={"ID":"1b010be2-87a5-46ab-9b8b-98135c7c83fe","Type":"ContainerStarted","Data":"36fce77221e4835fcc888d4bdede1d79149dde2b2ddf0f3d0872824594257afe"} Jul 13 18:42:16.190166 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:16.190126 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-psxhf" podStartSLOduration=1.334299927 podStartE2EDuration="2.190112252s" totalImagesPullingTime="855.812325ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:42:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:42:16.188667294 +0000 UTC m=+592.472095897" watchObservedRunningTime="2026-07-13 18:42:16.190112252 +0000 UTC m=+592.473540854" Jul 13 18:42:16.418175 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:16.418154 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-psxhf" Jul 13 18:42:16.487706 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:16.487678 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8m7s4\" (UniqueName: \"kubernetes.io/projected/5ba5cdee-af13-45d4-97f0-dab425529041-kube-api-access-8m7s4\") pod \"5ba5cdee-af13-45d4-97f0-dab425529041\" (UID: \"5ba5cdee-af13-45d4-97f0-dab425529041\") " Jul 13 18:42:16.489847 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:16.489826 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ba5cdee-af13-45d4-97f0-dab425529041-kube-api-access-8m7s4" (OuterVolumeSpecName: "kube-api-access-8m7s4") pod "5ba5cdee-af13-45d4-97f0-dab425529041" (UID: "5ba5cdee-af13-45d4-97f0-dab425529041"). InnerVolumeSpecName "kube-api-access-8m7s4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 18:42:16.589355 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:16.589277 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8m7s4\" (UniqueName: \"kubernetes.io/projected/5ba5cdee-af13-45d4-97f0-dab425529041-kube-api-access-8m7s4\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:42:17.181688 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:17.181652 2582 generic.go:356] "Generic (PLEG): container finished" podID="5ba5cdee-af13-45d4-97f0-dab425529041" containerID="77c4d620043d4872ff834d12ac50c255c18b778f9f466f008ea92ea263961a8e" exitCode=0 Jul 13 18:42:17.181889 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:17.181693 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-psxhf" Jul 13 18:42:17.181889 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:17.181742 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-psxhf" event={"ID":"5ba5cdee-af13-45d4-97f0-dab425529041","Type":"ContainerDied","Data":"77c4d620043d4872ff834d12ac50c255c18b778f9f466f008ea92ea263961a8e"} Jul 13 18:42:17.181889 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:17.181784 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-psxhf" event={"ID":"5ba5cdee-af13-45d4-97f0-dab425529041","Type":"ContainerDied","Data":"f8c8ab395d1cebb4172a16d749f8e3aceb38d72243e39df975b59b17c2dd2eec"} Jul 13 18:42:17.181889 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:17.181806 2582 scope.go:117] "RemoveContainer" containerID="77c4d620043d4872ff834d12ac50c255c18b778f9f466f008ea92ea263961a8e" Jul 13 18:42:17.190779 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:17.190761 2582 scope.go:117] "RemoveContainer" containerID="77c4d620043d4872ff834d12ac50c255c18b778f9f466f008ea92ea263961a8e" Jul 13 18:42:17.191054 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:42:17.191013 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77c4d620043d4872ff834d12ac50c255c18b778f9f466f008ea92ea263961a8e\": container with ID starting with 77c4d620043d4872ff834d12ac50c255c18b778f9f466f008ea92ea263961a8e not found: ID does not exist" containerID="77c4d620043d4872ff834d12ac50c255c18b778f9f466f008ea92ea263961a8e" Jul 13 18:42:17.191054 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:17.191033 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77c4d620043d4872ff834d12ac50c255c18b778f9f466f008ea92ea263961a8e"} err="failed to get container status \"77c4d620043d4872ff834d12ac50c255c18b778f9f466f008ea92ea263961a8e\": rpc error: code = NotFound desc = could not find container \"77c4d620043d4872ff834d12ac50c255c18b778f9f466f008ea92ea263961a8e\": container with ID starting with 77c4d620043d4872ff834d12ac50c255c18b778f9f466f008ea92ea263961a8e not found: ID does not exist" Jul 13 18:42:17.203905 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:17.203880 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-psxhf"] Jul 13 18:42:17.209221 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:17.209197 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-psxhf"] Jul 13 18:42:18.280549 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:18.280518 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ba5cdee-af13-45d4-97f0-dab425529041" path="/var/lib/kubelet/pods/5ba5cdee-af13-45d4-97f0-dab425529041/volumes" Jul 13 18:42:20.199893 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:20.199854 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-cd6bd8b54-c8v5g" event={"ID":"1b010be2-87a5-46ab-9b8b-98135c7c83fe","Type":"ContainerStarted","Data":"13c1281942e9acce5194a7860bf77a24c201541619dddbb56e5daba597702afb"} Jul 13 18:42:20.216152 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:20.216108 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-cd6bd8b54-c8v5g" podStartSLOduration=1.324470161 podStartE2EDuration="5.216093172s" totalImagesPullingTime="3.891623011s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:42:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:42:20.214348515 +0000 UTC m=+596.497777117" watchObservedRunningTime="2026-07-13 18:42:20.216093172 +0000 UTC m=+596.499521774" Jul 13 18:42:20.251123 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:20.251038 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-w94ml"] Jul 13 18:42:20.251317 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:20.251280 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-w94ml" podUID="710e57b0-4774-4494-b2f7-2af1aa5a361c" containerName="authorino" containerID="cri-o://93d307ee182263495c1db186034b58c9271d367db81866e5f19b27db3ca250ea" gracePeriod=30 Jul 13 18:42:20.410385 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:20.410345 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-65b6464f94-8lclh"] Jul 13 18:42:20.410764 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:20.410747 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ba5cdee-af13-45d4-97f0-dab425529041" containerName="authorino" Jul 13 18:42:20.410764 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:20.410765 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ba5cdee-af13-45d4-97f0-dab425529041" containerName="authorino" Jul 13 18:42:20.410907 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:20.410839 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ba5cdee-af13-45d4-97f0-dab425529041" containerName="authorino" Jul 13 18:42:20.428924 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:20.428897 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-65b6464f94-8lclh"] Jul 13 18:42:20.429075 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:20.428992 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65b6464f94-8lclh" Jul 13 18:42:20.432414 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:20.432385 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 13 18:42:20.522242 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:20.522218 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5ac368a0-78bb-44be-89ec-6a1ce1e18b41-tls-cert\") pod \"authorino-65b6464f94-8lclh\" (UID: \"5ac368a0-78bb-44be-89ec-6a1ce1e18b41\") " pod="kuadrant-system/authorino-65b6464f94-8lclh" Jul 13 18:42:20.522381 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:20.522260 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlggs\" (UniqueName: \"kubernetes.io/projected/5ac368a0-78bb-44be-89ec-6a1ce1e18b41-kube-api-access-hlggs\") pod \"authorino-65b6464f94-8lclh\" (UID: \"5ac368a0-78bb-44be-89ec-6a1ce1e18b41\") " pod="kuadrant-system/authorino-65b6464f94-8lclh" Jul 13 18:42:20.539482 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:20.539458 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-w94ml" Jul 13 18:42:20.623385 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:20.623343 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8wdk4\" (UniqueName: \"kubernetes.io/projected/710e57b0-4774-4494-b2f7-2af1aa5a361c-kube-api-access-8wdk4\") pod \"710e57b0-4774-4494-b2f7-2af1aa5a361c\" (UID: \"710e57b0-4774-4494-b2f7-2af1aa5a361c\") " Jul 13 18:42:20.623555 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:20.623526 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5ac368a0-78bb-44be-89ec-6a1ce1e18b41-tls-cert\") pod \"authorino-65b6464f94-8lclh\" (UID: \"5ac368a0-78bb-44be-89ec-6a1ce1e18b41\") " pod="kuadrant-system/authorino-65b6464f94-8lclh" Jul 13 18:42:20.623627 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:20.623581 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlggs\" (UniqueName: \"kubernetes.io/projected/5ac368a0-78bb-44be-89ec-6a1ce1e18b41-kube-api-access-hlggs\") pod \"authorino-65b6464f94-8lclh\" (UID: \"5ac368a0-78bb-44be-89ec-6a1ce1e18b41\") " pod="kuadrant-system/authorino-65b6464f94-8lclh" Jul 13 18:42:20.625596 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:20.625563 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/710e57b0-4774-4494-b2f7-2af1aa5a361c-kube-api-access-8wdk4" (OuterVolumeSpecName: "kube-api-access-8wdk4") pod "710e57b0-4774-4494-b2f7-2af1aa5a361c" (UID: "710e57b0-4774-4494-b2f7-2af1aa5a361c"). InnerVolumeSpecName "kube-api-access-8wdk4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 18:42:20.626094 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:20.626073 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5ac368a0-78bb-44be-89ec-6a1ce1e18b41-tls-cert\") pod \"authorino-65b6464f94-8lclh\" (UID: \"5ac368a0-78bb-44be-89ec-6a1ce1e18b41\") " pod="kuadrant-system/authorino-65b6464f94-8lclh" Jul 13 18:42:20.633626 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:20.633583 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlggs\" (UniqueName: \"kubernetes.io/projected/5ac368a0-78bb-44be-89ec-6a1ce1e18b41-kube-api-access-hlggs\") pod \"authorino-65b6464f94-8lclh\" (UID: \"5ac368a0-78bb-44be-89ec-6a1ce1e18b41\") " pod="kuadrant-system/authorino-65b6464f94-8lclh" Jul 13 18:42:20.724810 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:20.724779 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wdk4\" (UniqueName: \"kubernetes.io/projected/710e57b0-4774-4494-b2f7-2af1aa5a361c-kube-api-access-8wdk4\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:42:20.740203 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:20.740179 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65b6464f94-8lclh" Jul 13 18:42:20.911407 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:20.911385 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-65b6464f94-8lclh"] Jul 13 18:42:20.913249 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:42:20.913226 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5ac368a0_78bb_44be_89ec_6a1ce1e18b41.slice/crio-61d9a62679ed5c75077966f14bcc7351b2b5a702bb65ba6a01876166cecebd4e WatchSource:0}: Error finding container 61d9a62679ed5c75077966f14bcc7351b2b5a702bb65ba6a01876166cecebd4e: Status 404 returned error can't find the container with id 61d9a62679ed5c75077966f14bcc7351b2b5a702bb65ba6a01876166cecebd4e Jul 13 18:42:21.205811 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:21.205780 2582 generic.go:356] "Generic (PLEG): container finished" podID="710e57b0-4774-4494-b2f7-2af1aa5a361c" containerID="93d307ee182263495c1db186034b58c9271d367db81866e5f19b27db3ca250ea" exitCode=0 Jul 13 18:42:21.206285 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:21.205825 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-w94ml" Jul 13 18:42:21.206285 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:21.205860 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-w94ml" event={"ID":"710e57b0-4774-4494-b2f7-2af1aa5a361c","Type":"ContainerDied","Data":"93d307ee182263495c1db186034b58c9271d367db81866e5f19b27db3ca250ea"} Jul 13 18:42:21.206285 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:21.205908 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-w94ml" event={"ID":"710e57b0-4774-4494-b2f7-2af1aa5a361c","Type":"ContainerDied","Data":"473598efc52c6b69b2f751ca439d763d4a73fad7c55aa5ebcaebff240d742836"} Jul 13 18:42:21.206285 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:21.205937 2582 scope.go:117] "RemoveContainer" containerID="93d307ee182263495c1db186034b58c9271d367db81866e5f19b27db3ca250ea" Jul 13 18:42:21.207219 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:21.207187 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65b6464f94-8lclh" event={"ID":"5ac368a0-78bb-44be-89ec-6a1ce1e18b41","Type":"ContainerStarted","Data":"61d9a62679ed5c75077966f14bcc7351b2b5a702bb65ba6a01876166cecebd4e"} Jul 13 18:42:21.214830 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:21.214799 2582 scope.go:117] "RemoveContainer" containerID="93d307ee182263495c1db186034b58c9271d367db81866e5f19b27db3ca250ea" Jul 13 18:42:21.215065 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:42:21.215049 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93d307ee182263495c1db186034b58c9271d367db81866e5f19b27db3ca250ea\": container with ID starting with 93d307ee182263495c1db186034b58c9271d367db81866e5f19b27db3ca250ea not found: ID does not exist" containerID="93d307ee182263495c1db186034b58c9271d367db81866e5f19b27db3ca250ea" Jul 13 18:42:21.215111 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:21.215075 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93d307ee182263495c1db186034b58c9271d367db81866e5f19b27db3ca250ea"} err="failed to get container status \"93d307ee182263495c1db186034b58c9271d367db81866e5f19b27db3ca250ea\": rpc error: code = NotFound desc = could not find container \"93d307ee182263495c1db186034b58c9271d367db81866e5f19b27db3ca250ea\": container with ID starting with 93d307ee182263495c1db186034b58c9271d367db81866e5f19b27db3ca250ea not found: ID does not exist" Jul 13 18:42:21.234617 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:21.234579 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-w94ml"] Jul 13 18:42:21.238134 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:21.238116 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-w94ml"] Jul 13 18:42:22.213968 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:22.213933 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65b6464f94-8lclh" event={"ID":"5ac368a0-78bb-44be-89ec-6a1ce1e18b41","Type":"ContainerStarted","Data":"42f182f0b92730d1edb02a389aceb4f68b13bd7fa20dcf5643a6719e202e6790"} Jul 13 18:42:22.233266 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:22.233210 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-65b6464f94-8lclh" podStartSLOduration=1.404931296 podStartE2EDuration="2.233195975s" totalImagesPullingTime="828.264679ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:42:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:42:22.231581825 +0000 UTC m=+598.515010428" watchObservedRunningTime="2026-07-13 18:42:22.233195975 +0000 UTC m=+598.516624576" Jul 13 18:42:22.262066 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:22.262033 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-cd6bd8b54-c8v5g"] Jul 13 18:42:22.262225 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:22.262201 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-cd6bd8b54-c8v5g" podUID="1b010be2-87a5-46ab-9b8b-98135c7c83fe" containerName="authorino" containerID="cri-o://13c1281942e9acce5194a7860bf77a24c201541619dddbb56e5daba597702afb" gracePeriod=30 Jul 13 18:42:22.282237 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:22.282207 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="710e57b0-4774-4494-b2f7-2af1aa5a361c" path="/var/lib/kubelet/pods/710e57b0-4774-4494-b2f7-2af1aa5a361c/volumes" Jul 13 18:42:22.555081 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:22.555051 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cd6bd8b54-c8v5g" Jul 13 18:42:22.642418 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:22.642383 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlpwx\" (UniqueName: \"kubernetes.io/projected/1b010be2-87a5-46ab-9b8b-98135c7c83fe-kube-api-access-tlpwx\") pod \"1b010be2-87a5-46ab-9b8b-98135c7c83fe\" (UID: \"1b010be2-87a5-46ab-9b8b-98135c7c83fe\") " Jul 13 18:42:22.644571 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:22.644544 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b010be2-87a5-46ab-9b8b-98135c7c83fe-kube-api-access-tlpwx" (OuterVolumeSpecName: "kube-api-access-tlpwx") pod "1b010be2-87a5-46ab-9b8b-98135c7c83fe" (UID: "1b010be2-87a5-46ab-9b8b-98135c7c83fe"). InnerVolumeSpecName "kube-api-access-tlpwx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 18:42:22.743416 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:22.743387 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlpwx\" (UniqueName: \"kubernetes.io/projected/1b010be2-87a5-46ab-9b8b-98135c7c83fe-kube-api-access-tlpwx\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:42:23.220202 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:23.220167 2582 generic.go:356] "Generic (PLEG): container finished" podID="1b010be2-87a5-46ab-9b8b-98135c7c83fe" containerID="13c1281942e9acce5194a7860bf77a24c201541619dddbb56e5daba597702afb" exitCode=0 Jul 13 18:42:23.220598 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:23.220213 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cd6bd8b54-c8v5g" Jul 13 18:42:23.220598 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:23.220257 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-cd6bd8b54-c8v5g" event={"ID":"1b010be2-87a5-46ab-9b8b-98135c7c83fe","Type":"ContainerDied","Data":"13c1281942e9acce5194a7860bf77a24c201541619dddbb56e5daba597702afb"} Jul 13 18:42:23.220598 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:23.220296 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-cd6bd8b54-c8v5g" event={"ID":"1b010be2-87a5-46ab-9b8b-98135c7c83fe","Type":"ContainerDied","Data":"36fce77221e4835fcc888d4bdede1d79149dde2b2ddf0f3d0872824594257afe"} Jul 13 18:42:23.220598 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:23.220313 2582 scope.go:117] "RemoveContainer" containerID="13c1281942e9acce5194a7860bf77a24c201541619dddbb56e5daba597702afb" Jul 13 18:42:23.228885 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:23.228866 2582 scope.go:117] "RemoveContainer" containerID="13c1281942e9acce5194a7860bf77a24c201541619dddbb56e5daba597702afb" Jul 13 18:42:23.229130 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:42:23.229113 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13c1281942e9acce5194a7860bf77a24c201541619dddbb56e5daba597702afb\": container with ID starting with 13c1281942e9acce5194a7860bf77a24c201541619dddbb56e5daba597702afb not found: ID does not exist" containerID="13c1281942e9acce5194a7860bf77a24c201541619dddbb56e5daba597702afb" Jul 13 18:42:23.229188 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:23.229137 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13c1281942e9acce5194a7860bf77a24c201541619dddbb56e5daba597702afb"} err="failed to get container status \"13c1281942e9acce5194a7860bf77a24c201541619dddbb56e5daba597702afb\": rpc error: code = NotFound desc = could not find container \"13c1281942e9acce5194a7860bf77a24c201541619dddbb56e5daba597702afb\": container with ID starting with 13c1281942e9acce5194a7860bf77a24c201541619dddbb56e5daba597702afb not found: ID does not exist" Jul 13 18:42:23.250587 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:23.250560 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-cd6bd8b54-c8v5g"] Jul 13 18:42:23.252125 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:23.252108 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-cd6bd8b54-c8v5g"] Jul 13 18:42:24.141125 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:24.141102 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8l7w6_b14ef695-c589-4fca-bd12-2e951f04e466/ovn-acl-logging/0.log" Jul 13 18:42:24.141961 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:24.141942 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8l7w6_b14ef695-c589-4fca-bd12-2e951f04e466/ovn-acl-logging/0.log" Jul 13 18:42:24.281057 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:24.281030 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b010be2-87a5-46ab-9b8b-98135c7c83fe" path="/var/lib/kubelet/pods/1b010be2-87a5-46ab-9b8b-98135c7c83fe/volumes" Jul 13 18:42:25.132883 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:25.132847 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:42:25.227087 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:25.227042 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:42:25.413698 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:25.413590 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:42:29.011598 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:29.011558 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:42:29.111575 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:29.111540 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:42:48.886404 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:48.886325 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:42:50.486716 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:50.486683 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:42:59.810146 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:42:59.810106 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:43:00.196452 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:00.196418 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:43:00.294184 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:00.294150 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:43:00.756773 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:00.756740 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5648755864-phxl4"] Jul 13 18:43:00.757124 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:00.757106 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="710e57b0-4774-4494-b2f7-2af1aa5a361c" containerName="authorino" Jul 13 18:43:00.757124 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:00.757118 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="710e57b0-4774-4494-b2f7-2af1aa5a361c" containerName="authorino" Jul 13 18:43:00.757210 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:00.757138 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b010be2-87a5-46ab-9b8b-98135c7c83fe" containerName="authorino" Jul 13 18:43:00.757210 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:00.757143 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b010be2-87a5-46ab-9b8b-98135c7c83fe" containerName="authorino" Jul 13 18:43:00.757210 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:00.757201 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b010be2-87a5-46ab-9b8b-98135c7c83fe" containerName="authorino" Jul 13 18:43:00.757210 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:00.757210 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="710e57b0-4774-4494-b2f7-2af1aa5a361c" containerName="authorino" Jul 13 18:43:00.760470 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:00.760451 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5648755864-phxl4" Jul 13 18:43:00.763072 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:00.763052 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 13 18:43:00.772914 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:00.772895 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5648755864-phxl4"] Jul 13 18:43:00.880226 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:00.880199 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zspmx\" (UniqueName: \"kubernetes.io/projected/36417751-3059-46a2-9b61-d8e1583fd7f8-kube-api-access-zspmx\") pod \"maas-api-5648755864-phxl4\" (UID: \"36417751-3059-46a2-9b61-d8e1583fd7f8\") " pod="odh-ai-gateway-infra/maas-api-5648755864-phxl4" Jul 13 18:43:00.880564 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:00.880254 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36417751-3059-46a2-9b61-d8e1583fd7f8-openshift-service-ca\") pod \"maas-api-5648755864-phxl4\" (UID: \"36417751-3059-46a2-9b61-d8e1583fd7f8\") " pod="odh-ai-gateway-infra/maas-api-5648755864-phxl4" Jul 13 18:43:00.880564 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:00.880315 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36417751-3059-46a2-9b61-d8e1583fd7f8-maas-api-tls\") pod \"maas-api-5648755864-phxl4\" (UID: \"36417751-3059-46a2-9b61-d8e1583fd7f8\") " pod="odh-ai-gateway-infra/maas-api-5648755864-phxl4" Jul 13 18:43:00.981410 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:00.981375 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zspmx\" (UniqueName: \"kubernetes.io/projected/36417751-3059-46a2-9b61-d8e1583fd7f8-kube-api-access-zspmx\") pod \"maas-api-5648755864-phxl4\" (UID: \"36417751-3059-46a2-9b61-d8e1583fd7f8\") " pod="odh-ai-gateway-infra/maas-api-5648755864-phxl4" Jul 13 18:43:00.981577 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:00.981452 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36417751-3059-46a2-9b61-d8e1583fd7f8-openshift-service-ca\") pod \"maas-api-5648755864-phxl4\" (UID: \"36417751-3059-46a2-9b61-d8e1583fd7f8\") " pod="odh-ai-gateway-infra/maas-api-5648755864-phxl4" Jul 13 18:43:00.981577 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:00.981491 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36417751-3059-46a2-9b61-d8e1583fd7f8-maas-api-tls\") pod \"maas-api-5648755864-phxl4\" (UID: \"36417751-3059-46a2-9b61-d8e1583fd7f8\") " pod="odh-ai-gateway-infra/maas-api-5648755864-phxl4" Jul 13 18:43:00.982165 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:00.982138 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36417751-3059-46a2-9b61-d8e1583fd7f8-openshift-service-ca\") pod \"maas-api-5648755864-phxl4\" (UID: \"36417751-3059-46a2-9b61-d8e1583fd7f8\") " pod="odh-ai-gateway-infra/maas-api-5648755864-phxl4" Jul 13 18:43:00.984074 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:00.984057 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36417751-3059-46a2-9b61-d8e1583fd7f8-maas-api-tls\") pod \"maas-api-5648755864-phxl4\" (UID: \"36417751-3059-46a2-9b61-d8e1583fd7f8\") " pod="odh-ai-gateway-infra/maas-api-5648755864-phxl4" Jul 13 18:43:00.989866 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:00.989850 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zspmx\" (UniqueName: \"kubernetes.io/projected/36417751-3059-46a2-9b61-d8e1583fd7f8-kube-api-access-zspmx\") pod \"maas-api-5648755864-phxl4\" (UID: \"36417751-3059-46a2-9b61-d8e1583fd7f8\") " pod="odh-ai-gateway-infra/maas-api-5648755864-phxl4" Jul 13 18:43:01.004847 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:01.004827 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:43:01.071282 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:01.071202 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5648755864-phxl4" Jul 13 18:43:01.105319 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:01.105273 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:43:01.210583 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:01.210561 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5648755864-phxl4"] Jul 13 18:43:01.212290 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:43:01.212265 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36417751_3059_46a2_9b61_d8e1583fd7f8.slice/crio-34c712149c3f56733d55f671478e4d701c1a38ae8a7c5d1276ac601ee5124eff WatchSource:0}: Error finding container 34c712149c3f56733d55f671478e4d701c1a38ae8a7c5d1276ac601ee5124eff: Status 404 returned error can't find the container with id 34c712149c3f56733d55f671478e4d701c1a38ae8a7c5d1276ac601ee5124eff Jul 13 18:43:01.213834 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:01.213819 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 18:43:01.319116 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:01.319080 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:43:01.398017 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:01.397931 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5648755864-phxl4" event={"ID":"36417751-3059-46a2-9b61-d8e1583fd7f8","Type":"ContainerStarted","Data":"34c712149c3f56733d55f671478e4d701c1a38ae8a7c5d1276ac601ee5124eff"} Jul 13 18:43:01.420544 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:01.420516 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:43:03.410421 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:03.410381 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5648755864-phxl4" event={"ID":"36417751-3059-46a2-9b61-d8e1583fd7f8","Type":"ContainerStarted","Data":"f6e6f9a9616a94261e89709ab18a4a44e8e8de1038c64f478e770cad5f0c360c"} Jul 13 18:43:03.410797 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:03.410443 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5648755864-phxl4" Jul 13 18:43:03.433483 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:03.433436 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5648755864-phxl4" podStartSLOduration=1.782687588 podStartE2EDuration="3.433419346s" totalImagesPullingTime="1.650731758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:43:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:43:03.431799828 +0000 UTC m=+639.715228451" watchObservedRunningTime="2026-07-13 18:43:03.433419346 +0000 UTC m=+639.716847947" Jul 13 18:43:09.420620 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:09.420573 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5648755864-phxl4" Jul 13 18:43:11.896025 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:11.895995 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:43:12.010515 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:12.010479 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:43:17.484055 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:17.484019 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:43:25.955494 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:25.955458 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vt48k"] Jul 13 18:43:25.960493 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:25.960469 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vt48k" Jul 13 18:43:25.963319 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:25.963298 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 13 18:43:25.963319 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:25.963305 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 13 18:43:25.963508 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:25.963298 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-z76zx\"" Jul 13 18:43:25.964278 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:25.964257 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 13 18:43:25.970542 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:25.970522 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vt48k"] Jul 13 18:43:25.996797 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:25.996750 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:43:26.109296 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:26.109253 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5845\" (UniqueName: \"kubernetes.io/projected/48b1a3bd-df4a-4886-b7ab-3beae9539f93-kube-api-access-l5845\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vt48k\" (UID: \"48b1a3bd-df4a-4886-b7ab-3beae9539f93\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vt48k" Jul 13 18:43:26.109500 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:26.109350 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48b1a3bd-df4a-4886-b7ab-3beae9539f93-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vt48k\" (UID: \"48b1a3bd-df4a-4886-b7ab-3beae9539f93\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vt48k" Jul 13 18:43:26.109500 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:26.109394 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48b1a3bd-df4a-4886-b7ab-3beae9539f93-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vt48k\" (UID: \"48b1a3bd-df4a-4886-b7ab-3beae9539f93\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vt48k" Jul 13 18:43:26.109500 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:26.109416 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48b1a3bd-df4a-4886-b7ab-3beae9539f93-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vt48k\" (UID: \"48b1a3bd-df4a-4886-b7ab-3beae9539f93\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vt48k" Jul 13 18:43:26.109500 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:26.109467 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48b1a3bd-df4a-4886-b7ab-3beae9539f93-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vt48k\" (UID: \"48b1a3bd-df4a-4886-b7ab-3beae9539f93\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vt48k" Jul 13 18:43:26.109809 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:26.109567 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48b1a3bd-df4a-4886-b7ab-3beae9539f93-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vt48k\" (UID: \"48b1a3bd-df4a-4886-b7ab-3beae9539f93\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vt48k" Jul 13 18:43:26.211200 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:26.211109 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48b1a3bd-df4a-4886-b7ab-3beae9539f93-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vt48k\" (UID: \"48b1a3bd-df4a-4886-b7ab-3beae9539f93\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vt48k" Jul 13 18:43:26.211200 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:26.211175 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48b1a3bd-df4a-4886-b7ab-3beae9539f93-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vt48k\" (UID: \"48b1a3bd-df4a-4886-b7ab-3beae9539f93\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vt48k" Jul 13 18:43:26.211459 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:26.211206 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48b1a3bd-df4a-4886-b7ab-3beae9539f93-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vt48k\" (UID: \"48b1a3bd-df4a-4886-b7ab-3beae9539f93\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vt48k" Jul 13 18:43:26.211459 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:26.211298 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48b1a3bd-df4a-4886-b7ab-3beae9539f93-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vt48k\" (UID: \"48b1a3bd-df4a-4886-b7ab-3beae9539f93\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vt48k" Jul 13 18:43:26.211459 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:26.211325 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5845\" (UniqueName: \"kubernetes.io/projected/48b1a3bd-df4a-4886-b7ab-3beae9539f93-kube-api-access-l5845\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vt48k\" (UID: \"48b1a3bd-df4a-4886-b7ab-3beae9539f93\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vt48k" Jul 13 18:43:26.211459 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:26.211340 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48b1a3bd-df4a-4886-b7ab-3beae9539f93-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vt48k\" (UID: \"48b1a3bd-df4a-4886-b7ab-3beae9539f93\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vt48k" Jul 13 18:43:26.211459 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:26.211377 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48b1a3bd-df4a-4886-b7ab-3beae9539f93-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vt48k\" (UID: \"48b1a3bd-df4a-4886-b7ab-3beae9539f93\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vt48k" Jul 13 18:43:26.211668 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:26.211621 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48b1a3bd-df4a-4886-b7ab-3beae9539f93-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vt48k\" (UID: \"48b1a3bd-df4a-4886-b7ab-3beae9539f93\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vt48k" Jul 13 18:43:26.211668 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:26.211657 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48b1a3bd-df4a-4886-b7ab-3beae9539f93-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vt48k\" (UID: \"48b1a3bd-df4a-4886-b7ab-3beae9539f93\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vt48k" Jul 13 18:43:26.213546 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:26.213525 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48b1a3bd-df4a-4886-b7ab-3beae9539f93-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vt48k\" (UID: \"48b1a3bd-df4a-4886-b7ab-3beae9539f93\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vt48k" Jul 13 18:43:26.213720 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:26.213703 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48b1a3bd-df4a-4886-b7ab-3beae9539f93-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vt48k\" (UID: \"48b1a3bd-df4a-4886-b7ab-3beae9539f93\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vt48k" Jul 13 18:43:26.220275 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:26.220251 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5845\" (UniqueName: \"kubernetes.io/projected/48b1a3bd-df4a-4886-b7ab-3beae9539f93-kube-api-access-l5845\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vt48k\" (UID: \"48b1a3bd-df4a-4886-b7ab-3beae9539f93\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vt48k" Jul 13 18:43:26.273386 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:26.273337 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vt48k" Jul 13 18:43:26.415379 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:26.415356 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vt48k"] Jul 13 18:43:26.417779 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:43:26.417749 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48b1a3bd_df4a_4886_b7ab_3beae9539f93.slice/crio-17d8b782aaba8a0f7cf30d72d5ede4ac06b4dae7414bb4de6fc18639b22cd37d WatchSource:0}: Error finding container 17d8b782aaba8a0f7cf30d72d5ede4ac06b4dae7414bb4de6fc18639b22cd37d: Status 404 returned error can't find the container with id 17d8b782aaba8a0f7cf30d72d5ede4ac06b4dae7414bb4de6fc18639b22cd37d Jul 13 18:43:26.534906 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:26.534822 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vt48k" event={"ID":"48b1a3bd-df4a-4886-b7ab-3beae9539f93","Type":"ContainerStarted","Data":"17d8b782aaba8a0f7cf30d72d5ede4ac06b4dae7414bb4de6fc18639b22cd37d"} Jul 13 18:43:28.545993 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:28.545944 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vt48k" event={"ID":"48b1a3bd-df4a-4886-b7ab-3beae9539f93","Type":"ContainerStarted","Data":"f6e7f3a2c27c064026136c0fb854513c8e97025225f5e5d600a97ceaadf4c1b8"} Jul 13 18:43:28.568925 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:28.568880 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vt48k" podStartSLOduration=2.469185056 podStartE2EDuration="3.568863837s" totalImagesPullingTime="1.099678781s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:43:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:43:28.566416956 +0000 UTC m=+664.849845560" watchObservedRunningTime="2026-07-13 18:43:28.568863837 +0000 UTC m=+664.852292438" Jul 13 18:43:30.259596 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:30.259566 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-jmzbd"] Jul 13 18:43:30.263212 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:30.263196 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jmzbd" Jul 13 18:43:30.265878 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:30.265858 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 13 18:43:30.273467 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:30.273442 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-jmzbd"] Jul 13 18:43:30.286430 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:30.286400 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:43:30.351298 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:30.351253 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92869120-8a9e-4054-92f3-e9baa9bd383b-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jmzbd\" (UID: \"92869120-8a9e-4054-92f3-e9baa9bd383b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jmzbd" Jul 13 18:43:30.351505 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:30.351337 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxmxj\" (UniqueName: \"kubernetes.io/projected/92869120-8a9e-4054-92f3-e9baa9bd383b-kube-api-access-xxmxj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jmzbd\" (UID: \"92869120-8a9e-4054-92f3-e9baa9bd383b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jmzbd" Jul 13 18:43:30.351646 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:30.351624 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92869120-8a9e-4054-92f3-e9baa9bd383b-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jmzbd\" (UID: \"92869120-8a9e-4054-92f3-e9baa9bd383b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jmzbd" Jul 13 18:43:30.351741 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:30.351661 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92869120-8a9e-4054-92f3-e9baa9bd383b-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jmzbd\" (UID: \"92869120-8a9e-4054-92f3-e9baa9bd383b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jmzbd" Jul 13 18:43:30.351741 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:30.351720 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92869120-8a9e-4054-92f3-e9baa9bd383b-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jmzbd\" (UID: \"92869120-8a9e-4054-92f3-e9baa9bd383b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jmzbd" Jul 13 18:43:30.351892 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:30.351769 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92869120-8a9e-4054-92f3-e9baa9bd383b-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jmzbd\" (UID: \"92869120-8a9e-4054-92f3-e9baa9bd383b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jmzbd" Jul 13 18:43:30.452807 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:30.452763 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92869120-8a9e-4054-92f3-e9baa9bd383b-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jmzbd\" (UID: \"92869120-8a9e-4054-92f3-e9baa9bd383b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jmzbd" Jul 13 18:43:30.453006 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:30.452846 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92869120-8a9e-4054-92f3-e9baa9bd383b-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jmzbd\" (UID: \"92869120-8a9e-4054-92f3-e9baa9bd383b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jmzbd" Jul 13 18:43:30.453006 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:30.452899 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxmxj\" (UniqueName: \"kubernetes.io/projected/92869120-8a9e-4054-92f3-e9baa9bd383b-kube-api-access-xxmxj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jmzbd\" (UID: \"92869120-8a9e-4054-92f3-e9baa9bd383b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jmzbd" Jul 13 18:43:30.453006 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:30.452968 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92869120-8a9e-4054-92f3-e9baa9bd383b-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jmzbd\" (UID: \"92869120-8a9e-4054-92f3-e9baa9bd383b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jmzbd" Jul 13 18:43:30.453006 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:30.452998 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92869120-8a9e-4054-92f3-e9baa9bd383b-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jmzbd\" (UID: \"92869120-8a9e-4054-92f3-e9baa9bd383b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jmzbd" Jul 13 18:43:30.453211 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:30.453024 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92869120-8a9e-4054-92f3-e9baa9bd383b-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jmzbd\" (UID: \"92869120-8a9e-4054-92f3-e9baa9bd383b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jmzbd" Jul 13 18:43:30.453367 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:30.453330 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92869120-8a9e-4054-92f3-e9baa9bd383b-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jmzbd\" (UID: \"92869120-8a9e-4054-92f3-e9baa9bd383b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jmzbd" Jul 13 18:43:30.453367 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:30.453350 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92869120-8a9e-4054-92f3-e9baa9bd383b-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jmzbd\" (UID: \"92869120-8a9e-4054-92f3-e9baa9bd383b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jmzbd" Jul 13 18:43:30.453534 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:30.453394 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92869120-8a9e-4054-92f3-e9baa9bd383b-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jmzbd\" (UID: \"92869120-8a9e-4054-92f3-e9baa9bd383b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jmzbd" Jul 13 18:43:30.455262 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:30.455242 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92869120-8a9e-4054-92f3-e9baa9bd383b-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jmzbd\" (UID: \"92869120-8a9e-4054-92f3-e9baa9bd383b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jmzbd" Jul 13 18:43:30.455397 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:30.455378 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92869120-8a9e-4054-92f3-e9baa9bd383b-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jmzbd\" (UID: \"92869120-8a9e-4054-92f3-e9baa9bd383b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jmzbd" Jul 13 18:43:30.462417 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:30.462395 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxmxj\" (UniqueName: \"kubernetes.io/projected/92869120-8a9e-4054-92f3-e9baa9bd383b-kube-api-access-xxmxj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jmzbd\" (UID: \"92869120-8a9e-4054-92f3-e9baa9bd383b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jmzbd" Jul 13 18:43:30.574124 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:30.574040 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jmzbd" Jul 13 18:43:30.712844 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:30.712723 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-jmzbd"] Jul 13 18:43:30.715801 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:43:30.715772 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92869120_8a9e_4054_92f3_e9baa9bd383b.slice/crio-be0c229cf412aead7fa1663a9bdf52fdac6a3e997dc43a04bc209c31ac06d5dc WatchSource:0}: Error finding container be0c229cf412aead7fa1663a9bdf52fdac6a3e997dc43a04bc209c31ac06d5dc: Status 404 returned error can't find the container with id be0c229cf412aead7fa1663a9bdf52fdac6a3e997dc43a04bc209c31ac06d5dc Jul 13 18:43:31.563696 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:31.563656 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jmzbd" event={"ID":"92869120-8a9e-4054-92f3-e9baa9bd383b","Type":"ContainerStarted","Data":"f26fdda2596bb45aed34c9081274a61fd2adba4908975cea2d131a6464b958b1"} Jul 13 18:43:31.564109 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:31.563723 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jmzbd" event={"ID":"92869120-8a9e-4054-92f3-e9baa9bd383b","Type":"ContainerStarted","Data":"be0c229cf412aead7fa1663a9bdf52fdac6a3e997dc43a04bc209c31ac06d5dc"} Jul 13 18:43:31.582885 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:31.582826 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jmzbd" podStartSLOduration=1.384688926 podStartE2EDuration="1.582807629s" totalImagesPullingTime="198.118703ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:43:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:43:31.581903164 +0000 UTC m=+667.865331799" watchObservedRunningTime="2026-07-13 18:43:31.582807629 +0000 UTC m=+667.866236231" Jul 13 18:43:33.079338 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:33.079300 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:43:33.142064 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:33.142032 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:43:36.274029 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:36.273975 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vt48k" Jul 13 18:43:36.274430 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:36.274073 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vt48k" Jul 13 18:43:36.286739 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:36.286712 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vt48k" Jul 13 18:43:36.601514 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:36.601432 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vt48k" Jul 13 18:43:40.574664 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:40.574629 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jmzbd" Jul 13 18:43:40.575084 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:40.574714 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jmzbd" Jul 13 18:43:40.587090 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:40.587070 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jmzbd" Jul 13 18:43:40.621266 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:40.621241 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jmzbd" Jul 13 18:43:43.401092 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:43:43.401054 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:44:10.104108 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:44:10.104046 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:44:29.904269 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:44:29.904178 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:45:00.150431 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:00.150396 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732805-tnpdt"] Jul 13 18:45:00.153951 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:00.153927 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732805-tnpdt" Jul 13 18:45:00.156815 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:00.156797 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lkrf8\"" Jul 13 18:45:00.163462 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:00.163436 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732805-tnpdt"] Jul 13 18:45:00.164892 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:00.164874 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732805-tnpdt" Jul 13 18:45:00.301951 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:00.301918 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732805-tnpdt"] Jul 13 18:45:00.304245 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:45:00.304219 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa557cf2_3943_470e_99fb_40bc4620162d.slice/crio-9a5c271da0a5fb07db4c67c162437c98a0e6b89a00e9caba7395aa5705ac41b8 WatchSource:0}: Error finding container 9a5c271da0a5fb07db4c67c162437c98a0e6b89a00e9caba7395aa5705ac41b8: Status 404 returned error can't find the container with id 9a5c271da0a5fb07db4c67c162437c98a0e6b89a00e9caba7395aa5705ac41b8 Jul 13 18:45:01.007846 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:01.007814 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732805-tnpdt" event={"ID":"fa557cf2-3943-470e-99fb-40bc4620162d","Type":"ContainerStarted","Data":"9a5c271da0a5fb07db4c67c162437c98a0e6b89a00e9caba7395aa5705ac41b8"} Jul 13 18:45:03.020020 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:03.019982 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732805-tnpdt" event={"ID":"fa557cf2-3943-470e-99fb-40bc4620162d","Type":"ContainerStarted","Data":"a5a58b2fbd3cd615cd6331006f74b0c203dc1cc5a4b928df4ab3b0a6711bf2e1"} Jul 13 18:45:03.037124 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:03.036962 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732805-tnpdt" podStartSLOduration=0.752202756 podStartE2EDuration="3.036942341s" totalImagesPullingTime="2.284739585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:45:03.036829374 +0000 UTC m=+759.320258122" watchObservedRunningTime="2026-07-13 18:45:03.036942341 +0000 UTC m=+759.320370940" Jul 13 18:45:04.026289 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:04.026259 2582 generic.go:356] "Generic (PLEG): container finished" podID="fa557cf2-3943-470e-99fb-40bc4620162d" containerID="a5a58b2fbd3cd615cd6331006f74b0c203dc1cc5a4b928df4ab3b0a6711bf2e1" exitCode=0 Jul 13 18:45:04.026690 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:04.026347 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732805-tnpdt" event={"ID":"fa557cf2-3943-470e-99fb-40bc4620162d","Type":"ContainerDied","Data":"a5a58b2fbd3cd615cd6331006f74b0c203dc1cc5a4b928df4ab3b0a6711bf2e1"} Jul 13 18:45:05.163898 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:05.163875 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732805-tnpdt" Jul 13 18:45:06.036930 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:06.036905 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732805-tnpdt" Jul 13 18:45:06.037076 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:06.036901 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732805-tnpdt" event={"ID":"fa557cf2-3943-470e-99fb-40bc4620162d","Type":"ContainerDied","Data":"9a5c271da0a5fb07db4c67c162437c98a0e6b89a00e9caba7395aa5705ac41b8"} Jul 13 18:45:06.037076 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:06.037034 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9a5c271da0a5fb07db4c67c162437c98a0e6b89a00e9caba7395aa5705ac41b8" Jul 13 18:45:38.164984 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:38.164946 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7466b976c5-65jm2"] Jul 13 18:45:38.165433 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:38.165352 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa557cf2-3943-470e-99fb-40bc4620162d" containerName="cleanup" Jul 13 18:45:38.165433 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:38.165363 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa557cf2-3943-470e-99fb-40bc4620162d" containerName="cleanup" Jul 13 18:45:38.165433 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:38.165428 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa557cf2-3943-470e-99fb-40bc4620162d" containerName="cleanup" Jul 13 18:45:38.170353 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:38.170337 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7466b976c5-65jm2" Jul 13 18:45:38.176408 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:38.176384 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7466b976c5-65jm2"] Jul 13 18:45:38.262924 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:38.262889 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v5zk\" (UniqueName: \"kubernetes.io/projected/bf8799c3-1a9e-44b3-9ea8-821ce4309b84-kube-api-access-6v5zk\") pod \"authorino-7466b976c5-65jm2\" (UID: \"bf8799c3-1a9e-44b3-9ea8-821ce4309b84\") " pod="kuadrant-system/authorino-7466b976c5-65jm2" Jul 13 18:45:38.263089 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:38.262953 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bf8799c3-1a9e-44b3-9ea8-821ce4309b84-tls-cert\") pod \"authorino-7466b976c5-65jm2\" (UID: \"bf8799c3-1a9e-44b3-9ea8-821ce4309b84\") " pod="kuadrant-system/authorino-7466b976c5-65jm2" Jul 13 18:45:38.364247 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:38.364215 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v5zk\" (UniqueName: \"kubernetes.io/projected/bf8799c3-1a9e-44b3-9ea8-821ce4309b84-kube-api-access-6v5zk\") pod \"authorino-7466b976c5-65jm2\" (UID: \"bf8799c3-1a9e-44b3-9ea8-821ce4309b84\") " pod="kuadrant-system/authorino-7466b976c5-65jm2" Jul 13 18:45:38.364412 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:38.364270 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bf8799c3-1a9e-44b3-9ea8-821ce4309b84-tls-cert\") pod \"authorino-7466b976c5-65jm2\" (UID: \"bf8799c3-1a9e-44b3-9ea8-821ce4309b84\") " pod="kuadrant-system/authorino-7466b976c5-65jm2" Jul 13 18:45:38.366944 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:38.366919 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bf8799c3-1a9e-44b3-9ea8-821ce4309b84-tls-cert\") pod \"authorino-7466b976c5-65jm2\" (UID: \"bf8799c3-1a9e-44b3-9ea8-821ce4309b84\") " pod="kuadrant-system/authorino-7466b976c5-65jm2" Jul 13 18:45:38.372127 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:38.372106 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v5zk\" (UniqueName: \"kubernetes.io/projected/bf8799c3-1a9e-44b3-9ea8-821ce4309b84-kube-api-access-6v5zk\") pod \"authorino-7466b976c5-65jm2\" (UID: \"bf8799c3-1a9e-44b3-9ea8-821ce4309b84\") " pod="kuadrant-system/authorino-7466b976c5-65jm2" Jul 13 18:45:38.481255 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:38.481228 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7466b976c5-65jm2" Jul 13 18:45:38.614042 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:38.614013 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7466b976c5-65jm2"] Jul 13 18:45:38.615756 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:45:38.615732 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbf8799c3_1a9e_44b3_9ea8_821ce4309b84.slice/crio-48ff42f7e4ef6c155716f79a0df5cdabc1e4fc790a010e43f325b98b303e34df WatchSource:0}: Error finding container 48ff42f7e4ef6c155716f79a0df5cdabc1e4fc790a010e43f325b98b303e34df: Status 404 returned error can't find the container with id 48ff42f7e4ef6c155716f79a0df5cdabc1e4fc790a010e43f325b98b303e34df Jul 13 18:45:39.198437 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:39.198406 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7466b976c5-65jm2" event={"ID":"bf8799c3-1a9e-44b3-9ea8-821ce4309b84","Type":"ContainerStarted","Data":"48ff42f7e4ef6c155716f79a0df5cdabc1e4fc790a010e43f325b98b303e34df"} Jul 13 18:45:40.206342 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:40.206297 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7466b976c5-65jm2" event={"ID":"bf8799c3-1a9e-44b3-9ea8-821ce4309b84","Type":"ContainerStarted","Data":"27c702b17d1a5bc45c2386775bd495d3e445a14e4dfd9a0f5d939e704fb6ec86"} Jul 13 18:45:40.223873 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:40.223816 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7466b976c5-65jm2" podStartSLOduration=1.648724558 podStartE2EDuration="2.22379939s" totalImagesPullingTime="575.074832ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:45:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:45:40.221747835 +0000 UTC m=+796.505176434" watchObservedRunningTime="2026-07-13 18:45:40.22379939 +0000 UTC m=+796.507227991" Jul 13 18:45:40.251726 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:40.251692 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-65b6464f94-8lclh"] Jul 13 18:45:40.251924 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:40.251901 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-65b6464f94-8lclh" podUID="5ac368a0-78bb-44be-89ec-6a1ce1e18b41" containerName="authorino" containerID="cri-o://42f182f0b92730d1edb02a389aceb4f68b13bd7fa20dcf5643a6719e202e6790" gracePeriod=30 Jul 13 18:45:40.506565 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:40.506538 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65b6464f94-8lclh" Jul 13 18:45:40.585568 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:40.585522 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5ac368a0-78bb-44be-89ec-6a1ce1e18b41-tls-cert\") pod \"5ac368a0-78bb-44be-89ec-6a1ce1e18b41\" (UID: \"5ac368a0-78bb-44be-89ec-6a1ce1e18b41\") " Jul 13 18:45:40.585769 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:40.585674 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hlggs\" (UniqueName: \"kubernetes.io/projected/5ac368a0-78bb-44be-89ec-6a1ce1e18b41-kube-api-access-hlggs\") pod \"5ac368a0-78bb-44be-89ec-6a1ce1e18b41\" (UID: \"5ac368a0-78bb-44be-89ec-6a1ce1e18b41\") " Jul 13 18:45:40.587744 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:40.587714 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ac368a0-78bb-44be-89ec-6a1ce1e18b41-kube-api-access-hlggs" (OuterVolumeSpecName: "kube-api-access-hlggs") pod "5ac368a0-78bb-44be-89ec-6a1ce1e18b41" (UID: "5ac368a0-78bb-44be-89ec-6a1ce1e18b41"). InnerVolumeSpecName "kube-api-access-hlggs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 18:45:40.596356 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:40.596331 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ac368a0-78bb-44be-89ec-6a1ce1e18b41-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "5ac368a0-78bb-44be-89ec-6a1ce1e18b41" (UID: "5ac368a0-78bb-44be-89ec-6a1ce1e18b41"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 18:45:40.686949 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:40.686909 2582 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5ac368a0-78bb-44be-89ec-6a1ce1e18b41-tls-cert\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:45:40.686949 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:40.686948 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hlggs\" (UniqueName: \"kubernetes.io/projected/5ac368a0-78bb-44be-89ec-6a1ce1e18b41-kube-api-access-hlggs\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:45:40.821101 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:40.821005 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:45:40.920763 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:40.920723 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:45:41.212901 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:41.212865 2582 generic.go:356] "Generic (PLEG): container finished" podID="5ac368a0-78bb-44be-89ec-6a1ce1e18b41" containerID="42f182f0b92730d1edb02a389aceb4f68b13bd7fa20dcf5643a6719e202e6790" exitCode=0 Jul 13 18:45:41.212901 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:41.212911 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65b6464f94-8lclh" Jul 13 18:45:41.213425 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:41.212949 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65b6464f94-8lclh" event={"ID":"5ac368a0-78bb-44be-89ec-6a1ce1e18b41","Type":"ContainerDied","Data":"42f182f0b92730d1edb02a389aceb4f68b13bd7fa20dcf5643a6719e202e6790"} Jul 13 18:45:41.213425 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:41.212997 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65b6464f94-8lclh" event={"ID":"5ac368a0-78bb-44be-89ec-6a1ce1e18b41","Type":"ContainerDied","Data":"61d9a62679ed5c75077966f14bcc7351b2b5a702bb65ba6a01876166cecebd4e"} Jul 13 18:45:41.213425 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:41.213020 2582 scope.go:117] "RemoveContainer" containerID="42f182f0b92730d1edb02a389aceb4f68b13bd7fa20dcf5643a6719e202e6790" Jul 13 18:45:41.223078 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:41.223058 2582 scope.go:117] "RemoveContainer" containerID="42f182f0b92730d1edb02a389aceb4f68b13bd7fa20dcf5643a6719e202e6790" Jul 13 18:45:41.223347 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:45:41.223324 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42f182f0b92730d1edb02a389aceb4f68b13bd7fa20dcf5643a6719e202e6790\": container with ID starting with 42f182f0b92730d1edb02a389aceb4f68b13bd7fa20dcf5643a6719e202e6790 not found: ID does not exist" containerID="42f182f0b92730d1edb02a389aceb4f68b13bd7fa20dcf5643a6719e202e6790" Jul 13 18:45:41.223456 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:41.223354 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42f182f0b92730d1edb02a389aceb4f68b13bd7fa20dcf5643a6719e202e6790"} err="failed to get container status \"42f182f0b92730d1edb02a389aceb4f68b13bd7fa20dcf5643a6719e202e6790\": rpc error: code = NotFound desc = could not find container \"42f182f0b92730d1edb02a389aceb4f68b13bd7fa20dcf5643a6719e202e6790\": container with ID starting with 42f182f0b92730d1edb02a389aceb4f68b13bd7fa20dcf5643a6719e202e6790 not found: ID does not exist" Jul 13 18:45:41.235074 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:41.235039 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-65b6464f94-8lclh"] Jul 13 18:45:41.238910 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:41.238888 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-65b6464f94-8lclh"] Jul 13 18:45:42.084728 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:42.084693 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:45:42.177308 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:42.177275 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:45:42.280656 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:45:42.280625 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ac368a0-78bb-44be-89ec-6a1ce1e18b41" path="/var/lib/kubelet/pods/5ac368a0-78bb-44be-89ec-6a1ce1e18b41/volumes" Jul 13 18:46:22.874395 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:46:22.874363 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:46:53.766675 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:46:53.766620 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:46:55.055519 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:46:55.055480 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:47:24.175998 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:47:24.175911 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8l7w6_b14ef695-c589-4fca-bd12-2e951f04e466/ovn-acl-logging/0.log" Jul 13 18:47:24.177726 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:47:24.177702 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8l7w6_b14ef695-c589-4fca-bd12-2e951f04e466/ovn-acl-logging/0.log" Jul 13 18:47:29.949032 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:47:29.948998 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:47:30.061780 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:47:30.061742 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:47:30.185309 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:47:30.185276 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:47:36.072403 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:47:36.072365 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:47:36.144796 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:47:36.144760 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:47:36.262022 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:47:36.261994 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:47:36.419327 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:47:36.419231 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:47:41.630395 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:47:41.630360 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:47:41.691270 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:47:41.691213 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:47:42.437192 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:47:42.437137 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:47:42.550141 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:47:42.550103 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:48:17.390970 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:48:17.390938 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:50:48.232214 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:50:48.232177 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:50:48.350495 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:50:48.350462 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:51:31.342479 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:51:31.342444 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:52:24.210765 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:52:24.210738 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8l7w6_b14ef695-c589-4fca-bd12-2e951f04e466/ovn-acl-logging/0.log" Jul 13 18:52:24.213832 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:52:24.213814 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8l7w6_b14ef695-c589-4fca-bd12-2e951f04e466/ovn-acl-logging/0.log" Jul 13 18:54:31.375123 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:54:31.375086 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:55:05.372930 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:55:05.372847 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:55:22.972410 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:55:22.972377 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:57:24.243218 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:57:24.243191 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8l7w6_b14ef695-c589-4fca-bd12-2e951f04e466/ovn-acl-logging/0.log" Jul 13 18:57:24.246894 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:57:24.246873 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8l7w6_b14ef695-c589-4fca-bd12-2e951f04e466/ovn-acl-logging/0.log" Jul 13 18:58:06.964655 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:58:06.964535 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:58:42.830863 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:58:42.830822 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:58:50.569115 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:58:50.569072 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:58:58.834846 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:58:58.834809 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:03.889171 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:03.889128 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:06.871131 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:06.871099 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:06.971977 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:06.971947 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:07.362490 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:07.362459 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:13.405887 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:13.405845 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:13.477998 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:13.477964 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:13.626598 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:13.626547 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:13.796567 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:13.796526 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:14.803159 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:14.803066 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:14.907974 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:14.907922 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:15.025172 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:15.025135 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:15.511688 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:15.511652 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:15.610557 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:15.610522 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:17.606357 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:17.606307 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:19.826398 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:19.826367 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7c6e726-rwr9g"] Jul 13 18:59:19.826786 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:19.826778 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ac368a0-78bb-44be-89ec-6a1ce1e18b41" containerName="authorino" Jul 13 18:59:19.826827 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:19.826789 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ac368a0-78bb-44be-89ec-6a1ce1e18b41" containerName="authorino" Jul 13 18:59:19.826902 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:19.826877 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ac368a0-78bb-44be-89ec-6a1ce1e18b41" containerName="authorino" Jul 13 18:59:19.830050 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:19.830033 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7c6e726-rwr9g" Jul 13 18:59:19.832903 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:19.832879 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lkrf8\"" Jul 13 18:59:19.837749 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:19.837531 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7c6e726-rwr9g"] Jul 13 18:59:19.919134 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:19.919102 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/62bbf0e2-b358-4d2b-9406-2cfae65ae90b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f7c6e726-rwr9g\" (UID: \"62bbf0e2-b358-4d2b-9406-2cfae65ae90b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7c6e726-rwr9g" Jul 13 18:59:20.019961 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:20.019926 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/62bbf0e2-b358-4d2b-9406-2cfae65ae90b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f7c6e726-rwr9g\" (UID: \"62bbf0e2-b358-4d2b-9406-2cfae65ae90b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7c6e726-rwr9g" Jul 13 18:59:20.020527 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:20.020510 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/62bbf0e2-b358-4d2b-9406-2cfae65ae90b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f7c6e726-rwr9g\" (UID: \"62bbf0e2-b358-4d2b-9406-2cfae65ae90b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7c6e726-rwr9g" Jul 13 18:59:20.142540 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:20.142460 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7c6e726-rwr9g" Jul 13 18:59:20.278257 ip-10-0-129-191 kubenswrapper[2582]: W0713 18:59:20.278221 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62bbf0e2_b358_4d2b_9406_2cfae65ae90b.slice/crio-0e77143f078d88b3d2b7ceb4d9bb6d264eb178fe2e3b45f91e42cf9d0ea7392a WatchSource:0}: Error finding container 0e77143f078d88b3d2b7ceb4d9bb6d264eb178fe2e3b45f91e42cf9d0ea7392a: Status 404 returned error can't find the container with id 0e77143f078d88b3d2b7ceb4d9bb6d264eb178fe2e3b45f91e42cf9d0ea7392a Jul 13 18:59:20.282130 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:20.282111 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7c6e726-rwr9g"] Jul 13 18:59:20.399590 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:20.399505 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7c6e726-rwr9g" event={"ID":"62bbf0e2-b358-4d2b-9406-2cfae65ae90b","Type":"ContainerStarted","Data":"0e77143f078d88b3d2b7ceb4d9bb6d264eb178fe2e3b45f91e42cf9d0ea7392a"} Jul 13 18:59:21.406249 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:21.406216 2582 generic.go:356] "Generic (PLEG): container finished" podID="62bbf0e2-b358-4d2b-9406-2cfae65ae90b" containerID="80d8c23837e1d8ddf3f3254a64e2c1698d93beb12d3e33d56baf8dc8e64df49d" exitCode=7 Jul 13 18:59:21.406679 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:21.406310 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7c6e726-rwr9g" event={"ID":"62bbf0e2-b358-4d2b-9406-2cfae65ae90b","Type":"ContainerDied","Data":"80d8c23837e1d8ddf3f3254a64e2c1698d93beb12d3e33d56baf8dc8e64df49d"} Jul 13 18:59:21.406679 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:21.406467 2582 scope.go:117] "RemoveContainer" containerID="80d8c23837e1d8ddf3f3254a64e2c1698d93beb12d3e33d56baf8dc8e64df49d" Jul 13 18:59:22.413119 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:22.413027 2582 generic.go:356] "Generic (PLEG): container finished" podID="62bbf0e2-b358-4d2b-9406-2cfae65ae90b" containerID="4e1da336a5f294a72ccbd0f5c5830ecdfdab37ef8291c72cc75e0c03d9948758" exitCode=7 Jul 13 18:59:22.413541 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:22.413120 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7c6e726-rwr9g" event={"ID":"62bbf0e2-b358-4d2b-9406-2cfae65ae90b","Type":"ContainerDied","Data":"4e1da336a5f294a72ccbd0f5c5830ecdfdab37ef8291c72cc75e0c03d9948758"} Jul 13 18:59:22.413541 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:22.413178 2582 scope.go:117] "RemoveContainer" containerID="80d8c23837e1d8ddf3f3254a64e2c1698d93beb12d3e33d56baf8dc8e64df49d" Jul 13 18:59:22.413541 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:22.413454 2582 scope.go:117] "RemoveContainer" containerID="4e1da336a5f294a72ccbd0f5c5830ecdfdab37ef8291c72cc75e0c03d9948758" Jul 13 18:59:22.413753 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:59:22.413670 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f7c6e726-rwr9g_odh-ai-gateway-infra(62bbf0e2-b358-4d2b-9406-2cfae65ae90b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7c6e726-rwr9g" podUID="62bbf0e2-b358-4d2b-9406-2cfae65ae90b" Jul 13 18:59:23.420097 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:23.420072 2582 scope.go:117] "RemoveContainer" containerID="4e1da336a5f294a72ccbd0f5c5830ecdfdab37ef8291c72cc75e0c03d9948758" Jul 13 18:59:23.420458 ip-10-0-129-191 kubenswrapper[2582]: E0713 18:59:23.420268 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f7c6e726-rwr9g_odh-ai-gateway-infra(62bbf0e2-b358-4d2b-9406-2cfae65ae90b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7c6e726-rwr9g" podUID="62bbf0e2-b358-4d2b-9406-2cfae65ae90b" Jul 13 18:59:34.277147 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:34.277116 2582 scope.go:117] "RemoveContainer" containerID="4e1da336a5f294a72ccbd0f5c5830ecdfdab37ef8291c72cc75e0c03d9948758" Jul 13 18:59:35.481777 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:35.481740 2582 generic.go:356] "Generic (PLEG): container finished" podID="62bbf0e2-b358-4d2b-9406-2cfae65ae90b" containerID="b4353009094da740f407e0b0246595887f473998bcb4af97e1dab4db54be9768" exitCode=0 Jul 13 18:59:35.482295 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:35.481819 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7c6e726-rwr9g" event={"ID":"62bbf0e2-b358-4d2b-9406-2cfae65ae90b","Type":"ContainerDied","Data":"b4353009094da740f407e0b0246595887f473998bcb4af97e1dab4db54be9768"} Jul 13 18:59:35.482295 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:35.481880 2582 scope.go:117] "RemoveContainer" containerID="4e1da336a5f294a72ccbd0f5c5830ecdfdab37ef8291c72cc75e0c03d9948758" Jul 13 18:59:36.513948 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:36.513919 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7c6e726-rwr9g"] Jul 13 18:59:36.630499 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:36.630474 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7c6e726-rwr9g" Jul 13 18:59:36.668775 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:36.668752 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/62bbf0e2-b358-4d2b-9406-2cfae65ae90b-maas-api-service-ca\") pod \"62bbf0e2-b358-4d2b-9406-2cfae65ae90b\" (UID: \"62bbf0e2-b358-4d2b-9406-2cfae65ae90b\") " Jul 13 18:59:36.669061 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:36.669042 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62bbf0e2-b358-4d2b-9406-2cfae65ae90b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "62bbf0e2-b358-4d2b-9406-2cfae65ae90b" (UID: "62bbf0e2-b358-4d2b-9406-2cfae65ae90b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 18:59:36.770207 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:36.770134 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/62bbf0e2-b358-4d2b-9406-2cfae65ae90b-maas-api-service-ca\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 18:59:37.494170 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:37.494138 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7c6e726-rwr9g" event={"ID":"62bbf0e2-b358-4d2b-9406-2cfae65ae90b","Type":"ContainerDied","Data":"0e77143f078d88b3d2b7ceb4d9bb6d264eb178fe2e3b45f91e42cf9d0ea7392a"} Jul 13 18:59:37.494344 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:37.494184 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0e77143f078d88b3d2b7ceb4d9bb6d264eb178fe2e3b45f91e42cf9d0ea7392a" Jul 13 18:59:37.494344 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:37.494146 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7c6e726-rwr9g" Jul 13 18:59:37.522992 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:37.522967 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7c6e726-rwr9g"] Jul 13 18:59:37.538690 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:37.538660 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7c6e726-rwr9g"] Jul 13 18:59:38.280925 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:38.280892 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62bbf0e2-b358-4d2b-9406-2cfae65ae90b" path="/var/lib/kubelet/pods/62bbf0e2-b358-4d2b-9406-2cfae65ae90b/volumes" Jul 13 18:59:49.469900 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:49.469860 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:52.371208 ip-10-0-129-191 kubenswrapper[2582]: I0713 18:59:52.371165 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:00:00.141598 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:00.141562 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-f7c6e726-29732820-22lxl"] Jul 13 19:00:00.142218 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:00.142196 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62bbf0e2-b358-4d2b-9406-2cfae65ae90b" containerName="revoke-keys" Jul 13 19:00:00.142322 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:00.142221 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="62bbf0e2-b358-4d2b-9406-2cfae65ae90b" containerName="revoke-keys" Jul 13 19:00:00.142322 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:00.142253 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62bbf0e2-b358-4d2b-9406-2cfae65ae90b" containerName="revoke-keys" Jul 13 19:00:00.142322 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:00.142261 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="62bbf0e2-b358-4d2b-9406-2cfae65ae90b" containerName="revoke-keys" Jul 13 19:00:00.142322 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:00.142270 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62bbf0e2-b358-4d2b-9406-2cfae65ae90b" containerName="revoke-keys" Jul 13 19:00:00.142322 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:00.142279 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="62bbf0e2-b358-4d2b-9406-2cfae65ae90b" containerName="revoke-keys" Jul 13 19:00:00.142574 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:00.142367 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="62bbf0e2-b358-4d2b-9406-2cfae65ae90b" containerName="revoke-keys" Jul 13 19:00:00.142574 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:00.142380 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="62bbf0e2-b358-4d2b-9406-2cfae65ae90b" containerName="revoke-keys" Jul 13 19:00:00.145402 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:00.145379 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732820-cw7x6"] Jul 13 19:00:00.145537 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:00.145497 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-f7c6e726-29732820-22lxl" Jul 13 19:00:00.145959 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:00.145940 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="62bbf0e2-b358-4d2b-9406-2cfae65ae90b" containerName="revoke-keys" Jul 13 19:00:00.148498 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:00.148483 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lkrf8\"" Jul 13 19:00:00.149086 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:00.148944 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732820-cw7x6" Jul 13 19:00:00.152405 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:00.152387 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-f7c6e726-29732820-22lxl"] Jul 13 19:00:00.159000 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:00.158984 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-f7c6e726-29732820-22lxl" Jul 13 19:00:00.166853 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:00.166830 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732820-cw7x6"] Jul 13 19:00:00.168520 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:00.168485 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732820-cw7x6" Jul 13 19:00:00.319734 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:00.319702 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-f7c6e726-29732820-22lxl"] Jul 13 19:00:00.321256 ip-10-0-129-191 kubenswrapper[2582]: W0713 19:00:00.321226 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9e6907c_10f4_484a_bf38_c786b88c3809.slice/crio-fb3274e8f1b5bdf6a81b583f95abc001edf1a7d026215627272bd12cb7308323 WatchSource:0}: Error finding container fb3274e8f1b5bdf6a81b583f95abc001edf1a7d026215627272bd12cb7308323: Status 404 returned error can't find the container with id fb3274e8f1b5bdf6a81b583f95abc001edf1a7d026215627272bd12cb7308323 Jul 13 19:00:00.345298 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:00.345270 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732820-cw7x6"] Jul 13 19:00:00.348876 ip-10-0-129-191 kubenswrapper[2582]: W0713 19:00:00.348845 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c8f49e3_4619_4e18_a1c3_81abbf2b8115.slice/crio-d1970736b0d45e3e21ba91fc4ee7bf921e85130033be3a993b20ccb03da40682 WatchSource:0}: Error finding container d1970736b0d45e3e21ba91fc4ee7bf921e85130033be3a993b20ccb03da40682: Status 404 returned error can't find the container with id d1970736b0d45e3e21ba91fc4ee7bf921e85130033be3a993b20ccb03da40682 Jul 13 19:00:00.605851 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:00.605819 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732820-cw7x6" event={"ID":"2c8f49e3-4619-4e18-a1c3-81abbf2b8115","Type":"ContainerStarted","Data":"d1970736b0d45e3e21ba91fc4ee7bf921e85130033be3a993b20ccb03da40682"} Jul 13 19:00:00.606968 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:00.606944 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-f7c6e726-29732820-22lxl" event={"ID":"e9e6907c-10f4-484a-bf38-c786b88c3809","Type":"ContainerStarted","Data":"fb3274e8f1b5bdf6a81b583f95abc001edf1a7d026215627272bd12cb7308323"} Jul 13 19:00:01.613596 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:01.613505 2582 generic.go:356] "Generic (PLEG): container finished" podID="2c8f49e3-4619-4e18-a1c3-81abbf2b8115" containerID="cb07828e58f3d6790169f1a6e49d276e22fbbfd094244da97e9bdb6cebb4bfaf" exitCode=0 Jul 13 19:00:01.613596 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:01.613592 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732820-cw7x6" event={"ID":"2c8f49e3-4619-4e18-a1c3-81abbf2b8115","Type":"ContainerDied","Data":"cb07828e58f3d6790169f1a6e49d276e22fbbfd094244da97e9bdb6cebb4bfaf"} Jul 13 19:00:01.615014 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:01.614991 2582 generic.go:356] "Generic (PLEG): container finished" podID="e9e6907c-10f4-484a-bf38-c786b88c3809" containerID="accd7af7c738f782ff8cb81b770684db79d3901ff0086e272c379ba42c3b5ef5" exitCode=0 Jul 13 19:00:01.615129 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:01.615043 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-f7c6e726-29732820-22lxl" event={"ID":"e9e6907c-10f4-484a-bf38-c786b88c3809","Type":"ContainerDied","Data":"accd7af7c738f782ff8cb81b770684db79d3901ff0086e272c379ba42c3b5ef5"} Jul 13 19:00:02.760696 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:02.760668 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732820-cw7x6" Jul 13 19:00:02.793389 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:02.793366 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-f7c6e726-29732820-22lxl" Jul 13 19:00:03.626503 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:03.626473 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732820-cw7x6" Jul 13 19:00:03.626710 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:03.626474 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732820-cw7x6" event={"ID":"2c8f49e3-4619-4e18-a1c3-81abbf2b8115","Type":"ContainerDied","Data":"d1970736b0d45e3e21ba91fc4ee7bf921e85130033be3a993b20ccb03da40682"} Jul 13 19:00:03.626710 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:03.626593 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d1970736b0d45e3e21ba91fc4ee7bf921e85130033be3a993b20ccb03da40682" Jul 13 19:00:03.627811 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:03.627792 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-f7c6e726-29732820-22lxl" Jul 13 19:00:03.627920 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:03.627819 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-f7c6e726-29732820-22lxl" event={"ID":"e9e6907c-10f4-484a-bf38-c786b88c3809","Type":"ContainerDied","Data":"fb3274e8f1b5bdf6a81b583f95abc001edf1a7d026215627272bd12cb7308323"} Jul 13 19:00:03.627920 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:03.627857 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fb3274e8f1b5bdf6a81b583f95abc001edf1a7d026215627272bd12cb7308323" Jul 13 19:00:09.874816 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:09.874785 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7c6e726-8x98p"] Jul 13 19:00:09.875261 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:09.875199 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c8f49e3-4619-4e18-a1c3-81abbf2b8115" containerName="cleanup" Jul 13 19:00:09.875261 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:09.875211 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c8f49e3-4619-4e18-a1c3-81abbf2b8115" containerName="cleanup" Jul 13 19:00:09.875261 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:09.875227 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9e6907c-10f4-484a-bf38-c786b88c3809" containerName="cleanup" Jul 13 19:00:09.875261 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:09.875233 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9e6907c-10f4-484a-bf38-c786b88c3809" containerName="cleanup" Jul 13 19:00:09.875410 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:09.875300 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c8f49e3-4619-4e18-a1c3-81abbf2b8115" containerName="cleanup" Jul 13 19:00:09.875410 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:09.875308 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9e6907c-10f4-484a-bf38-c786b88c3809" containerName="cleanup" Jul 13 19:00:09.878701 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:09.878684 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7c6e726-8x98p" Jul 13 19:00:09.881983 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:09.881960 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lkrf8\"" Jul 13 19:00:09.888806 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:09.888781 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7c6e726-8x98p"] Jul 13 19:00:09.978719 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:09.978686 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc6c6f41-6d9a-46d6-ac6f-1a50c3fbdab1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f7c6e726-8x98p\" (UID: \"dc6c6f41-6d9a-46d6-ac6f-1a50c3fbdab1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7c6e726-8x98p" Jul 13 19:00:10.080116 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:10.080078 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc6c6f41-6d9a-46d6-ac6f-1a50c3fbdab1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f7c6e726-8x98p\" (UID: \"dc6c6f41-6d9a-46d6-ac6f-1a50c3fbdab1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7c6e726-8x98p" Jul 13 19:00:10.080742 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:10.080716 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc6c6f41-6d9a-46d6-ac6f-1a50c3fbdab1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f7c6e726-8x98p\" (UID: \"dc6c6f41-6d9a-46d6-ac6f-1a50c3fbdab1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7c6e726-8x98p" Jul 13 19:00:10.190306 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:10.190204 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7c6e726-8x98p" Jul 13 19:00:10.330682 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:10.330654 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7c6e726-8x98p"] Jul 13 19:00:10.333228 ip-10-0-129-191 kubenswrapper[2582]: W0713 19:00:10.333201 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc6c6f41_6d9a_46d6_ac6f_1a50c3fbdab1.slice/crio-d82ce1654f4b3772abe78d67e8afded1f4ff2aa644e0a377b2a0294023504078 WatchSource:0}: Error finding container d82ce1654f4b3772abe78d67e8afded1f4ff2aa644e0a377b2a0294023504078: Status 404 returned error can't find the container with id d82ce1654f4b3772abe78d67e8afded1f4ff2aa644e0a377b2a0294023504078 Jul 13 19:00:10.664348 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:10.664316 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7c6e726-8x98p" event={"ID":"dc6c6f41-6d9a-46d6-ac6f-1a50c3fbdab1","Type":"ContainerStarted","Data":"bb39b6b029c3c835ad80dbceb8b0f5f3b9a7c82ec119aaa8ec96c48798f2f7b1"} Jul 13 19:00:10.664470 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:10.664393 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7c6e726-8x98p" event={"ID":"dc6c6f41-6d9a-46d6-ac6f-1a50c3fbdab1","Type":"ContainerStarted","Data":"d82ce1654f4b3772abe78d67e8afded1f4ff2aa644e0a377b2a0294023504078"} Jul 13 19:00:11.671554 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:11.671514 2582 generic.go:356] "Generic (PLEG): container finished" podID="dc6c6f41-6d9a-46d6-ac6f-1a50c3fbdab1" containerID="bb39b6b029c3c835ad80dbceb8b0f5f3b9a7c82ec119aaa8ec96c48798f2f7b1" exitCode=0 Jul 13 19:00:11.671983 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:11.671571 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7c6e726-8x98p" event={"ID":"dc6c6f41-6d9a-46d6-ac6f-1a50c3fbdab1","Type":"ContainerDied","Data":"bb39b6b029c3c835ad80dbceb8b0f5f3b9a7c82ec119aaa8ec96c48798f2f7b1"} Jul 13 19:00:12.817757 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:12.817734 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7c6e726-8x98p" Jul 13 19:00:12.907532 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:12.907493 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc6c6f41-6d9a-46d6-ac6f-1a50c3fbdab1-maas-api-service-ca\") pod \"dc6c6f41-6d9a-46d6-ac6f-1a50c3fbdab1\" (UID: \"dc6c6f41-6d9a-46d6-ac6f-1a50c3fbdab1\") " Jul 13 19:00:12.907873 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:12.907846 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc6c6f41-6d9a-46d6-ac6f-1a50c3fbdab1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dc6c6f41-6d9a-46d6-ac6f-1a50c3fbdab1" (UID: "dc6c6f41-6d9a-46d6-ac6f-1a50c3fbdab1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:00:13.008924 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:13.008889 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc6c6f41-6d9a-46d6-ac6f-1a50c3fbdab1-maas-api-service-ca\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:00:13.683521 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:13.683491 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7c6e726-8x98p" Jul 13 19:00:13.683723 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:13.683491 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7c6e726-8x98p" event={"ID":"dc6c6f41-6d9a-46d6-ac6f-1a50c3fbdab1","Type":"ContainerDied","Data":"d82ce1654f4b3772abe78d67e8afded1f4ff2aa644e0a377b2a0294023504078"} Jul 13 19:00:13.683723 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:13.683635 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d82ce1654f4b3772abe78d67e8afded1f4ff2aa644e0a377b2a0294023504078" Jul 13 19:00:20.346824 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:20.346793 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-f7c6e726-29732820-22lxl"] Jul 13 19:00:20.351193 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:20.351160 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:00:20.352270 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:20.352251 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-f7c6e726-29732820-22lxl"] Jul 13 19:00:22.281521 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:22.281485 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9e6907c-10f4-484a-bf38-c786b88c3809" path="/var/lib/kubelet/pods/e9e6907c-10f4-484a-bf38-c786b88c3809/volumes" Jul 13 19:00:24.969751 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:24.969682 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:00:31.144588 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:31.144533 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:00:31.228971 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:31.228935 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:00:35.166119 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:35.166082 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:00:35.605138 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:35.605105 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:00:35.718241 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:35.718203 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:00:35.759134 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:35.759100 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj"] Jul 13 19:00:35.759540 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:35.759526 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc6c6f41-6d9a-46d6-ac6f-1a50c3fbdab1" containerName="revoke-keys" Jul 13 19:00:35.759619 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:35.759544 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc6c6f41-6d9a-46d6-ac6f-1a50c3fbdab1" containerName="revoke-keys" Jul 13 19:00:35.759739 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:35.759726 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc6c6f41-6d9a-46d6-ac6f-1a50c3fbdab1" containerName="revoke-keys" Jul 13 19:00:35.763269 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:35.763248 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:00:35.769109 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:35.769086 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 13 19:00:35.769238 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:35.769151 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-78327326-gw-openshift-default-dockercfg-v6qcl\"" Jul 13 19:00:35.779330 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:35.779305 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj"] Jul 13 19:00:35.830435 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:35.830405 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:00:35.924107 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:35.924013 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-istio-data\") pod \"e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj\" (UID: \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\") " pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:00:35.924107 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:35.924056 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-credential-socket\") pod \"e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj\" (UID: \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\") " pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:00:35.924107 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:35.924091 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-workload-socket\") pod \"e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj\" (UID: \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\") " pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:00:35.924386 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:35.924218 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-istiod-ca-cert\") pod \"e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj\" (UID: \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\") " pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:00:35.924386 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:35.924254 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wbpz\" (UniqueName: \"kubernetes.io/projected/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-kube-api-access-6wbpz\") pod \"e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj\" (UID: \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\") " pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:00:35.924386 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:35.924273 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-istio-envoy\") pod \"e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj\" (UID: \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\") " pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:00:35.924386 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:35.924358 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-istio-podinfo\") pod \"e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj\" (UID: \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\") " pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:00:35.924572 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:35.924428 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-istio-token\") pod \"e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj\" (UID: \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\") " pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:00:35.924572 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:35.924470 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-workload-certs\") pod \"e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj\" (UID: \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\") " pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:00:36.026002 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:36.025967 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-istiod-ca-cert\") pod \"e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj\" (UID: \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\") " pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:00:36.026002 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:36.026005 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wbpz\" (UniqueName: \"kubernetes.io/projected/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-kube-api-access-6wbpz\") pod \"e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj\" (UID: \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\") " pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:00:36.026248 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:36.026023 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-istio-envoy\") pod \"e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj\" (UID: \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\") " pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:00:36.026248 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:36.026167 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-istio-podinfo\") pod \"e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj\" (UID: \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\") " pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:00:36.026248 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:36.026231 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-istio-token\") pod \"e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj\" (UID: \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\") " pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:00:36.026436 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:36.026271 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-workload-certs\") pod \"e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj\" (UID: \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\") " pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:00:36.026436 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:36.026311 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-istio-data\") pod \"e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj\" (UID: \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\") " pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:00:36.026436 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:36.026353 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-credential-socket\") pod \"e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj\" (UID: \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\") " pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:00:36.026436 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:36.026399 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-workload-socket\") pod \"e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj\" (UID: \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\") " pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:00:36.026747 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:36.026727 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-credential-socket\") pod \"e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj\" (UID: \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\") " pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:00:36.026838 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:36.026819 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-workload-certs\") pod \"e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj\" (UID: \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\") " pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:00:36.026896 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:36.026868 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-istiod-ca-cert\") pod \"e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj\" (UID: \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\") " pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:00:36.026944 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:36.026891 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-workload-socket\") pod \"e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj\" (UID: \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\") " pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:00:36.026944 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:36.026923 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-istio-data\") pod \"e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj\" (UID: \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\") " pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:00:36.028747 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:36.028721 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-istio-envoy\") pod \"e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj\" (UID: \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\") " pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:00:36.028888 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:36.028872 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-istio-podinfo\") pod \"e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj\" (UID: \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\") " pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:00:36.036491 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:36.036465 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-istio-token\") pod \"e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj\" (UID: \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\") " pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:00:36.036594 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:36.036516 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wbpz\" (UniqueName: \"kubernetes.io/projected/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-kube-api-access-6wbpz\") pod \"e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj\" (UID: \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\") " pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:00:36.074696 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:36.074648 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:00:36.221779 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:36.221750 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj"] Jul 13 19:00:36.223574 ip-10-0-129-191 kubenswrapper[2582]: W0713 19:00:36.223546 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d9c1099_81ce_4cb8_8bb1_52bd5c386285.slice/crio-49b939076cee8b8d341bebc7dccf54945ea155b31fa269e75115eba6a39f3928 WatchSource:0}: Error finding container 49b939076cee8b8d341bebc7dccf54945ea155b31fa269e75115eba6a39f3928: Status 404 returned error can't find the container with id 49b939076cee8b8d341bebc7dccf54945ea155b31fa269e75115eba6a39f3928 Jul 13 19:00:36.225588 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:36.225573 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 19:00:36.801967 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:36.801932 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" event={"ID":"1d9c1099-81ce-4cb8-8bb1-52bd5c386285","Type":"ContainerStarted","Data":"49b939076cee8b8d341bebc7dccf54945ea155b31fa269e75115eba6a39f3928"} Jul 13 19:00:37.096318 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:37.096168 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:00:37.209778 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:37.209738 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:00:37.686188 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:37.686148 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:00:37.785902 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:37.785857 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:00:38.030704 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:38.030665 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:00:39.033734 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:39.033688 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 19:00:39.034128 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:39.033799 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 19:00:39.034128 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:39.033847 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 19:00:39.821755 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:39.821717 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" event={"ID":"1d9c1099-81ce-4cb8-8bb1-52bd5c386285","Type":"ContainerStarted","Data":"566ebc951b26d592b775854e22d48e86ecc84f0315fb805ca20a15326b7fe831"} Jul 13 19:00:39.846793 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:39.846742 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" podStartSLOduration=2.039145545 podStartE2EDuration="4.846724436s" totalImagesPullingTime="2.807578891s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:00:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:00:39.84427355 +0000 UTC m=+1696.127702192" watchObservedRunningTime="2026-07-13 19:00:39.846724436 +0000 UTC m=+1696.130153038" Jul 13 19:00:40.075906 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:40.075830 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:00:40.081000 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:40.080972 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:00:40.827998 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:40.827968 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:00:40.829036 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:40.829013 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:00:41.447520 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:41.447485 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:00:41.511814 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:41.511766 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:00:41.846138 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:41.846102 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:00:41.904454 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:41.904422 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:00:46.077043 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:46.076980 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:00:53.912372 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:53.912336 2582 generic.go:356] "Generic (PLEG): container finished" podID="1d9c1099-81ce-4cb8-8bb1-52bd5c386285" containerID="566ebc951b26d592b775854e22d48e86ecc84f0315fb805ca20a15326b7fe831" exitCode=0 Jul 13 19:00:53.912871 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:53.912410 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" event={"ID":"1d9c1099-81ce-4cb8-8bb1-52bd5c386285","Type":"ContainerDied","Data":"566ebc951b26d592b775854e22d48e86ecc84f0315fb805ca20a15326b7fe831"} Jul 13 19:00:53.912871 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:53.912831 2582 scope.go:117] "RemoveContainer" containerID="566ebc951b26d592b775854e22d48e86ecc84f0315fb805ca20a15326b7fe831" Jul 13 19:00:53.914735 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:53.914706 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 19:00:53.914845 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:53.914792 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 19:00:53.914845 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:53.914839 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 19:00:54.920181 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:54.920149 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" event={"ID":"1d9c1099-81ce-4cb8-8bb1-52bd5c386285","Type":"ContainerStarted","Data":"d565d9531abb7fed5be90cee5c82fd15e9837d8954475b3af75ee9b9a949a4f1"} Jul 13 19:00:55.075029 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:55.074990 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:00:55.076045 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:55.076015 2582 patch_prober.go:28] interesting pod/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 13 19:00:55.076165 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:55.076081 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" podUID="1d9c1099-81ce-4cb8-8bb1-52bd5c386285" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 13 19:00:55.828539 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:55.828501 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:00:56.075436 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:56.075396 2582 patch_prober.go:28] interesting pod/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 13 19:00:56.075436 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:56.075454 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" podUID="1d9c1099-81ce-4cb8-8bb1-52bd5c386285" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 13 19:00:57.075662 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:57.075588 2582 patch_prober.go:28] interesting pod/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 13 19:00:57.076043 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:57.075733 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" podUID="1d9c1099-81ce-4cb8-8bb1-52bd5c386285" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 13 19:00:58.075270 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:58.075233 2582 patch_prober.go:28] interesting pod/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 13 19:00:58.075477 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:58.075299 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" podUID="1d9c1099-81ce-4cb8-8bb1-52bd5c386285" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 13 19:00:59.079206 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:59.079175 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:00:59.080225 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:00:59.080207 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:01:41.652412 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:41.652376 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj"] Jul 13 19:01:41.652927 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:41.652596 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" podUID="1d9c1099-81ce-4cb8-8bb1-52bd5c386285" containerName="istio-proxy" containerID="cri-o://d565d9531abb7fed5be90cee5c82fd15e9837d8954475b3af75ee9b9a949a4f1" gracePeriod=30 Jul 13 19:01:41.688929 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:41.688897 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:01:41.834095 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:41.834061 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:01:44.081763 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:44.081726 2582 patch_prober.go:28] interesting pod/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 13 19:01:44.082161 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:44.081796 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" podUID="1d9c1099-81ce-4cb8-8bb1-52bd5c386285" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 19:01:46.912510 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:46.912484 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:01:47.049278 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.049245 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-istiod-ca-cert\") pod \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\" (UID: \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\") " Jul 13 19:01:47.049465 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.049288 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-credential-socket\") pod \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\" (UID: \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\") " Jul 13 19:01:47.049465 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.049306 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6wbpz\" (UniqueName: \"kubernetes.io/projected/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-kube-api-access-6wbpz\") pod \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\" (UID: \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\") " Jul 13 19:01:47.049465 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.049366 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-workload-certs\") pod \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\" (UID: \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\") " Jul 13 19:01:47.049465 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.049393 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-istio-envoy\") pod \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\" (UID: \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\") " Jul 13 19:01:47.049465 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.049419 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-istio-podinfo\") pod \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\" (UID: \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\") " Jul 13 19:01:47.049465 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.049445 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-istio-token\") pod \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\" (UID: \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\") " Jul 13 19:01:47.049783 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.049474 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-workload-socket\") pod \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\" (UID: \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\") " Jul 13 19:01:47.049783 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.049534 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-istio-data\") pod \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\" (UID: \"1d9c1099-81ce-4cb8-8bb1-52bd5c386285\") " Jul 13 19:01:47.049783 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.049675 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1d9c1099-81ce-4cb8-8bb1-52bd5c386285" (UID: "1d9c1099-81ce-4cb8-8bb1-52bd5c386285"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:01:47.049783 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.049707 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1d9c1099-81ce-4cb8-8bb1-52bd5c386285" (UID: "1d9c1099-81ce-4cb8-8bb1-52bd5c386285"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:01:47.050002 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.049819 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1d9c1099-81ce-4cb8-8bb1-52bd5c386285" (UID: "1d9c1099-81ce-4cb8-8bb1-52bd5c386285"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:01:47.050002 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.049923 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1d9c1099-81ce-4cb8-8bb1-52bd5c386285" (UID: "1d9c1099-81ce-4cb8-8bb1-52bd5c386285"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:01:47.050166 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.050147 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-workload-socket\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:01:47.050226 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.050170 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-istiod-ca-cert\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:01:47.050226 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.050185 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-credential-socket\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:01:47.050226 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.050198 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-workload-certs\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:01:47.050567 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.050490 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-istio-data" (OuterVolumeSpecName: "istio-data") pod "1d9c1099-81ce-4cb8-8bb1-52bd5c386285" (UID: "1d9c1099-81ce-4cb8-8bb1-52bd5c386285"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:01:47.051964 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.051940 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-kube-api-access-6wbpz" (OuterVolumeSpecName: "kube-api-access-6wbpz") pod "1d9c1099-81ce-4cb8-8bb1-52bd5c386285" (UID: "1d9c1099-81ce-4cb8-8bb1-52bd5c386285"). InnerVolumeSpecName "kube-api-access-6wbpz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:01:47.052079 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.052063 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-istio-token" (OuterVolumeSpecName: "istio-token") pod "1d9c1099-81ce-4cb8-8bb1-52bd5c386285" (UID: "1d9c1099-81ce-4cb8-8bb1-52bd5c386285"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:01:47.052249 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.052223 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1d9c1099-81ce-4cb8-8bb1-52bd5c386285" (UID: "1d9c1099-81ce-4cb8-8bb1-52bd5c386285"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 19:01:47.052347 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.052240 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1d9c1099-81ce-4cb8-8bb1-52bd5c386285" (UID: "1d9c1099-81ce-4cb8-8bb1-52bd5c386285"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:01:47.151153 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.151116 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-istio-podinfo\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:01:47.151153 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.151146 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-istio-token\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:01:47.151153 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.151156 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-istio-data\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:01:47.151153 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.151164 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6wbpz\" (UniqueName: \"kubernetes.io/projected/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-kube-api-access-6wbpz\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:01:47.151427 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.151174 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d9c1099-81ce-4cb8-8bb1-52bd5c386285-istio-envoy\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:01:47.195102 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.195065 2582 generic.go:356] "Generic (PLEG): container finished" podID="1d9c1099-81ce-4cb8-8bb1-52bd5c386285" containerID="d565d9531abb7fed5be90cee5c82fd15e9837d8954475b3af75ee9b9a949a4f1" exitCode=0 Jul 13 19:01:47.195260 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.195116 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" Jul 13 19:01:47.195260 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.195118 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" event={"ID":"1d9c1099-81ce-4cb8-8bb1-52bd5c386285","Type":"ContainerDied","Data":"d565d9531abb7fed5be90cee5c82fd15e9837d8954475b3af75ee9b9a949a4f1"} Jul 13 19:01:47.195260 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.195171 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj" event={"ID":"1d9c1099-81ce-4cb8-8bb1-52bd5c386285","Type":"ContainerDied","Data":"49b939076cee8b8d341bebc7dccf54945ea155b31fa269e75115eba6a39f3928"} Jul 13 19:01:47.195260 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.195195 2582 scope.go:117] "RemoveContainer" containerID="d565d9531abb7fed5be90cee5c82fd15e9837d8954475b3af75ee9b9a949a4f1" Jul 13 19:01:47.205095 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.205076 2582 scope.go:117] "RemoveContainer" containerID="566ebc951b26d592b775854e22d48e86ecc84f0315fb805ca20a15326b7fe831" Jul 13 19:01:47.214270 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.214245 2582 scope.go:117] "RemoveContainer" containerID="d565d9531abb7fed5be90cee5c82fd15e9837d8954475b3af75ee9b9a949a4f1" Jul 13 19:01:47.214544 ip-10-0-129-191 kubenswrapper[2582]: E0713 19:01:47.214524 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d565d9531abb7fed5be90cee5c82fd15e9837d8954475b3af75ee9b9a949a4f1\": container with ID starting with d565d9531abb7fed5be90cee5c82fd15e9837d8954475b3af75ee9b9a949a4f1 not found: ID does not exist" containerID="d565d9531abb7fed5be90cee5c82fd15e9837d8954475b3af75ee9b9a949a4f1" Jul 13 19:01:47.214617 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.214555 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d565d9531abb7fed5be90cee5c82fd15e9837d8954475b3af75ee9b9a949a4f1"} err="failed to get container status \"d565d9531abb7fed5be90cee5c82fd15e9837d8954475b3af75ee9b9a949a4f1\": rpc error: code = NotFound desc = could not find container \"d565d9531abb7fed5be90cee5c82fd15e9837d8954475b3af75ee9b9a949a4f1\": container with ID starting with d565d9531abb7fed5be90cee5c82fd15e9837d8954475b3af75ee9b9a949a4f1 not found: ID does not exist" Jul 13 19:01:47.214617 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.214573 2582 scope.go:117] "RemoveContainer" containerID="566ebc951b26d592b775854e22d48e86ecc84f0315fb805ca20a15326b7fe831" Jul 13 19:01:47.214843 ip-10-0-129-191 kubenswrapper[2582]: E0713 19:01:47.214826 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"566ebc951b26d592b775854e22d48e86ecc84f0315fb805ca20a15326b7fe831\": container with ID starting with 566ebc951b26d592b775854e22d48e86ecc84f0315fb805ca20a15326b7fe831 not found: ID does not exist" containerID="566ebc951b26d592b775854e22d48e86ecc84f0315fb805ca20a15326b7fe831" Jul 13 19:01:47.214891 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.214850 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"566ebc951b26d592b775854e22d48e86ecc84f0315fb805ca20a15326b7fe831"} err="failed to get container status \"566ebc951b26d592b775854e22d48e86ecc84f0315fb805ca20a15326b7fe831\": rpc error: code = NotFound desc = could not find container \"566ebc951b26d592b775854e22d48e86ecc84f0315fb805ca20a15326b7fe831\": container with ID starting with 566ebc951b26d592b775854e22d48e86ecc84f0315fb805ca20a15326b7fe831 not found: ID does not exist" Jul 13 19:01:47.223207 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.223134 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj"] Jul 13 19:01:47.232966 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.232941 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-78327326-gw-openshift-default-6fdbc9f647-4w9nj"] Jul 13 19:01:47.783087 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:47.783052 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:01:48.026789 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:48.026741 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:01:48.281007 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:48.280971 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d9c1099-81ce-4cb8-8bb1-52bd5c386285" path="/var/lib/kubelet/pods/1d9c1099-81ce-4cb8-8bb1-52bd5c386285/volumes" Jul 13 19:01:48.702776 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:48.702738 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:01:54.974420 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:54.974374 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:01:55.110584 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:55.110547 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:01:55.284821 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:55.284731 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:01:55.376961 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:55.376928 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:01:56.447543 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:56.447510 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:01:56.758370 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:56.758279 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:01:56.865952 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:56.865914 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:01:57.330347 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:57.330309 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:01:57.430859 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:57.430822 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:01:58.667108 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:01:58.667075 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:02:01.662761 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:01.662727 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-258c642c-m5c6r"] Jul 13 19:02:01.663190 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:01.663178 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d9c1099-81ce-4cb8-8bb1-52bd5c386285" containerName="istio-proxy" Jul 13 19:02:01.663234 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:01.663192 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d9c1099-81ce-4cb8-8bb1-52bd5c386285" containerName="istio-proxy" Jul 13 19:02:01.663279 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:01.663269 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d9c1099-81ce-4cb8-8bb1-52bd5c386285" containerName="istio-proxy" Jul 13 19:02:01.666504 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:01.666488 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-258c642c-m5c6r" Jul 13 19:02:01.669510 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:01.669492 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lkrf8\"" Jul 13 19:02:01.676137 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:01.676110 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-258c642c-m5c6r"] Jul 13 19:02:01.691163 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:01.691132 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3009142c-c0ac-427b-8180-bbb3dfb21563-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-258c642c-m5c6r\" (UID: \"3009142c-c0ac-427b-8180-bbb3dfb21563\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-258c642c-m5c6r" Jul 13 19:02:01.792226 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:01.792175 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3009142c-c0ac-427b-8180-bbb3dfb21563-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-258c642c-m5c6r\" (UID: \"3009142c-c0ac-427b-8180-bbb3dfb21563\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-258c642c-m5c6r" Jul 13 19:02:01.792850 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:01.792829 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3009142c-c0ac-427b-8180-bbb3dfb21563-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-258c642c-m5c6r\" (UID: \"3009142c-c0ac-427b-8180-bbb3dfb21563\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-258c642c-m5c6r" Jul 13 19:02:01.977680 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:01.977628 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-258c642c-m5c6r" Jul 13 19:02:02.117203 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:02.117172 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-258c642c-m5c6r"] Jul 13 19:02:02.119204 ip-10-0-129-191 kubenswrapper[2582]: W0713 19:02:02.119166 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3009142c_c0ac_427b_8180_bbb3dfb21563.slice/crio-e75d152a6a8ce95424901a744aa922bb7650bd25bb2d7ee6df02e341f928f917 WatchSource:0}: Error finding container e75d152a6a8ce95424901a744aa922bb7650bd25bb2d7ee6df02e341f928f917: Status 404 returned error can't find the container with id e75d152a6a8ce95424901a744aa922bb7650bd25bb2d7ee6df02e341f928f917 Jul 13 19:02:02.280772 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:02.280692 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-258c642c-m5c6r" event={"ID":"3009142c-c0ac-427b-8180-bbb3dfb21563","Type":"ContainerStarted","Data":"e75d152a6a8ce95424901a744aa922bb7650bd25bb2d7ee6df02e341f928f917"} Jul 13 19:02:03.283307 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:03.283272 2582 generic.go:356] "Generic (PLEG): container finished" podID="3009142c-c0ac-427b-8180-bbb3dfb21563" containerID="16d0112634fd74a64d2fa918ddb4ea1f00141b8e772d2269c8c2642021b319b7" exitCode=7 Jul 13 19:02:03.283766 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:03.283356 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-258c642c-m5c6r" event={"ID":"3009142c-c0ac-427b-8180-bbb3dfb21563","Type":"ContainerDied","Data":"16d0112634fd74a64d2fa918ddb4ea1f00141b8e772d2269c8c2642021b319b7"} Jul 13 19:02:03.283766 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:03.283571 2582 scope.go:117] "RemoveContainer" containerID="16d0112634fd74a64d2fa918ddb4ea1f00141b8e772d2269c8c2642021b319b7" Jul 13 19:02:04.289661 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:04.289628 2582 generic.go:356] "Generic (PLEG): container finished" podID="3009142c-c0ac-427b-8180-bbb3dfb21563" containerID="7a1fbd11311be3afa56432ebc14997ac6ff5567e824bf1a4a030f37ee8f0433c" exitCode=7 Jul 13 19:02:04.290133 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:04.289711 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-258c642c-m5c6r" event={"ID":"3009142c-c0ac-427b-8180-bbb3dfb21563","Type":"ContainerDied","Data":"7a1fbd11311be3afa56432ebc14997ac6ff5567e824bf1a4a030f37ee8f0433c"} Jul 13 19:02:04.290133 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:04.289768 2582 scope.go:117] "RemoveContainer" containerID="16d0112634fd74a64d2fa918ddb4ea1f00141b8e772d2269c8c2642021b319b7" Jul 13 19:02:04.290133 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:04.290029 2582 scope.go:117] "RemoveContainer" containerID="7a1fbd11311be3afa56432ebc14997ac6ff5567e824bf1a4a030f37ee8f0433c" Jul 13 19:02:04.290331 ip-10-0-129-191 kubenswrapper[2582]: E0713 19:02:04.290310 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-258c642c-m5c6r_odh-ai-gateway-infra(3009142c-c0ac-427b-8180-bbb3dfb21563)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-258c642c-m5c6r" podUID="3009142c-c0ac-427b-8180-bbb3dfb21563" Jul 13 19:02:05.296297 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:05.296272 2582 scope.go:117] "RemoveContainer" containerID="7a1fbd11311be3afa56432ebc14997ac6ff5567e824bf1a4a030f37ee8f0433c" Jul 13 19:02:05.296715 ip-10-0-129-191 kubenswrapper[2582]: E0713 19:02:05.296439 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-258c642c-m5c6r_odh-ai-gateway-infra(3009142c-c0ac-427b-8180-bbb3dfb21563)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-258c642c-m5c6r" podUID="3009142c-c0ac-427b-8180-bbb3dfb21563" Jul 13 19:02:16.274929 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:16.274838 2582 scope.go:117] "RemoveContainer" containerID="7a1fbd11311be3afa56432ebc14997ac6ff5567e824bf1a4a030f37ee8f0433c" Jul 13 19:02:17.358928 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:17.358846 2582 generic.go:356] "Generic (PLEG): container finished" podID="3009142c-c0ac-427b-8180-bbb3dfb21563" containerID="5c08201fbf3b0b86dc24e405958a16df51293a8f2e5f3bbbc5200cf16657c9a6" exitCode=0 Jul 13 19:02:17.359353 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:17.358928 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-258c642c-m5c6r" event={"ID":"3009142c-c0ac-427b-8180-bbb3dfb21563","Type":"ContainerDied","Data":"5c08201fbf3b0b86dc24e405958a16df51293a8f2e5f3bbbc5200cf16657c9a6"} Jul 13 19:02:17.359353 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:17.358984 2582 scope.go:117] "RemoveContainer" containerID="7a1fbd11311be3afa56432ebc14997ac6ff5567e824bf1a4a030f37ee8f0433c" Jul 13 19:02:17.831406 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:17.831375 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:02:17.931046 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:17.931014 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:02:18.397031 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:18.397002 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-258c642c-m5c6r"] Jul 13 19:02:18.500074 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:18.500048 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-258c642c-m5c6r" Jul 13 19:02:18.659784 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:18.659646 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3009142c-c0ac-427b-8180-bbb3dfb21563-maas-api-service-ca\") pod \"3009142c-c0ac-427b-8180-bbb3dfb21563\" (UID: \"3009142c-c0ac-427b-8180-bbb3dfb21563\") " Jul 13 19:02:18.660058 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:18.660038 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3009142c-c0ac-427b-8180-bbb3dfb21563-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3009142c-c0ac-427b-8180-bbb3dfb21563" (UID: "3009142c-c0ac-427b-8180-bbb3dfb21563"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:02:18.760971 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:18.760937 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3009142c-c0ac-427b-8180-bbb3dfb21563-maas-api-service-ca\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:02:18.838555 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:18.838516 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:02:18.939398 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:18.939304 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:02:19.372161 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:19.372137 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-258c642c-m5c6r" Jul 13 19:02:19.372161 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:19.372156 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-258c642c-m5c6r" event={"ID":"3009142c-c0ac-427b-8180-bbb3dfb21563","Type":"ContainerDied","Data":"e75d152a6a8ce95424901a744aa922bb7650bd25bb2d7ee6df02e341f928f917"} Jul 13 19:02:19.372392 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:19.372202 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e75d152a6a8ce95424901a744aa922bb7650bd25bb2d7ee6df02e341f928f917" Jul 13 19:02:19.411721 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:19.411693 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-258c642c-m5c6r"] Jul 13 19:02:19.416157 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:19.416131 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-258c642c-m5c6r"] Jul 13 19:02:20.281399 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:20.281367 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3009142c-c0ac-427b-8180-bbb3dfb21563" path="/var/lib/kubelet/pods/3009142c-c0ac-427b-8180-bbb3dfb21563/volumes" Jul 13 19:02:24.277436 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:24.277410 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8l7w6_b14ef695-c589-4fca-bd12-2e951f04e466/ovn-acl-logging/0.log" Jul 13 19:02:24.282737 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:24.282710 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8l7w6_b14ef695-c589-4fca-bd12-2e951f04e466/ovn-acl-logging/0.log" Jul 13 19:02:51.706364 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:51.706329 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-258c642c-7stgk"] Jul 13 19:02:51.706818 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:51.706736 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3009142c-c0ac-427b-8180-bbb3dfb21563" containerName="revoke-keys" Jul 13 19:02:51.706818 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:51.706747 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3009142c-c0ac-427b-8180-bbb3dfb21563" containerName="revoke-keys" Jul 13 19:02:51.706818 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:51.706755 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3009142c-c0ac-427b-8180-bbb3dfb21563" containerName="revoke-keys" Jul 13 19:02:51.706818 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:51.706761 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3009142c-c0ac-427b-8180-bbb3dfb21563" containerName="revoke-keys" Jul 13 19:02:51.706818 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:51.706788 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d9c1099-81ce-4cb8-8bb1-52bd5c386285" containerName="istio-proxy" Jul 13 19:02:51.706818 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:51.706794 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d9c1099-81ce-4cb8-8bb1-52bd5c386285" containerName="istio-proxy" Jul 13 19:02:51.707016 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:51.706852 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3009142c-c0ac-427b-8180-bbb3dfb21563" containerName="revoke-keys" Jul 13 19:02:51.707016 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:51.706859 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3009142c-c0ac-427b-8180-bbb3dfb21563" containerName="revoke-keys" Jul 13 19:02:51.707016 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:51.706866 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d9c1099-81ce-4cb8-8bb1-52bd5c386285" containerName="istio-proxy" Jul 13 19:02:51.710154 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:51.710137 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-258c642c-7stgk" Jul 13 19:02:51.712833 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:51.712815 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lkrf8\"" Jul 13 19:02:51.718201 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:51.718163 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-258c642c-7stgk"] Jul 13 19:02:51.780978 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:51.780922 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6eb5f51c-4adf-4c95-b794-ba8ef7e3ab7c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-258c642c-7stgk\" (UID: \"6eb5f51c-4adf-4c95-b794-ba8ef7e3ab7c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-258c642c-7stgk" Jul 13 19:02:51.882418 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:51.882380 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6eb5f51c-4adf-4c95-b794-ba8ef7e3ab7c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-258c642c-7stgk\" (UID: \"6eb5f51c-4adf-4c95-b794-ba8ef7e3ab7c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-258c642c-7stgk" Jul 13 19:02:51.883091 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:51.883065 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6eb5f51c-4adf-4c95-b794-ba8ef7e3ab7c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-258c642c-7stgk\" (UID: \"6eb5f51c-4adf-4c95-b794-ba8ef7e3ab7c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-258c642c-7stgk" Jul 13 19:02:52.022408 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:52.022305 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-258c642c-7stgk" Jul 13 19:02:52.156904 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:52.156879 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-258c642c-7stgk"] Jul 13 19:02:52.159002 ip-10-0-129-191 kubenswrapper[2582]: W0713 19:02:52.158975 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6eb5f51c_4adf_4c95_b794_ba8ef7e3ab7c.slice/crio-3bd6fde5dcc4de454c537f0a14d20d72c6bcdb1fc2603193d4c7ff1b0393400b WatchSource:0}: Error finding container 3bd6fde5dcc4de454c537f0a14d20d72c6bcdb1fc2603193d4c7ff1b0393400b: Status 404 returned error can't find the container with id 3bd6fde5dcc4de454c537f0a14d20d72c6bcdb1fc2603193d4c7ff1b0393400b Jul 13 19:02:52.549017 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:52.548979 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-258c642c-7stgk" event={"ID":"6eb5f51c-4adf-4c95-b794-ba8ef7e3ab7c","Type":"ContainerStarted","Data":"a0f5fa880e7233a559d3065213d0122de767001e7464b3cef801cf5428d2302a"} Jul 13 19:02:52.549193 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:52.549034 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-258c642c-7stgk" event={"ID":"6eb5f51c-4adf-4c95-b794-ba8ef7e3ab7c","Type":"ContainerStarted","Data":"3bd6fde5dcc4de454c537f0a14d20d72c6bcdb1fc2603193d4c7ff1b0393400b"} Jul 13 19:02:52.565160 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:52.565113 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-258c642c-7stgk" podStartSLOduration=1.5650981800000001 podStartE2EDuration="1.56509818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:02:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:02:52.564028856 +0000 UTC m=+1828.847457499" watchObservedRunningTime="2026-07-13 19:02:52.56509818 +0000 UTC m=+1828.848526782" Jul 13 19:02:53.555634 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:53.555584 2582 generic.go:356] "Generic (PLEG): container finished" podID="6eb5f51c-4adf-4c95-b794-ba8ef7e3ab7c" containerID="a0f5fa880e7233a559d3065213d0122de767001e7464b3cef801cf5428d2302a" exitCode=0 Jul 13 19:02:53.556037 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:53.555674 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-258c642c-7stgk" event={"ID":"6eb5f51c-4adf-4c95-b794-ba8ef7e3ab7c","Type":"ContainerDied","Data":"a0f5fa880e7233a559d3065213d0122de767001e7464b3cef801cf5428d2302a"} Jul 13 19:02:54.721574 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:54.721542 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-258c642c-7stgk" Jul 13 19:02:54.812481 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:54.812383 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6eb5f51c-4adf-4c95-b794-ba8ef7e3ab7c-maas-api-service-ca\") pod \"6eb5f51c-4adf-4c95-b794-ba8ef7e3ab7c\" (UID: \"6eb5f51c-4adf-4c95-b794-ba8ef7e3ab7c\") " Jul 13 19:02:54.812873 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:54.812846 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6eb5f51c-4adf-4c95-b794-ba8ef7e3ab7c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6eb5f51c-4adf-4c95-b794-ba8ef7e3ab7c" (UID: "6eb5f51c-4adf-4c95-b794-ba8ef7e3ab7c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:02:54.913708 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:54.913660 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6eb5f51c-4adf-4c95-b794-ba8ef7e3ab7c-maas-api-service-ca\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:02:55.567509 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:55.567483 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-258c642c-7stgk" Jul 13 19:02:55.567509 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:55.567508 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-258c642c-7stgk" event={"ID":"6eb5f51c-4adf-4c95-b794-ba8ef7e3ab7c","Type":"ContainerDied","Data":"3bd6fde5dcc4de454c537f0a14d20d72c6bcdb1fc2603193d4c7ff1b0393400b"} Jul 13 19:02:55.567740 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:02:55.567548 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3bd6fde5dcc4de454c537f0a14d20d72c6bcdb1fc2603193d4c7ff1b0393400b" Jul 13 19:03:02.212466 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:02.212424 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:06.794128 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:06.794098 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:13.644840 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:13.644802 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:13.739989 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:13.739947 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:17.928008 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:17.927974 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:18.237967 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.237934 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4"] Jul 13 19:03:18.238366 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.238350 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3009142c-c0ac-427b-8180-bbb3dfb21563" containerName="revoke-keys" Jul 13 19:03:18.238366 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.238366 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3009142c-c0ac-427b-8180-bbb3dfb21563" containerName="revoke-keys" Jul 13 19:03:18.238483 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.238376 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6eb5f51c-4adf-4c95-b794-ba8ef7e3ab7c" containerName="revoke-keys" Jul 13 19:03:18.238483 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.238382 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6eb5f51c-4adf-4c95-b794-ba8ef7e3ab7c" containerName="revoke-keys" Jul 13 19:03:18.238483 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.238455 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6eb5f51c-4adf-4c95-b794-ba8ef7e3ab7c" containerName="revoke-keys" Jul 13 19:03:18.238483 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.238464 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3009142c-c0ac-427b-8180-bbb3dfb21563" containerName="revoke-keys" Jul 13 19:03:18.241654 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.241634 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:18.244559 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.244537 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 13 19:03:18.244787 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.244767 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-695ca96e-openshift-default-dockercfg-glgjf\"" Jul 13 19:03:18.256889 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.256860 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4"] Jul 13 19:03:18.272059 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.272029 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:18.346241 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.346211 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-istio-token\") pod \"e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4\" (UID: \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\") " pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:18.346391 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.346249 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-istio-data\") pod \"e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4\" (UID: \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\") " pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:18.346391 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.346275 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-workload-socket\") pod \"e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4\" (UID: \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\") " pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:18.346391 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.346352 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-credential-socket\") pod \"e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4\" (UID: \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\") " pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:18.346509 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.346410 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-workload-certs\") pod \"e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4\" (UID: \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\") " pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:18.346509 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.346443 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-istiod-ca-cert\") pod \"e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4\" (UID: \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\") " pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:18.346649 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.346507 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-istio-envoy\") pod \"e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4\" (UID: \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\") " pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:18.346649 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.346574 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zknq4\" (UniqueName: \"kubernetes.io/projected/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-kube-api-access-zknq4\") pod \"e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4\" (UID: \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\") " pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:18.346649 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.346634 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-istio-podinfo\") pod \"e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4\" (UID: \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\") " pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:18.447138 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.447101 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-istiod-ca-cert\") pod \"e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4\" (UID: \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\") " pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:18.447332 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.447183 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-istio-envoy\") pod \"e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4\" (UID: \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\") " pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:18.447332 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.447223 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zknq4\" (UniqueName: \"kubernetes.io/projected/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-kube-api-access-zknq4\") pod \"e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4\" (UID: \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\") " pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:18.447332 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.447249 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-istio-podinfo\") pod \"e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4\" (UID: \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\") " pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:18.447332 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.447291 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-istio-token\") pod \"e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4\" (UID: \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\") " pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:18.447332 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.447320 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-istio-data\") pod \"e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4\" (UID: \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\") " pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:18.447660 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.447344 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-workload-socket\") pod \"e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4\" (UID: \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\") " pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:18.447660 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.447372 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-credential-socket\") pod \"e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4\" (UID: \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\") " pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:18.447660 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.447434 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-workload-certs\") pod \"e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4\" (UID: \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\") " pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:18.447830 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.447742 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-istio-data\") pod \"e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4\" (UID: \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\") " pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:18.447830 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.447804 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-workload-certs\") pod \"e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4\" (UID: \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\") " pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:18.447923 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.447853 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-workload-socket\") pod \"e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4\" (UID: \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\") " pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:18.447965 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.447926 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-istiod-ca-cert\") pod \"e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4\" (UID: \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\") " pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:18.447965 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.447935 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-credential-socket\") pod \"e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4\" (UID: \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\") " pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:18.450122 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.450103 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-istio-envoy\") pod \"e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4\" (UID: \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\") " pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:18.450432 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.450409 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-istio-podinfo\") pod \"e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4\" (UID: \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\") " pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:18.455845 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.455827 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-istio-token\") pod \"e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4\" (UID: \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\") " pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:18.456220 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.456198 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zknq4\" (UniqueName: \"kubernetes.io/projected/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-kube-api-access-zknq4\") pod \"e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4\" (UID: \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\") " pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:18.552434 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.552358 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:18.695543 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.695490 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4"] Jul 13 19:03:18.698053 ip-10-0-129-191 kubenswrapper[2582]: W0713 19:03:18.698017 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3663330_e6bc_4a8a_b39e_0c3fde07cc8f.slice/crio-da0c42a2d5364d2052767a0b0af6957ee74eb1243a1978a4b28451d9773fa179 WatchSource:0}: Error finding container da0c42a2d5364d2052767a0b0af6957ee74eb1243a1978a4b28451d9773fa179: Status 404 returned error can't find the container with id da0c42a2d5364d2052767a0b0af6957ee74eb1243a1978a4b28451d9773fa179 Jul 13 19:03:18.700300 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.700270 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 19:03:18.700403 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.700340 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 19:03:18.700403 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.700368 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 19:03:18.706856 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:18.706831 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" event={"ID":"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f","Type":"ContainerStarted","Data":"da0c42a2d5364d2052767a0b0af6957ee74eb1243a1978a4b28451d9773fa179"} Jul 13 19:03:19.441901 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:19.441864 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:19.544474 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:19.544446 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:19.713465 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:19.713435 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" event={"ID":"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f","Type":"ContainerStarted","Data":"0e042327ad27c668584245d49eaa1e84255b1d95632ee5deaadfde15ca4e3230"} Jul 13 19:03:19.717104 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:19.717074 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:19.736478 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:19.736432 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" podStartSLOduration=1.7363931529999999 podStartE2EDuration="1.736393153s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:03:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:03:19.736221295 +0000 UTC m=+1856.019649926" watchObservedRunningTime="2026-07-13 19:03:19.736393153 +0000 UTC m=+1856.019821755" Jul 13 19:03:20.120706 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:20.120589 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:20.223420 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:20.223389 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:20.324925 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:20.324889 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:20.553352 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:20.553327 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:20.558400 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:20.558378 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:20.718961 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:20.718933 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:20.719827 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:20.719808 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:31.222098 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:31.222059 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:31.326178 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:31.326139 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:34.675958 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:34.675916 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:34.779656 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:34.779589 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:35.720179 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:35.720144 2582 patch_prober.go:28] interesting pod/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 13 19:03:35.720623 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:35.720235 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" podUID="b3663330-e6bc-4a8a-b39e-0c3fde07cc8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 13 19:03:35.805521 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:35.805485 2582 generic.go:356] "Generic (PLEG): container finished" podID="b3663330-e6bc-4a8a-b39e-0c3fde07cc8f" containerID="0e042327ad27c668584245d49eaa1e84255b1d95632ee5deaadfde15ca4e3230" exitCode=0 Jul 13 19:03:35.805704 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:35.805552 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" event={"ID":"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f","Type":"ContainerDied","Data":"0e042327ad27c668584245d49eaa1e84255b1d95632ee5deaadfde15ca4e3230"} Jul 13 19:03:35.805996 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:35.805982 2582 scope.go:117] "RemoveContainer" containerID="0e042327ad27c668584245d49eaa1e84255b1d95632ee5deaadfde15ca4e3230" Jul 13 19:03:35.806968 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:35.806946 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 19:03:35.807047 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:35.807009 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 19:03:35.807047 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:35.807042 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 19:03:36.813235 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:36.813199 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" event={"ID":"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f","Type":"ContainerStarted","Data":"676d51df55ad1329fe45514dcf1f21251e963b4528152d239a2f2056fca08c99"} Jul 13 19:03:37.553806 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:37.553776 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:37.554208 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:37.554106 2582 patch_prober.go:28] interesting pod/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 13 19:03:37.554410 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:37.554384 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" podUID="b3663330-e6bc-4a8a-b39e-0c3fde07cc8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 13 19:03:38.554193 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:38.554152 2582 patch_prober.go:28] interesting pod/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 13 19:03:38.554770 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:38.554726 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" podUID="b3663330-e6bc-4a8a-b39e-0c3fde07cc8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 13 19:03:38.993150 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:38.993112 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:39.553810 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:39.553765 2582 patch_prober.go:28] interesting pod/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 13 19:03:39.554028 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:39.553832 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" podUID="b3663330-e6bc-4a8a-b39e-0c3fde07cc8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 13 19:03:40.556628 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:40.553498 2582 patch_prober.go:28] interesting pod/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 13 19:03:40.556628 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:40.553574 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" podUID="b3663330-e6bc-4a8a-b39e-0c3fde07cc8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 13 19:03:41.557309 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:41.557283 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:41.557729 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:41.557660 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:41.558397 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:41.558381 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:45.095950 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:45.095877 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:45.158676 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:45.158638 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:49.147289 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:49.147252 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4"] Jul 13 19:03:49.147683 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:49.147478 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" podUID="b3663330-e6bc-4a8a-b39e-0c3fde07cc8f" containerName="istio-proxy" containerID="cri-o://676d51df55ad1329fe45514dcf1f21251e963b4528152d239a2f2056fca08c99" gracePeriod=30 Jul 13 19:03:49.171384 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:49.171355 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:49.728843 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:49.728806 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:49.827955 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:49.827916 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:49.935668 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:49.935597 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:51.435658 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:51.435623 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:54.413722 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.413653 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:54.512059 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.512026 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-istio-token\") pod \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\" (UID: \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\") " Jul 13 19:03:54.512231 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.512082 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-istio-podinfo\") pod \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\" (UID: \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\") " Jul 13 19:03:54.512231 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.512117 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-workload-certs\") pod \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\" (UID: \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\") " Jul 13 19:03:54.512231 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.512142 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-istio-envoy\") pod \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\" (UID: \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\") " Jul 13 19:03:54.512231 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.512168 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-credential-socket\") pod \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\" (UID: \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\") " Jul 13 19:03:54.512448 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.512239 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-istio-data\") pod \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\" (UID: \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\") " Jul 13 19:03:54.512448 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.512277 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-istiod-ca-cert\") pod \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\" (UID: \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\") " Jul 13 19:03:54.512448 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.512342 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-workload-socket\") pod \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\" (UID: \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\") " Jul 13 19:03:54.512448 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.512367 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zknq4\" (UniqueName: \"kubernetes.io/projected/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-kube-api-access-zknq4\") pod \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\" (UID: \"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f\") " Jul 13 19:03:54.512666 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.512557 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b3663330-e6bc-4a8a-b39e-0c3fde07cc8f" (UID: "b3663330-e6bc-4a8a-b39e-0c3fde07cc8f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:03:54.513049 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.512790 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-credential-socket\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:03:54.513049 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.512938 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b3663330-e6bc-4a8a-b39e-0c3fde07cc8f" (UID: "b3663330-e6bc-4a8a-b39e-0c3fde07cc8f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:03:54.513049 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.513015 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b3663330-e6bc-4a8a-b39e-0c3fde07cc8f" (UID: "b3663330-e6bc-4a8a-b39e-0c3fde07cc8f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:03:54.513318 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.513062 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b3663330-e6bc-4a8a-b39e-0c3fde07cc8f" (UID: "b3663330-e6bc-4a8a-b39e-0c3fde07cc8f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:03:54.513427 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.513404 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-istio-data" (OuterVolumeSpecName: "istio-data") pod "b3663330-e6bc-4a8a-b39e-0c3fde07cc8f" (UID: "b3663330-e6bc-4a8a-b39e-0c3fde07cc8f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:03:54.514317 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.514295 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-istio-token" (OuterVolumeSpecName: "istio-token") pod "b3663330-e6bc-4a8a-b39e-0c3fde07cc8f" (UID: "b3663330-e6bc-4a8a-b39e-0c3fde07cc8f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:03:54.515360 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.515255 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-kube-api-access-zknq4" (OuterVolumeSpecName: "kube-api-access-zknq4") pod "b3663330-e6bc-4a8a-b39e-0c3fde07cc8f" (UID: "b3663330-e6bc-4a8a-b39e-0c3fde07cc8f"). InnerVolumeSpecName "kube-api-access-zknq4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:03:54.515360 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.515338 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b3663330-e6bc-4a8a-b39e-0c3fde07cc8f" (UID: "b3663330-e6bc-4a8a-b39e-0c3fde07cc8f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:03:54.515721 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.515705 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b3663330-e6bc-4a8a-b39e-0c3fde07cc8f" (UID: "b3663330-e6bc-4a8a-b39e-0c3fde07cc8f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 19:03:54.613279 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.613240 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-istio-token\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:03:54.613279 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.613270 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-istio-podinfo\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:03:54.613279 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.613283 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-workload-certs\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:03:54.613515 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.613294 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-istio-envoy\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:03:54.613515 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.613306 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-istio-data\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:03:54.613515 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.613316 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-istiod-ca-cert\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:03:54.613515 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.613326 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-workload-socket\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:03:54.613515 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.613337 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zknq4\" (UniqueName: \"kubernetes.io/projected/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f-kube-api-access-zknq4\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:03:54.909797 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.909758 2582 generic.go:356] "Generic (PLEG): container finished" podID="b3663330-e6bc-4a8a-b39e-0c3fde07cc8f" containerID="676d51df55ad1329fe45514dcf1f21251e963b4528152d239a2f2056fca08c99" exitCode=0 Jul 13 19:03:54.910017 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.909817 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" Jul 13 19:03:54.910017 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.909844 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" event={"ID":"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f","Type":"ContainerDied","Data":"676d51df55ad1329fe45514dcf1f21251e963b4528152d239a2f2056fca08c99"} Jul 13 19:03:54.910017 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.909926 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4" event={"ID":"b3663330-e6bc-4a8a-b39e-0c3fde07cc8f","Type":"ContainerDied","Data":"da0c42a2d5364d2052767a0b0af6957ee74eb1243a1978a4b28451d9773fa179"} Jul 13 19:03:54.910017 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.909951 2582 scope.go:117] "RemoveContainer" containerID="676d51df55ad1329fe45514dcf1f21251e963b4528152d239a2f2056fca08c99" Jul 13 19:03:54.920091 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.920070 2582 scope.go:117] "RemoveContainer" containerID="0e042327ad27c668584245d49eaa1e84255b1d95632ee5deaadfde15ca4e3230" Jul 13 19:03:54.928491 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.928471 2582 scope.go:117] "RemoveContainer" containerID="676d51df55ad1329fe45514dcf1f21251e963b4528152d239a2f2056fca08c99" Jul 13 19:03:54.928773 ip-10-0-129-191 kubenswrapper[2582]: E0713 19:03:54.928755 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"676d51df55ad1329fe45514dcf1f21251e963b4528152d239a2f2056fca08c99\": container with ID starting with 676d51df55ad1329fe45514dcf1f21251e963b4528152d239a2f2056fca08c99 not found: ID does not exist" containerID="676d51df55ad1329fe45514dcf1f21251e963b4528152d239a2f2056fca08c99" Jul 13 19:03:54.928857 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.928781 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"676d51df55ad1329fe45514dcf1f21251e963b4528152d239a2f2056fca08c99"} err="failed to get container status \"676d51df55ad1329fe45514dcf1f21251e963b4528152d239a2f2056fca08c99\": rpc error: code = NotFound desc = could not find container \"676d51df55ad1329fe45514dcf1f21251e963b4528152d239a2f2056fca08c99\": container with ID starting with 676d51df55ad1329fe45514dcf1f21251e963b4528152d239a2f2056fca08c99 not found: ID does not exist" Jul 13 19:03:54.928857 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.928800 2582 scope.go:117] "RemoveContainer" containerID="0e042327ad27c668584245d49eaa1e84255b1d95632ee5deaadfde15ca4e3230" Jul 13 19:03:54.929055 ip-10-0-129-191 kubenswrapper[2582]: E0713 19:03:54.929038 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e042327ad27c668584245d49eaa1e84255b1d95632ee5deaadfde15ca4e3230\": container with ID starting with 0e042327ad27c668584245d49eaa1e84255b1d95632ee5deaadfde15ca4e3230 not found: ID does not exist" containerID="0e042327ad27c668584245d49eaa1e84255b1d95632ee5deaadfde15ca4e3230" Jul 13 19:03:54.929111 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.929061 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e042327ad27c668584245d49eaa1e84255b1d95632ee5deaadfde15ca4e3230"} err="failed to get container status \"0e042327ad27c668584245d49eaa1e84255b1d95632ee5deaadfde15ca4e3230\": rpc error: code = NotFound desc = could not find container \"0e042327ad27c668584245d49eaa1e84255b1d95632ee5deaadfde15ca4e3230\": container with ID starting with 0e042327ad27c668584245d49eaa1e84255b1d95632ee5deaadfde15ca4e3230 not found: ID does not exist" Jul 13 19:03:54.935933 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.935910 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4"] Jul 13 19:03:54.940353 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:54.940329 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-695ca96e-openshift-default-654cc969bc-hffz4"] Jul 13 19:03:55.804265 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:55.804231 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:56.282851 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:56.282814 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3663330-e6bc-4a8a-b39e-0c3fde07cc8f" path="/var/lib/kubelet/pods/b3663330-e6bc-4a8a-b39e-0c3fde07cc8f/volumes" Jul 13 19:03:57.656366 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:57.656332 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:57.767323 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:57.767289 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:57.972504 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:57.972474 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:58.288671 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:58.288559 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:58.416775 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:03:58.416747 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:04:05.885946 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:04:05.885911 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:04:24.335326 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:04:24.335267 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:04:30.612303 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:04:30.612268 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:04:30.688510 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:04:30.688469 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:04:30.791409 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:04:30.791377 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:04:46.684103 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:04:46.684064 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:04:52.927318 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:04:52.927281 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:04:55.752146 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:04:55.751972 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:03.653355 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:05:03.653323 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:04.458389 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:05:04.458353 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:10.425112 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:05:10.425077 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:10.582717 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:05:10.582679 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:10.717414 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:05:10.717379 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:10.835666 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:05:10.835630 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:11.564884 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:05:11.564851 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:11.720786 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:05:11.720752 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:13.491813 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:05:13.491767 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:19.722450 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:05:19.722360 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:19.826332 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:05:19.826300 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:20.468812 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:05:20.468767 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:26.936037 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:05:26.936002 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:27.034694 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:05:27.034664 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:27.177463 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:05:27.177427 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:27.817870 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:05:27.817841 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:28.144132 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:05:28.144040 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:30.521580 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:05:30.521547 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:30.624242 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:05:30.624211 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:30.761111 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:05:30.761080 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:31.082364 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:05:31.082325 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:31.353123 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:05:31.353025 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:31.455897 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:05:31.455870 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:35.059209 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:05:35.059168 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:35.791967 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:05:35.791913 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:35.886234 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:05:35.886200 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:36.365125 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:05:36.365084 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:42.495307 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:05:42.495268 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:42.595757 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:05:42.595719 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:53.407968 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:05:53.407927 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:58.016146 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:05:58.016096 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:04.397854 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:06:04.397813 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:04.473680 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:06:04.473637 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:08.853979 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:06:08.853943 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:08.955071 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:06:08.955039 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:09.063783 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:06:09.063739 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:09.157793 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:06:09.157709 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:09.289477 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:06:09.289436 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:15.057037 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:06:15.057003 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:15.130059 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:06:15.130022 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:15.214042 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:06:15.213999 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:15.432087 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:06:15.431989 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:21.444430 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:06:21.444389 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:21.508871 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:06:21.508843 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:24.354763 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:06:24.354735 2582 scope.go:117] "RemoveContainer" containerID="accd7af7c738f782ff8cb81b770684db79d3901ff0086e272c379ba42c3b5ef5" Jul 13 19:06:24.364175 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:06:24.364153 2582 scope.go:117] "RemoveContainer" containerID="b4353009094da740f407e0b0246595887f473998bcb4af97e1dab4db54be9768" Jul 13 19:06:28.259449 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:06:28.259409 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:28.569972 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:06:28.569891 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:28.744333 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:06:28.744303 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:31.888973 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:06:31.888928 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:31.981121 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:06:31.981084 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:32.247463 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:06:32.247432 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:32.526790 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:06:32.526701 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:32.739424 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:06:32.739389 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:32.908656 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:06:32.908545 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:36.292367 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:06:36.292337 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:55.794434 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:06:55.794337 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:59.224674 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:06:59.224633 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:59.325547 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:06:59.325517 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:07:18.617047 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:07:18.617008 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:07:18.718393 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:07:18.718362 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:07:24.310319 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:07:24.310280 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8l7w6_b14ef695-c589-4fca-bd12-2e951f04e466/ovn-acl-logging/0.log" Jul 13 19:07:24.315770 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:07:24.315750 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8l7w6_b14ef695-c589-4fca-bd12-2e951f04e466/ovn-acl-logging/0.log" Jul 13 19:07:25.839828 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:07:25.839789 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:07:25.940043 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:07:25.940001 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:07:35.698153 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:07:35.698117 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:07:40.248445 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:07:40.248410 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:07:45.279901 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:07:45.279858 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:07:48.457094 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:07:48.457057 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:08:21.988927 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:08:21.988838 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:08:22.977379 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:08:22.977340 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:08:24.398785 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:08:24.398755 2582 scope.go:117] "RemoveContainer" containerID="5c08201fbf3b0b86dc24e405958a16df51293a8f2e5f3bbbc5200cf16657c9a6" Jul 13 19:08:28.776445 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:08:28.776412 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:08:28.880006 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:08:28.879971 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:08:32.493044 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:08:32.493007 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:08:32.719358 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:08:32.719327 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:08:33.778661 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:08:33.778624 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:08:33.876888 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:08:33.876848 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:08:33.979425 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:08:33.979397 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:08:34.650841 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:08:34.650800 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:08:34.756750 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:08:34.756719 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:08:34.861661 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:08:34.861628 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:08:36.220964 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:08:36.220923 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:08:42.770695 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:08:42.770662 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:08:42.869804 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:08:42.869769 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:08:43.426988 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:08:43.426952 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:08:49.724038 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:08:49.724004 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:08:49.827012 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:08:49.826977 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:08:49.923626 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:08:49.923552 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:09:00.443943 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:09:00.443900 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:09:00.807634 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:09:00.807536 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:09:14.741986 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:09:14.741945 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:09:21.730804 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:09:21.730760 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:09:28.080914 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:09:28.080865 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:09:28.181047 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:09:28.181011 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:09:28.643825 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:09:28.643789 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:09:28.780043 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:09:28.780008 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:09:29.201012 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:09:29.200970 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:09:31.864071 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:09:31.864035 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:09:31.954308 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:09:31.954275 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:09:32.087364 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:09:32.087325 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:09:33.075198 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:09:33.075165 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:09:33.192020 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:09:33.191984 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:09:36.304395 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:09:36.304356 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:09:36.647649 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:09:36.647520 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:09:36.978947 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:09:36.978911 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:09:37.535376 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:09:37.535333 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:09:38.297302 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:09:38.297268 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:09:40.498247 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:09:40.498204 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:09:41.151320 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:09:41.151289 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:09:41.251935 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:09:41.251903 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:09:44.320295 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:09:44.320253 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:09:50.585841 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:09:50.585805 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:09:53.526000 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:09:53.525965 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:09:53.620884 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:09:53.620842 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:09:57.290171 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:09:57.290126 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:10:02.332120 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:10:02.332078 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:10:02.527720 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:10:02.527681 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:10:10.619143 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:10:10.619102 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:10:27.657079 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:10:27.657041 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:10:28.655077 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:10:28.655036 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:10:29.240231 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:10:29.240199 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:10:29.341648 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:10:29.341614 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:10:39.192450 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:10:39.192416 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:10:43.794856 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:10:43.794818 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:10:50.097558 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:10:50.097521 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:10:50.195689 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:10:50.195639 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:10:54.511755 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:10:54.511702 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:10:54.782191 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:10:54.782083 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:10:55.556726 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:10:55.556691 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:10:55.656301 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:10:55.656263 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:11:05.376756 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:11:05.376720 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:11:09.965799 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:11:09.965755 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:11:16.106542 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:11:16.106457 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:11:16.222180 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:11:16.222139 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:11:20.676111 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:11:20.676074 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:11:20.906579 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:11:20.906541 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:11:21.268572 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:11:21.268537 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:11:21.680028 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:11:21.679948 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:11:21.966170 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:11:21.966135 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:11:26.892485 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:11:26.892441 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:11:26.997483 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:11:26.997436 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:11:27.081669 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:11:27.081633 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:11:27.718432 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:11:27.718398 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:11:27.813463 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:11:27.813425 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:11:27.913260 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:11:27.913227 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:11:31.409539 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:11:31.409492 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:11:31.910925 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:11:31.910887 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:11:33.954011 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:11:33.953979 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:11:34.055002 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:11:34.054969 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:11:35.333886 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:11:35.333844 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:11:35.409255 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:11:35.409223 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:11:35.498001 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:11:35.497956 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:11:35.905653 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:11:35.905598 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:11:36.027327 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:11:36.027285 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:11:36.128749 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:11:36.128716 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:11:39.832204 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:11:39.832171 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:11:48.590498 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:11:48.590454 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:11:48.688805 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:11:48.688766 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:11:56.823253 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:11:56.823207 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:11:56.923837 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:11:56.923804 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:11:58.665361 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:11:58.665326 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:11:58.766166 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:11:58.766120 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:11:58.988159 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:11:58.988127 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:03.016992 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:03.016957 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:09.248085 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:09.248039 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:09.427571 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:09.427530 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:13.728105 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:13.728060 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:13.967715 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:13.967662 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:14.792329 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:14.792289 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:14.892740 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:14.892708 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:19.523211 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:19.523175 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:19.626699 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:19.626660 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:24.343010 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:24.342979 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8l7w6_b14ef695-c589-4fca-bd12-2e951f04e466/ovn-acl-logging/0.log" Jul 13 19:12:24.350519 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:24.350496 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8l7w6_b14ef695-c589-4fca-bd12-2e951f04e466/ovn-acl-logging/0.log" Jul 13 19:12:24.766488 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:24.766452 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:24.867624 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:24.867568 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:25.061263 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:25.061170 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:29.212277 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:29.212239 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:35.414148 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:35.414109 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:35.493135 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:35.493095 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:39.919869 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:39.919836 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:40.177904 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:40.177810 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:40.527396 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:40.527357 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:40.871639 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:40.871526 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:41.749866 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:41.749824 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:44.352890 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:44.352855 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:44.461930 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:44.461888 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:44.580620 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:44.580578 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:44.965478 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:44.965443 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:45.191240 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:45.191213 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:45.292951 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:45.292856 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:48.863052 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:48.863018 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:49.218365 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:49.218336 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:49.563837 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:49.563745 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:49.727951 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:49.727914 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:52.725242 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:52.725207 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:52.814162 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:52.814127 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:52.902064 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:52.902030 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:53.046909 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:53.046827 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:53.556881 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:53.556850 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:53.655534 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:53.655494 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:57.235427 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:12:57.235391 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:13:06.148133 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:06.148078 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:13:06.248890 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:06.248858 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:13:07.485167 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:07.485129 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-7b883e/auth-test-model-7b883e-kserve-54f7b85fc9-w77n5"] Jul 13 19:13:07.485538 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:07.485526 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3663330-e6bc-4a8a-b39e-0c3fde07cc8f" containerName="istio-proxy" Jul 13 19:13:07.485538 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:07.485536 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3663330-e6bc-4a8a-b39e-0c3fde07cc8f" containerName="istio-proxy" Jul 13 19:13:07.485642 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:07.485560 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3663330-e6bc-4a8a-b39e-0c3fde07cc8f" containerName="istio-proxy" Jul 13 19:13:07.485642 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:07.485565 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3663330-e6bc-4a8a-b39e-0c3fde07cc8f" containerName="istio-proxy" Jul 13 19:13:07.485642 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:07.485636 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3663330-e6bc-4a8a-b39e-0c3fde07cc8f" containerName="istio-proxy" Jul 13 19:13:07.488881 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:07.488865 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7b883e/auth-test-model-7b883e-kserve-54f7b85fc9-w77n5" Jul 13 19:13:07.493200 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:07.493174 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7b883e\"/\"kube-root-ca.crt\"" Jul 13 19:13:07.493332 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:07.493174 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7b883e\"/\"default-dockercfg-ckvwb\"" Jul 13 19:13:07.493332 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:07.493178 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7b883e\"/\"openshift-service-ca.crt\"" Jul 13 19:13:07.493668 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:07.493651 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7b883e\"/\"auth-test-model-7b883e-kserve-self-signed-certs\"" Jul 13 19:13:07.510489 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:07.510462 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7b883e/auth-test-model-7b883e-kserve-54f7b85fc9-w77n5"] Jul 13 19:13:07.526966 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:07.526927 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:13:07.633588 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:07.633546 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c910e465-1013-4f67-aed2-13872f0e00a0-tmp-dir\") pod \"auth-test-model-7b883e-kserve-54f7b85fc9-w77n5\" (UID: \"c910e465-1013-4f67-aed2-13872f0e00a0\") " pod="ai-tenant-e2e-shared-a-7b883e/auth-test-model-7b883e-kserve-54f7b85fc9-w77n5" Jul 13 19:13:07.633832 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:07.633653 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c910e465-1013-4f67-aed2-13872f0e00a0-dshm\") pod \"auth-test-model-7b883e-kserve-54f7b85fc9-w77n5\" (UID: \"c910e465-1013-4f67-aed2-13872f0e00a0\") " pod="ai-tenant-e2e-shared-a-7b883e/auth-test-model-7b883e-kserve-54f7b85fc9-w77n5" Jul 13 19:13:07.633832 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:07.633708 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzkpc\" (UniqueName: \"kubernetes.io/projected/c910e465-1013-4f67-aed2-13872f0e00a0-kube-api-access-pzkpc\") pod \"auth-test-model-7b883e-kserve-54f7b85fc9-w77n5\" (UID: \"c910e465-1013-4f67-aed2-13872f0e00a0\") " pod="ai-tenant-e2e-shared-a-7b883e/auth-test-model-7b883e-kserve-54f7b85fc9-w77n5" Jul 13 19:13:07.633832 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:07.633751 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c910e465-1013-4f67-aed2-13872f0e00a0-home\") pod \"auth-test-model-7b883e-kserve-54f7b85fc9-w77n5\" (UID: \"c910e465-1013-4f67-aed2-13872f0e00a0\") " pod="ai-tenant-e2e-shared-a-7b883e/auth-test-model-7b883e-kserve-54f7b85fc9-w77n5" Jul 13 19:13:07.633985 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:07.633845 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c910e465-1013-4f67-aed2-13872f0e00a0-tls-certs\") pod \"auth-test-model-7b883e-kserve-54f7b85fc9-w77n5\" (UID: \"c910e465-1013-4f67-aed2-13872f0e00a0\") " pod="ai-tenant-e2e-shared-a-7b883e/auth-test-model-7b883e-kserve-54f7b85fc9-w77n5" Jul 13 19:13:07.633985 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:07.633875 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c910e465-1013-4f67-aed2-13872f0e00a0-model-cache\") pod \"auth-test-model-7b883e-kserve-54f7b85fc9-w77n5\" (UID: \"c910e465-1013-4f67-aed2-13872f0e00a0\") " pod="ai-tenant-e2e-shared-a-7b883e/auth-test-model-7b883e-kserve-54f7b85fc9-w77n5" Jul 13 19:13:07.735480 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:07.735381 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c910e465-1013-4f67-aed2-13872f0e00a0-tls-certs\") pod \"auth-test-model-7b883e-kserve-54f7b85fc9-w77n5\" (UID: \"c910e465-1013-4f67-aed2-13872f0e00a0\") " pod="ai-tenant-e2e-shared-a-7b883e/auth-test-model-7b883e-kserve-54f7b85fc9-w77n5" Jul 13 19:13:07.735480 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:07.735419 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c910e465-1013-4f67-aed2-13872f0e00a0-model-cache\") pod \"auth-test-model-7b883e-kserve-54f7b85fc9-w77n5\" (UID: \"c910e465-1013-4f67-aed2-13872f0e00a0\") " pod="ai-tenant-e2e-shared-a-7b883e/auth-test-model-7b883e-kserve-54f7b85fc9-w77n5" Jul 13 19:13:07.735761 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:07.735505 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c910e465-1013-4f67-aed2-13872f0e00a0-tmp-dir\") pod \"auth-test-model-7b883e-kserve-54f7b85fc9-w77n5\" (UID: \"c910e465-1013-4f67-aed2-13872f0e00a0\") " pod="ai-tenant-e2e-shared-a-7b883e/auth-test-model-7b883e-kserve-54f7b85fc9-w77n5" Jul 13 19:13:07.735761 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:07.735560 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c910e465-1013-4f67-aed2-13872f0e00a0-dshm\") pod \"auth-test-model-7b883e-kserve-54f7b85fc9-w77n5\" (UID: \"c910e465-1013-4f67-aed2-13872f0e00a0\") " pod="ai-tenant-e2e-shared-a-7b883e/auth-test-model-7b883e-kserve-54f7b85fc9-w77n5" Jul 13 19:13:07.735761 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:07.735646 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzkpc\" (UniqueName: \"kubernetes.io/projected/c910e465-1013-4f67-aed2-13872f0e00a0-kube-api-access-pzkpc\") pod \"auth-test-model-7b883e-kserve-54f7b85fc9-w77n5\" (UID: \"c910e465-1013-4f67-aed2-13872f0e00a0\") " pod="ai-tenant-e2e-shared-a-7b883e/auth-test-model-7b883e-kserve-54f7b85fc9-w77n5" Jul 13 19:13:07.735761 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:07.735675 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c910e465-1013-4f67-aed2-13872f0e00a0-home\") pod \"auth-test-model-7b883e-kserve-54f7b85fc9-w77n5\" (UID: \"c910e465-1013-4f67-aed2-13872f0e00a0\") " pod="ai-tenant-e2e-shared-a-7b883e/auth-test-model-7b883e-kserve-54f7b85fc9-w77n5" Jul 13 19:13:07.735978 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:07.735927 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c910e465-1013-4f67-aed2-13872f0e00a0-model-cache\") pod \"auth-test-model-7b883e-kserve-54f7b85fc9-w77n5\" (UID: \"c910e465-1013-4f67-aed2-13872f0e00a0\") " pod="ai-tenant-e2e-shared-a-7b883e/auth-test-model-7b883e-kserve-54f7b85fc9-w77n5" Jul 13 19:13:07.735978 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:07.735952 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c910e465-1013-4f67-aed2-13872f0e00a0-tmp-dir\") pod \"auth-test-model-7b883e-kserve-54f7b85fc9-w77n5\" (UID: \"c910e465-1013-4f67-aed2-13872f0e00a0\") " pod="ai-tenant-e2e-shared-a-7b883e/auth-test-model-7b883e-kserve-54f7b85fc9-w77n5" Jul 13 19:13:07.736075 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:07.736007 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c910e465-1013-4f67-aed2-13872f0e00a0-home\") pod \"auth-test-model-7b883e-kserve-54f7b85fc9-w77n5\" (UID: \"c910e465-1013-4f67-aed2-13872f0e00a0\") " pod="ai-tenant-e2e-shared-a-7b883e/auth-test-model-7b883e-kserve-54f7b85fc9-w77n5" Jul 13 19:13:07.737912 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:07.737887 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c910e465-1013-4f67-aed2-13872f0e00a0-dshm\") pod \"auth-test-model-7b883e-kserve-54f7b85fc9-w77n5\" (UID: \"c910e465-1013-4f67-aed2-13872f0e00a0\") " pod="ai-tenant-e2e-shared-a-7b883e/auth-test-model-7b883e-kserve-54f7b85fc9-w77n5" Jul 13 19:13:07.738092 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:07.738073 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c910e465-1013-4f67-aed2-13872f0e00a0-tls-certs\") pod \"auth-test-model-7b883e-kserve-54f7b85fc9-w77n5\" (UID: \"c910e465-1013-4f67-aed2-13872f0e00a0\") " pod="ai-tenant-e2e-shared-a-7b883e/auth-test-model-7b883e-kserve-54f7b85fc9-w77n5" Jul 13 19:13:07.744281 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:07.744256 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzkpc\" (UniqueName: \"kubernetes.io/projected/c910e465-1013-4f67-aed2-13872f0e00a0-kube-api-access-pzkpc\") pod \"auth-test-model-7b883e-kserve-54f7b85fc9-w77n5\" (UID: \"c910e465-1013-4f67-aed2-13872f0e00a0\") " pod="ai-tenant-e2e-shared-a-7b883e/auth-test-model-7b883e-kserve-54f7b85fc9-w77n5" Jul 13 19:13:07.798795 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:07.798760 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7b883e/auth-test-model-7b883e-kserve-54f7b85fc9-w77n5" Jul 13 19:13:07.940157 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:07.940126 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7b883e/auth-test-model-7b883e-kserve-54f7b85fc9-w77n5"] Jul 13 19:13:07.941749 ip-10-0-129-191 kubenswrapper[2582]: W0713 19:13:07.941722 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc910e465_1013_4f67_aed2_13872f0e00a0.slice/crio-519100eb5f8620d557b02e206958bafc211ddd0ea7de9a2acdc759bb0c5ccbfa WatchSource:0}: Error finding container 519100eb5f8620d557b02e206958bafc211ddd0ea7de9a2acdc759bb0c5ccbfa: Status 404 returned error can't find the container with id 519100eb5f8620d557b02e206958bafc211ddd0ea7de9a2acdc759bb0c5ccbfa Jul 13 19:13:08.765179 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:08.765136 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7b883e/auth-test-model-7b883e-kserve-54f7b85fc9-w77n5" event={"ID":"c910e465-1013-4f67-aed2-13872f0e00a0","Type":"ContainerStarted","Data":"36cba3738071704b7e687dd8054fb0955e3f53acb9a95cde9e1b1ae4d4881cde"} Jul 13 19:13:08.765555 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:08.765192 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7b883e/auth-test-model-7b883e-kserve-54f7b85fc9-w77n5" event={"ID":"c910e465-1013-4f67-aed2-13872f0e00a0","Type":"ContainerStarted","Data":"519100eb5f8620d557b02e206958bafc211ddd0ea7de9a2acdc759bb0c5ccbfa"} Jul 13 19:13:08.783051 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:08.783001 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-7b883e/auth-test-model-7b883e-kserve-54f7b85fc9-w77n5" podStartSLOduration=1.7829834930000001 podStartE2EDuration="1.782983493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:13:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:13:08.781329204 +0000 UTC m=+2445.064757821" watchObservedRunningTime="2026-07-13 19:13:08.782983493 +0000 UTC m=+2445.066412122" Jul 13 19:13:14.512870 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:14.512828 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:13:14.613221 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:14.613190 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:13:17.799159 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:17.799121 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-7b883e/auth-test-model-7b883e-kserve-54f7b85fc9-w77n5" Jul 13 19:13:17.799159 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:17.799170 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-7b883e/auth-test-model-7b883e-kserve-54f7b85fc9-w77n5" Jul 13 19:13:17.812048 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:17.812021 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-7b883e/auth-test-model-7b883e-kserve-54f7b85fc9-w77n5" Jul 13 19:13:17.824003 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:13:17.823973 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-7b883e/auth-test-model-7b883e-kserve-54f7b85fc9-w77n5" Jul 13 19:14:06.202827 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:14:06.202789 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:14:30.304549 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:14:30.304515 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:16:15.693620 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:15.693519 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7b883e/auth-test-model-7b883e-kserve-54f7b85fc9-w77n5"] Jul 13 19:16:15.750709 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:15.750678 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:16:19.365539 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:19.365500 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:16:24.164383 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:24.164347 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-7b883e/sub-test-model-7b883e-kserve-7d757c48d5-zzhf5"] Jul 13 19:16:24.165048 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:24.165028 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3663330-e6bc-4a8a-b39e-0c3fde07cc8f" containerName="istio-proxy" Jul 13 19:16:24.168208 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:24.168186 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7b883e/sub-test-model-7b883e-kserve-7d757c48d5-zzhf5" Jul 13 19:16:24.171074 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:24.171050 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7b883e\"/\"sub-test-model-7b883e-kserve-self-signed-certs\"" Jul 13 19:16:24.179139 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:24.179117 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7b883e/sub-test-model-7b883e-kserve-7d757c48d5-zzhf5"] Jul 13 19:16:24.286957 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:24.286928 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7779a896-eee7-4ac7-872f-669f05df01dd-model-cache\") pod \"sub-test-model-7b883e-kserve-7d757c48d5-zzhf5\" (UID: \"7779a896-eee7-4ac7-872f-669f05df01dd\") " pod="ai-tenant-e2e-shared-a-7b883e/sub-test-model-7b883e-kserve-7d757c48d5-zzhf5" Jul 13 19:16:24.287124 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:24.286981 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vt4s9\" (UniqueName: \"kubernetes.io/projected/7779a896-eee7-4ac7-872f-669f05df01dd-kube-api-access-vt4s9\") pod \"sub-test-model-7b883e-kserve-7d757c48d5-zzhf5\" (UID: \"7779a896-eee7-4ac7-872f-669f05df01dd\") " pod="ai-tenant-e2e-shared-a-7b883e/sub-test-model-7b883e-kserve-7d757c48d5-zzhf5" Jul 13 19:16:24.287124 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:24.287039 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7779a896-eee7-4ac7-872f-669f05df01dd-dshm\") pod \"sub-test-model-7b883e-kserve-7d757c48d5-zzhf5\" (UID: \"7779a896-eee7-4ac7-872f-669f05df01dd\") " pod="ai-tenant-e2e-shared-a-7b883e/sub-test-model-7b883e-kserve-7d757c48d5-zzhf5" Jul 13 19:16:24.287124 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:24.287054 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7779a896-eee7-4ac7-872f-669f05df01dd-tls-certs\") pod \"sub-test-model-7b883e-kserve-7d757c48d5-zzhf5\" (UID: \"7779a896-eee7-4ac7-872f-669f05df01dd\") " pod="ai-tenant-e2e-shared-a-7b883e/sub-test-model-7b883e-kserve-7d757c48d5-zzhf5" Jul 13 19:16:24.287124 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:24.287097 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7779a896-eee7-4ac7-872f-669f05df01dd-tmp-dir\") pod \"sub-test-model-7b883e-kserve-7d757c48d5-zzhf5\" (UID: \"7779a896-eee7-4ac7-872f-669f05df01dd\") " pod="ai-tenant-e2e-shared-a-7b883e/sub-test-model-7b883e-kserve-7d757c48d5-zzhf5" Jul 13 19:16:24.287124 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:24.287115 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7779a896-eee7-4ac7-872f-669f05df01dd-home\") pod \"sub-test-model-7b883e-kserve-7d757c48d5-zzhf5\" (UID: \"7779a896-eee7-4ac7-872f-669f05df01dd\") " pod="ai-tenant-e2e-shared-a-7b883e/sub-test-model-7b883e-kserve-7d757c48d5-zzhf5" Jul 13 19:16:24.327348 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:24.327320 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:16:24.388023 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:24.387981 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7779a896-eee7-4ac7-872f-669f05df01dd-dshm\") pod \"sub-test-model-7b883e-kserve-7d757c48d5-zzhf5\" (UID: \"7779a896-eee7-4ac7-872f-669f05df01dd\") " pod="ai-tenant-e2e-shared-a-7b883e/sub-test-model-7b883e-kserve-7d757c48d5-zzhf5" Jul 13 19:16:24.388023 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:24.388016 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7779a896-eee7-4ac7-872f-669f05df01dd-tls-certs\") pod \"sub-test-model-7b883e-kserve-7d757c48d5-zzhf5\" (UID: \"7779a896-eee7-4ac7-872f-669f05df01dd\") " pod="ai-tenant-e2e-shared-a-7b883e/sub-test-model-7b883e-kserve-7d757c48d5-zzhf5" Jul 13 19:16:24.388286 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:24.388178 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7779a896-eee7-4ac7-872f-669f05df01dd-tmp-dir\") pod \"sub-test-model-7b883e-kserve-7d757c48d5-zzhf5\" (UID: \"7779a896-eee7-4ac7-872f-669f05df01dd\") " pod="ai-tenant-e2e-shared-a-7b883e/sub-test-model-7b883e-kserve-7d757c48d5-zzhf5" Jul 13 19:16:24.388433 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:24.388241 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7779a896-eee7-4ac7-872f-669f05df01dd-home\") pod \"sub-test-model-7b883e-kserve-7d757c48d5-zzhf5\" (UID: \"7779a896-eee7-4ac7-872f-669f05df01dd\") " pod="ai-tenant-e2e-shared-a-7b883e/sub-test-model-7b883e-kserve-7d757c48d5-zzhf5" Jul 13 19:16:24.388591 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:24.388569 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7779a896-eee7-4ac7-872f-669f05df01dd-home\") pod \"sub-test-model-7b883e-kserve-7d757c48d5-zzhf5\" (UID: \"7779a896-eee7-4ac7-872f-669f05df01dd\") " pod="ai-tenant-e2e-shared-a-7b883e/sub-test-model-7b883e-kserve-7d757c48d5-zzhf5" Jul 13 19:16:24.388702 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:24.388533 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7779a896-eee7-4ac7-872f-669f05df01dd-tmp-dir\") pod \"sub-test-model-7b883e-kserve-7d757c48d5-zzhf5\" (UID: \"7779a896-eee7-4ac7-872f-669f05df01dd\") " pod="ai-tenant-e2e-shared-a-7b883e/sub-test-model-7b883e-kserve-7d757c48d5-zzhf5" Jul 13 19:16:24.389926 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:24.389062 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7779a896-eee7-4ac7-872f-669f05df01dd-model-cache\") pod \"sub-test-model-7b883e-kserve-7d757c48d5-zzhf5\" (UID: \"7779a896-eee7-4ac7-872f-669f05df01dd\") " pod="ai-tenant-e2e-shared-a-7b883e/sub-test-model-7b883e-kserve-7d757c48d5-zzhf5" Jul 13 19:16:24.389926 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:24.389239 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt4s9\" (UniqueName: \"kubernetes.io/projected/7779a896-eee7-4ac7-872f-669f05df01dd-kube-api-access-vt4s9\") pod \"sub-test-model-7b883e-kserve-7d757c48d5-zzhf5\" (UID: \"7779a896-eee7-4ac7-872f-669f05df01dd\") " pod="ai-tenant-e2e-shared-a-7b883e/sub-test-model-7b883e-kserve-7d757c48d5-zzhf5" Jul 13 19:16:24.389926 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:24.389698 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7779a896-eee7-4ac7-872f-669f05df01dd-model-cache\") pod \"sub-test-model-7b883e-kserve-7d757c48d5-zzhf5\" (UID: \"7779a896-eee7-4ac7-872f-669f05df01dd\") " pod="ai-tenant-e2e-shared-a-7b883e/sub-test-model-7b883e-kserve-7d757c48d5-zzhf5" Jul 13 19:16:24.390742 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:24.390699 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7779a896-eee7-4ac7-872f-669f05df01dd-dshm\") pod \"sub-test-model-7b883e-kserve-7d757c48d5-zzhf5\" (UID: \"7779a896-eee7-4ac7-872f-669f05df01dd\") " pod="ai-tenant-e2e-shared-a-7b883e/sub-test-model-7b883e-kserve-7d757c48d5-zzhf5" Jul 13 19:16:24.391088 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:24.391068 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7779a896-eee7-4ac7-872f-669f05df01dd-tls-certs\") pod \"sub-test-model-7b883e-kserve-7d757c48d5-zzhf5\" (UID: \"7779a896-eee7-4ac7-872f-669f05df01dd\") " pod="ai-tenant-e2e-shared-a-7b883e/sub-test-model-7b883e-kserve-7d757c48d5-zzhf5" Jul 13 19:16:24.399459 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:24.399435 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vt4s9\" (UniqueName: \"kubernetes.io/projected/7779a896-eee7-4ac7-872f-669f05df01dd-kube-api-access-vt4s9\") pod \"sub-test-model-7b883e-kserve-7d757c48d5-zzhf5\" (UID: \"7779a896-eee7-4ac7-872f-669f05df01dd\") " pod="ai-tenant-e2e-shared-a-7b883e/sub-test-model-7b883e-kserve-7d757c48d5-zzhf5" Jul 13 19:16:24.479022 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:24.478985 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7b883e/sub-test-model-7b883e-kserve-7d757c48d5-zzhf5" Jul 13 19:16:24.615094 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:24.615064 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7b883e/sub-test-model-7b883e-kserve-7d757c48d5-zzhf5"] Jul 13 19:16:24.616903 ip-10-0-129-191 kubenswrapper[2582]: W0713 19:16:24.616872 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7779a896_eee7_4ac7_872f_669f05df01dd.slice/crio-bdb843319b7d4a4985e3f9c08d97ffbd766f0a1b7f91c8edf56cfecb67e1be19 WatchSource:0}: Error finding container bdb843319b7d4a4985e3f9c08d97ffbd766f0a1b7f91c8edf56cfecb67e1be19: Status 404 returned error can't find the container with id bdb843319b7d4a4985e3f9c08d97ffbd766f0a1b7f91c8edf56cfecb67e1be19 Jul 13 19:16:24.797808 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:24.797712 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7b883e/sub-test-model-7b883e-kserve-7d757c48d5-zzhf5" event={"ID":"7779a896-eee7-4ac7-872f-669f05df01dd","Type":"ContainerStarted","Data":"4a373f1a764abdad61009500c06e88429d52270d6a0e099ceb3a211b9232e543"} Jul 13 19:16:24.797808 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:24.797754 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7b883e/sub-test-model-7b883e-kserve-7d757c48d5-zzhf5" event={"ID":"7779a896-eee7-4ac7-872f-669f05df01dd","Type":"ContainerStarted","Data":"bdb843319b7d4a4985e3f9c08d97ffbd766f0a1b7f91c8edf56cfecb67e1be19"} Jul 13 19:16:24.816806 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:24.816751 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-7b883e/sub-test-model-7b883e-kserve-7d757c48d5-zzhf5" podStartSLOduration=0.816734163 podStartE2EDuration="816.734163ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:16:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:16:24.815644608 +0000 UTC m=+2641.099073211" watchObservedRunningTime="2026-07-13 19:16:24.816734163 +0000 UTC m=+2641.100162764" Jul 13 19:16:30.579310 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:30.579280 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:16:30.715142 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:30.715080 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-7b883e/auth-test-model-7b883e-kserve-54f7b85fc9-w77n5" podUID="c910e465-1013-4f67-aed2-13872f0e00a0" containerName="main" containerID="cri-o://36cba3738071704b7e687dd8054fb0955e3f53acb9a95cde9e1b1ae4d4881cde" gracePeriod=45 Jul 13 19:16:30.831281 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:30.831212 2582 generic.go:356] "Generic (PLEG): container finished" podID="c910e465-1013-4f67-aed2-13872f0e00a0" containerID="36cba3738071704b7e687dd8054fb0955e3f53acb9a95cde9e1b1ae4d4881cde" exitCode=0 Jul 13 19:16:30.831281 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:30.831253 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7b883e/auth-test-model-7b883e-kserve-54f7b85fc9-w77n5" event={"ID":"c910e465-1013-4f67-aed2-13872f0e00a0","Type":"ContainerDied","Data":"36cba3738071704b7e687dd8054fb0955e3f53acb9a95cde9e1b1ae4d4881cde"} Jul 13 19:16:30.983971 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:30.983940 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7b883e/auth-test-model-7b883e-kserve-54f7b85fc9-w77n5" Jul 13 19:16:31.054492 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:31.054451 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c910e465-1013-4f67-aed2-13872f0e00a0-home\") pod \"c910e465-1013-4f67-aed2-13872f0e00a0\" (UID: \"c910e465-1013-4f67-aed2-13872f0e00a0\") " Jul 13 19:16:31.054724 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:31.054569 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c910e465-1013-4f67-aed2-13872f0e00a0-tmp-dir\") pod \"c910e465-1013-4f67-aed2-13872f0e00a0\" (UID: \"c910e465-1013-4f67-aed2-13872f0e00a0\") " Jul 13 19:16:31.054724 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:31.054629 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c910e465-1013-4f67-aed2-13872f0e00a0-dshm\") pod \"c910e465-1013-4f67-aed2-13872f0e00a0\" (UID: \"c910e465-1013-4f67-aed2-13872f0e00a0\") " Jul 13 19:16:31.054724 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:31.054656 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c910e465-1013-4f67-aed2-13872f0e00a0-model-cache\") pod \"c910e465-1013-4f67-aed2-13872f0e00a0\" (UID: \"c910e465-1013-4f67-aed2-13872f0e00a0\") " Jul 13 19:16:31.054724 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:31.054705 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pzkpc\" (UniqueName: \"kubernetes.io/projected/c910e465-1013-4f67-aed2-13872f0e00a0-kube-api-access-pzkpc\") pod \"c910e465-1013-4f67-aed2-13872f0e00a0\" (UID: \"c910e465-1013-4f67-aed2-13872f0e00a0\") " Jul 13 19:16:31.054951 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:31.054756 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c910e465-1013-4f67-aed2-13872f0e00a0-tls-certs\") pod \"c910e465-1013-4f67-aed2-13872f0e00a0\" (UID: \"c910e465-1013-4f67-aed2-13872f0e00a0\") " Jul 13 19:16:31.054951 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:31.054840 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c910e465-1013-4f67-aed2-13872f0e00a0-home" (OuterVolumeSpecName: "home") pod "c910e465-1013-4f67-aed2-13872f0e00a0" (UID: "c910e465-1013-4f67-aed2-13872f0e00a0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:16:31.054951 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:31.054860 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c910e465-1013-4f67-aed2-13872f0e00a0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c910e465-1013-4f67-aed2-13872f0e00a0" (UID: "c910e465-1013-4f67-aed2-13872f0e00a0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:16:31.054951 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:31.054905 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c910e465-1013-4f67-aed2-13872f0e00a0-model-cache" (OuterVolumeSpecName: "model-cache") pod "c910e465-1013-4f67-aed2-13872f0e00a0" (UID: "c910e465-1013-4f67-aed2-13872f0e00a0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:16:31.055092 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:31.055066 2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c910e465-1013-4f67-aed2-13872f0e00a0-home\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:16:31.055092 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:31.055078 2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c910e465-1013-4f67-aed2-13872f0e00a0-tmp-dir\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:16:31.055092 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:31.055088 2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c910e465-1013-4f67-aed2-13872f0e00a0-model-cache\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:16:31.057023 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:31.056998 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c910e465-1013-4f67-aed2-13872f0e00a0-dshm" (OuterVolumeSpecName: "dshm") pod "c910e465-1013-4f67-aed2-13872f0e00a0" (UID: "c910e465-1013-4f67-aed2-13872f0e00a0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:16:31.057023 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:31.057007 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c910e465-1013-4f67-aed2-13872f0e00a0-kube-api-access-pzkpc" (OuterVolumeSpecName: "kube-api-access-pzkpc") pod "c910e465-1013-4f67-aed2-13872f0e00a0" (UID: "c910e465-1013-4f67-aed2-13872f0e00a0"). InnerVolumeSpecName "kube-api-access-pzkpc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:16:31.057023 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:31.057017 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c910e465-1013-4f67-aed2-13872f0e00a0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c910e465-1013-4f67-aed2-13872f0e00a0" (UID: "c910e465-1013-4f67-aed2-13872f0e00a0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:16:31.155917 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:31.155833 2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c910e465-1013-4f67-aed2-13872f0e00a0-tls-certs\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:16:31.155917 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:31.155864 2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c910e465-1013-4f67-aed2-13872f0e00a0-dshm\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:16:31.155917 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:31.155874 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzkpc\" (UniqueName: \"kubernetes.io/projected/c910e465-1013-4f67-aed2-13872f0e00a0-kube-api-access-pzkpc\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:16:31.837926 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:31.837882 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7b883e/auth-test-model-7b883e-kserve-54f7b85fc9-w77n5" Jul 13 19:16:31.838411 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:31.837922 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7b883e/auth-test-model-7b883e-kserve-54f7b85fc9-w77n5" event={"ID":"c910e465-1013-4f67-aed2-13872f0e00a0","Type":"ContainerDied","Data":"519100eb5f8620d557b02e206958bafc211ddd0ea7de9a2acdc759bb0c5ccbfa"} Jul 13 19:16:31.838411 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:31.837976 2582 scope.go:117] "RemoveContainer" containerID="36cba3738071704b7e687dd8054fb0955e3f53acb9a95cde9e1b1ae4d4881cde" Jul 13 19:16:31.863192 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:31.863164 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7b883e/auth-test-model-7b883e-kserve-54f7b85fc9-w77n5"] Jul 13 19:16:31.868183 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:31.868160 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-7b883e/auth-test-model-7b883e-kserve-54f7b85fc9-w77n5"] Jul 13 19:16:32.281868 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:32.281838 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c910e465-1013-4f67-aed2-13872f0e00a0" path="/var/lib/kubelet/pods/c910e465-1013-4f67-aed2-13872f0e00a0/volumes" Jul 13 19:16:34.480036 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:34.480005 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-7b883e/sub-test-model-7b883e-kserve-7d757c48d5-zzhf5" Jul 13 19:16:34.480529 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:34.480060 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-7b883e/sub-test-model-7b883e-kserve-7d757c48d5-zzhf5" Jul 13 19:16:34.492804 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:34.492779 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-7b883e/sub-test-model-7b883e-kserve-7d757c48d5-zzhf5" Jul 13 19:16:34.866278 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:34.866201 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-7b883e/sub-test-model-7b883e-kserve-7d757c48d5-zzhf5" Jul 13 19:16:44.995295 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:44.995261 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:16:49.471670 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:49.471634 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:16:49.579999 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:16:49.579967 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:17:20.082390 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:20.082316 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7b883e/sub-test-model-7b883e-kserve-7d757c48d5-zzhf5"] Jul 13 19:17:20.105547 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:20.105510 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:17:20.401959 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:20.401870 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:17:20.502059 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:20.502027 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:17:20.677419 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:20.677339 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:17:20.997593 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:20.997553 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:17:24.376930 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:24.376886 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8l7w6_b14ef695-c589-4fca-bd12-2e951f04e466/ovn-acl-logging/0.log" Jul 13 19:17:24.385699 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:24.385677 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8l7w6_b14ef695-c589-4fca-bd12-2e951f04e466/ovn-acl-logging/0.log" Jul 13 19:17:25.555852 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:25.555819 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-7b883e/rate-test-model-7b883e-kserve-7776bcb76c-2mptq"] Jul 13 19:17:25.556489 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:25.556465 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c910e465-1013-4f67-aed2-13872f0e00a0" containerName="main" Jul 13 19:17:25.556578 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:25.556492 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c910e465-1013-4f67-aed2-13872f0e00a0" containerName="main" Jul 13 19:17:25.556673 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:25.556594 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c910e465-1013-4f67-aed2-13872f0e00a0" containerName="main" Jul 13 19:17:25.560878 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:25.560854 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7b883e/rate-test-model-7b883e-kserve-7776bcb76c-2mptq" Jul 13 19:17:25.564224 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:25.564200 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7b883e\"/\"rate-test-model-7b883e-kserve-self-signed-certs\"" Jul 13 19:17:25.571128 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:25.571107 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7b883e/rate-test-model-7b883e-kserve-7776bcb76c-2mptq"] Jul 13 19:17:25.581493 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:25.581462 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:17:25.593845 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:25.593821 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed65f12d-f80a-4986-9101-44ce26fc95f8-model-cache\") pod \"rate-test-model-7b883e-kserve-7776bcb76c-2mptq\" (UID: \"ed65f12d-f80a-4986-9101-44ce26fc95f8\") " pod="ai-tenant-e2e-shared-a-7b883e/rate-test-model-7b883e-kserve-7776bcb76c-2mptq" Jul 13 19:17:25.593974 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:25.593855 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed65f12d-f80a-4986-9101-44ce26fc95f8-tls-certs\") pod \"rate-test-model-7b883e-kserve-7776bcb76c-2mptq\" (UID: \"ed65f12d-f80a-4986-9101-44ce26fc95f8\") " pod="ai-tenant-e2e-shared-a-7b883e/rate-test-model-7b883e-kserve-7776bcb76c-2mptq" Jul 13 19:17:25.593974 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:25.593934 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed65f12d-f80a-4986-9101-44ce26fc95f8-dshm\") pod \"rate-test-model-7b883e-kserve-7776bcb76c-2mptq\" (UID: \"ed65f12d-f80a-4986-9101-44ce26fc95f8\") " pod="ai-tenant-e2e-shared-a-7b883e/rate-test-model-7b883e-kserve-7776bcb76c-2mptq" Jul 13 19:17:25.594054 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:25.593997 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwqmt\" (UniqueName: \"kubernetes.io/projected/ed65f12d-f80a-4986-9101-44ce26fc95f8-kube-api-access-mwqmt\") pod \"rate-test-model-7b883e-kserve-7776bcb76c-2mptq\" (UID: \"ed65f12d-f80a-4986-9101-44ce26fc95f8\") " pod="ai-tenant-e2e-shared-a-7b883e/rate-test-model-7b883e-kserve-7776bcb76c-2mptq" Jul 13 19:17:25.594054 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:25.594030 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed65f12d-f80a-4986-9101-44ce26fc95f8-home\") pod \"rate-test-model-7b883e-kserve-7776bcb76c-2mptq\" (UID: \"ed65f12d-f80a-4986-9101-44ce26fc95f8\") " pod="ai-tenant-e2e-shared-a-7b883e/rate-test-model-7b883e-kserve-7776bcb76c-2mptq" Jul 13 19:17:25.594119 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:25.594060 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed65f12d-f80a-4986-9101-44ce26fc95f8-tmp-dir\") pod \"rate-test-model-7b883e-kserve-7776bcb76c-2mptq\" (UID: \"ed65f12d-f80a-4986-9101-44ce26fc95f8\") " pod="ai-tenant-e2e-shared-a-7b883e/rate-test-model-7b883e-kserve-7776bcb76c-2mptq" Jul 13 19:17:25.695406 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:25.695368 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed65f12d-f80a-4986-9101-44ce26fc95f8-dshm\") pod \"rate-test-model-7b883e-kserve-7776bcb76c-2mptq\" (UID: \"ed65f12d-f80a-4986-9101-44ce26fc95f8\") " pod="ai-tenant-e2e-shared-a-7b883e/rate-test-model-7b883e-kserve-7776bcb76c-2mptq" Jul 13 19:17:25.695406 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:25.695411 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwqmt\" (UniqueName: \"kubernetes.io/projected/ed65f12d-f80a-4986-9101-44ce26fc95f8-kube-api-access-mwqmt\") pod \"rate-test-model-7b883e-kserve-7776bcb76c-2mptq\" (UID: \"ed65f12d-f80a-4986-9101-44ce26fc95f8\") " pod="ai-tenant-e2e-shared-a-7b883e/rate-test-model-7b883e-kserve-7776bcb76c-2mptq" Jul 13 19:17:25.695712 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:25.695437 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed65f12d-f80a-4986-9101-44ce26fc95f8-home\") pod \"rate-test-model-7b883e-kserve-7776bcb76c-2mptq\" (UID: \"ed65f12d-f80a-4986-9101-44ce26fc95f8\") " pod="ai-tenant-e2e-shared-a-7b883e/rate-test-model-7b883e-kserve-7776bcb76c-2mptq" Jul 13 19:17:25.695712 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:25.695471 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed65f12d-f80a-4986-9101-44ce26fc95f8-tmp-dir\") pod \"rate-test-model-7b883e-kserve-7776bcb76c-2mptq\" (UID: \"ed65f12d-f80a-4986-9101-44ce26fc95f8\") " pod="ai-tenant-e2e-shared-a-7b883e/rate-test-model-7b883e-kserve-7776bcb76c-2mptq" Jul 13 19:17:25.695712 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:25.695539 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed65f12d-f80a-4986-9101-44ce26fc95f8-model-cache\") pod \"rate-test-model-7b883e-kserve-7776bcb76c-2mptq\" (UID: \"ed65f12d-f80a-4986-9101-44ce26fc95f8\") " pod="ai-tenant-e2e-shared-a-7b883e/rate-test-model-7b883e-kserve-7776bcb76c-2mptq" Jul 13 19:17:25.695712 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:25.695568 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed65f12d-f80a-4986-9101-44ce26fc95f8-tls-certs\") pod \"rate-test-model-7b883e-kserve-7776bcb76c-2mptq\" (UID: \"ed65f12d-f80a-4986-9101-44ce26fc95f8\") " pod="ai-tenant-e2e-shared-a-7b883e/rate-test-model-7b883e-kserve-7776bcb76c-2mptq" Jul 13 19:17:25.695990 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:25.695965 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed65f12d-f80a-4986-9101-44ce26fc95f8-model-cache\") pod \"rate-test-model-7b883e-kserve-7776bcb76c-2mptq\" (UID: \"ed65f12d-f80a-4986-9101-44ce26fc95f8\") " pod="ai-tenant-e2e-shared-a-7b883e/rate-test-model-7b883e-kserve-7776bcb76c-2mptq" Jul 13 19:17:25.696056 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:25.695979 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed65f12d-f80a-4986-9101-44ce26fc95f8-tmp-dir\") pod \"rate-test-model-7b883e-kserve-7776bcb76c-2mptq\" (UID: \"ed65f12d-f80a-4986-9101-44ce26fc95f8\") " pod="ai-tenant-e2e-shared-a-7b883e/rate-test-model-7b883e-kserve-7776bcb76c-2mptq" Jul 13 19:17:25.696056 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:25.696023 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed65f12d-f80a-4986-9101-44ce26fc95f8-home\") pod \"rate-test-model-7b883e-kserve-7776bcb76c-2mptq\" (UID: \"ed65f12d-f80a-4986-9101-44ce26fc95f8\") " pod="ai-tenant-e2e-shared-a-7b883e/rate-test-model-7b883e-kserve-7776bcb76c-2mptq" Jul 13 19:17:25.697847 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:25.697825 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed65f12d-f80a-4986-9101-44ce26fc95f8-dshm\") pod \"rate-test-model-7b883e-kserve-7776bcb76c-2mptq\" (UID: \"ed65f12d-f80a-4986-9101-44ce26fc95f8\") " pod="ai-tenant-e2e-shared-a-7b883e/rate-test-model-7b883e-kserve-7776bcb76c-2mptq" Jul 13 19:17:25.698105 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:25.698089 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed65f12d-f80a-4986-9101-44ce26fc95f8-tls-certs\") pod \"rate-test-model-7b883e-kserve-7776bcb76c-2mptq\" (UID: \"ed65f12d-f80a-4986-9101-44ce26fc95f8\") " pod="ai-tenant-e2e-shared-a-7b883e/rate-test-model-7b883e-kserve-7776bcb76c-2mptq" Jul 13 19:17:25.704148 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:25.704128 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwqmt\" (UniqueName: \"kubernetes.io/projected/ed65f12d-f80a-4986-9101-44ce26fc95f8-kube-api-access-mwqmt\") pod \"rate-test-model-7b883e-kserve-7776bcb76c-2mptq\" (UID: \"ed65f12d-f80a-4986-9101-44ce26fc95f8\") " pod="ai-tenant-e2e-shared-a-7b883e/rate-test-model-7b883e-kserve-7776bcb76c-2mptq" Jul 13 19:17:25.870881 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:25.870789 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7b883e/rate-test-model-7b883e-kserve-7776bcb76c-2mptq" Jul 13 19:17:26.007303 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:26.007275 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7b883e/rate-test-model-7b883e-kserve-7776bcb76c-2mptq"] Jul 13 19:17:26.009375 ip-10-0-129-191 kubenswrapper[2582]: W0713 19:17:26.009334 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poded65f12d_f80a_4986_9101_44ce26fc95f8.slice/crio-35b2d4bee7f067942ff61b3dc9869b04b72cf203ac697aa34a40525a1fc08fb3 WatchSource:0}: Error finding container 35b2d4bee7f067942ff61b3dc9869b04b72cf203ac697aa34a40525a1fc08fb3: Status 404 returned error can't find the container with id 35b2d4bee7f067942ff61b3dc9869b04b72cf203ac697aa34a40525a1fc08fb3 Jul 13 19:17:26.122362 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:26.122271 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7b883e/rate-test-model-7b883e-kserve-7776bcb76c-2mptq" event={"ID":"ed65f12d-f80a-4986-9101-44ce26fc95f8","Type":"ContainerStarted","Data":"c89a3cc4d99f607551359bc0a13a38000e3f5aa65b213828236bbd52a49a0a07"} Jul 13 19:17:26.122362 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:26.122310 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7b883e/rate-test-model-7b883e-kserve-7776bcb76c-2mptq" event={"ID":"ed65f12d-f80a-4986-9101-44ce26fc95f8","Type":"ContainerStarted","Data":"35b2d4bee7f067942ff61b3dc9869b04b72cf203ac697aa34a40525a1fc08fb3"} Jul 13 19:17:26.143546 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:26.143498 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-7b883e/rate-test-model-7b883e-kserve-7776bcb76c-2mptq" podStartSLOduration=1.143479612 podStartE2EDuration="1.143479612s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:17:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:17:26.14231074 +0000 UTC m=+2702.425739379" watchObservedRunningTime="2026-07-13 19:17:26.143479612 +0000 UTC m=+2702.426908214" Jul 13 19:17:35.103904 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:35.103860 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-7b883e/sub-test-model-7b883e-kserve-7d757c48d5-zzhf5" podUID="7779a896-eee7-4ac7-872f-669f05df01dd" containerName="main" containerID="cri-o://4a373f1a764abdad61009500c06e88429d52270d6a0e099ceb3a211b9232e543" gracePeriod=45 Jul 13 19:17:35.369968 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:35.369903 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7b883e/sub-test-model-7b883e-kserve-7d757c48d5-zzhf5" Jul 13 19:17:35.486020 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:35.485984 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7779a896-eee7-4ac7-872f-669f05df01dd-tmp-dir\") pod \"7779a896-eee7-4ac7-872f-669f05df01dd\" (UID: \"7779a896-eee7-4ac7-872f-669f05df01dd\") " Jul 13 19:17:35.486213 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:35.486050 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7779a896-eee7-4ac7-872f-669f05df01dd-model-cache\") pod \"7779a896-eee7-4ac7-872f-669f05df01dd\" (UID: \"7779a896-eee7-4ac7-872f-669f05df01dd\") " Jul 13 19:17:35.486213 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:35.486099 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7779a896-eee7-4ac7-872f-669f05df01dd-home\") pod \"7779a896-eee7-4ac7-872f-669f05df01dd\" (UID: \"7779a896-eee7-4ac7-872f-669f05df01dd\") " Jul 13 19:17:35.486213 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:35.486143 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7779a896-eee7-4ac7-872f-669f05df01dd-tls-certs\") pod \"7779a896-eee7-4ac7-872f-669f05df01dd\" (UID: \"7779a896-eee7-4ac7-872f-669f05df01dd\") " Jul 13 19:17:35.486213 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:35.486199 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vt4s9\" (UniqueName: \"kubernetes.io/projected/7779a896-eee7-4ac7-872f-669f05df01dd-kube-api-access-vt4s9\") pod \"7779a896-eee7-4ac7-872f-669f05df01dd\" (UID: \"7779a896-eee7-4ac7-872f-669f05df01dd\") " Jul 13 19:17:35.486448 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:35.486270 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7779a896-eee7-4ac7-872f-669f05df01dd-model-cache" (OuterVolumeSpecName: "model-cache") pod "7779a896-eee7-4ac7-872f-669f05df01dd" (UID: "7779a896-eee7-4ac7-872f-669f05df01dd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:17:35.486448 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:35.486285 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7779a896-eee7-4ac7-872f-669f05df01dd-dshm\") pod \"7779a896-eee7-4ac7-872f-669f05df01dd\" (UID: \"7779a896-eee7-4ac7-872f-669f05df01dd\") " Jul 13 19:17:35.486448 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:35.486311 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7779a896-eee7-4ac7-872f-669f05df01dd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7779a896-eee7-4ac7-872f-669f05df01dd" (UID: "7779a896-eee7-4ac7-872f-669f05df01dd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:17:35.486448 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:35.486388 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7779a896-eee7-4ac7-872f-669f05df01dd-home" (OuterVolumeSpecName: "home") pod "7779a896-eee7-4ac7-872f-669f05df01dd" (UID: "7779a896-eee7-4ac7-872f-669f05df01dd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:17:35.486692 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:35.486659 2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7779a896-eee7-4ac7-872f-669f05df01dd-tmp-dir\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:17:35.486692 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:35.486675 2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7779a896-eee7-4ac7-872f-669f05df01dd-model-cache\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:17:35.486692 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:35.486683 2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7779a896-eee7-4ac7-872f-669f05df01dd-home\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:17:35.488419 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:35.488393 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7779a896-eee7-4ac7-872f-669f05df01dd-dshm" (OuterVolumeSpecName: "dshm") pod "7779a896-eee7-4ac7-872f-669f05df01dd" (UID: "7779a896-eee7-4ac7-872f-669f05df01dd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:17:35.488419 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:35.488409 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7779a896-eee7-4ac7-872f-669f05df01dd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7779a896-eee7-4ac7-872f-669f05df01dd" (UID: "7779a896-eee7-4ac7-872f-669f05df01dd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:17:35.488567 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:35.488426 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7779a896-eee7-4ac7-872f-669f05df01dd-kube-api-access-vt4s9" (OuterVolumeSpecName: "kube-api-access-vt4s9") pod "7779a896-eee7-4ac7-872f-669f05df01dd" (UID: "7779a896-eee7-4ac7-872f-669f05df01dd"). InnerVolumeSpecName "kube-api-access-vt4s9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:17:35.587845 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:35.587797 2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7779a896-eee7-4ac7-872f-669f05df01dd-tls-certs\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:17:35.587845 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:35.587840 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vt4s9\" (UniqueName: \"kubernetes.io/projected/7779a896-eee7-4ac7-872f-669f05df01dd-kube-api-access-vt4s9\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:17:35.587845 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:35.587853 2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7779a896-eee7-4ac7-872f-669f05df01dd-dshm\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:17:35.871432 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:35.871396 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-7b883e/rate-test-model-7b883e-kserve-7776bcb76c-2mptq" Jul 13 19:17:35.871653 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:35.871450 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-7b883e/rate-test-model-7b883e-kserve-7776bcb76c-2mptq" Jul 13 19:17:35.884175 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:35.884146 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-7b883e/rate-test-model-7b883e-kserve-7776bcb76c-2mptq" Jul 13 19:17:36.179972 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:36.179888 2582 generic.go:356] "Generic (PLEG): container finished" podID="7779a896-eee7-4ac7-872f-669f05df01dd" containerID="4a373f1a764abdad61009500c06e88429d52270d6a0e099ceb3a211b9232e543" exitCode=0 Jul 13 19:17:36.179972 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:36.179937 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7b883e/sub-test-model-7b883e-kserve-7d757c48d5-zzhf5" Jul 13 19:17:36.180413 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:36.179969 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7b883e/sub-test-model-7b883e-kserve-7d757c48d5-zzhf5" event={"ID":"7779a896-eee7-4ac7-872f-669f05df01dd","Type":"ContainerDied","Data":"4a373f1a764abdad61009500c06e88429d52270d6a0e099ceb3a211b9232e543"} Jul 13 19:17:36.180413 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:36.180011 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7b883e/sub-test-model-7b883e-kserve-7d757c48d5-zzhf5" event={"ID":"7779a896-eee7-4ac7-872f-669f05df01dd","Type":"ContainerDied","Data":"bdb843319b7d4a4985e3f9c08d97ffbd766f0a1b7f91c8edf56cfecb67e1be19"} Jul 13 19:17:36.180413 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:36.180028 2582 scope.go:117] "RemoveContainer" containerID="4a373f1a764abdad61009500c06e88429d52270d6a0e099ceb3a211b9232e543" Jul 13 19:17:36.190047 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:36.190030 2582 scope.go:117] "RemoveContainer" containerID="4a373f1a764abdad61009500c06e88429d52270d6a0e099ceb3a211b9232e543" Jul 13 19:17:36.190358 ip-10-0-129-191 kubenswrapper[2582]: E0713 19:17:36.190341 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a373f1a764abdad61009500c06e88429d52270d6a0e099ceb3a211b9232e543\": container with ID starting with 4a373f1a764abdad61009500c06e88429d52270d6a0e099ceb3a211b9232e543 not found: ID does not exist" containerID="4a373f1a764abdad61009500c06e88429d52270d6a0e099ceb3a211b9232e543" Jul 13 19:17:36.190419 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:36.190368 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a373f1a764abdad61009500c06e88429d52270d6a0e099ceb3a211b9232e543"} err="failed to get container status \"4a373f1a764abdad61009500c06e88429d52270d6a0e099ceb3a211b9232e543\": rpc error: code = NotFound desc = could not find container \"4a373f1a764abdad61009500c06e88429d52270d6a0e099ceb3a211b9232e543\": container with ID starting with 4a373f1a764abdad61009500c06e88429d52270d6a0e099ceb3a211b9232e543 not found: ID does not exist" Jul 13 19:17:36.192241 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:36.192222 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-7b883e/rate-test-model-7b883e-kserve-7776bcb76c-2mptq" Jul 13 19:17:36.202362 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:36.202336 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7b883e/sub-test-model-7b883e-kserve-7d757c48d5-zzhf5"] Jul 13 19:17:36.206360 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:36.206339 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-7b883e/sub-test-model-7b883e-kserve-7d757c48d5-zzhf5"] Jul 13 19:17:36.282693 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:36.282663 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7779a896-eee7-4ac7-872f-669f05df01dd" path="/var/lib/kubelet/pods/7779a896-eee7-4ac7-872f-669f05df01dd/volumes" Jul 13 19:17:58.541325 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:17:58.541293 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:18:06.906516 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:06.906456 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:18:07.006680 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:07.006645 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:18:15.359966 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:15.359933 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:18:15.459471 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:15.459436 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:18:26.905170 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:26.905129 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:18:45.263204 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:45.263121 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7b883e/rate-test-model-7b883e-kserve-7776bcb76c-2mptq"] Jul 13 19:18:45.278067 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:45.278034 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:18:46.200639 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:46.200584 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:18:47.213465 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:47.213433 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7356-140124767536560"] Jul 13 19:18:47.213877 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:47.213863 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7779a896-eee7-4ac7-872f-669f05df01dd" containerName="main" Jul 13 19:18:47.213922 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:47.213878 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7779a896-eee7-4ac7-872f-669f05df01dd" containerName="main" Jul 13 19:18:47.213965 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:47.213957 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7779a896-eee7-4ac7-872f-669f05df01dd" containerName="main" Jul 13 19:18:47.216963 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:47.216948 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7356-140124767536560" Jul 13 19:18:47.219861 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:47.219840 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-dkgzq\"" Jul 13 19:18:47.224584 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:47.224432 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7356-140124767536560"] Jul 13 19:18:47.283419 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:47.283388 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phflc\" (UniqueName: \"kubernetes.io/projected/011e5c94-e091-4cf4-a86e-cdc878bd49c6-kube-api-access-phflc\") pod \"test-curl-7356-140124767536560\" (UID: \"011e5c94-e091-4cf4-a86e-cdc878bd49c6\") " pod="openshift-ingress/test-curl-7356-140124767536560" Jul 13 19:18:47.384547 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:47.384512 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phflc\" (UniqueName: \"kubernetes.io/projected/011e5c94-e091-4cf4-a86e-cdc878bd49c6-kube-api-access-phflc\") pod \"test-curl-7356-140124767536560\" (UID: \"011e5c94-e091-4cf4-a86e-cdc878bd49c6\") " pod="openshift-ingress/test-curl-7356-140124767536560" Jul 13 19:18:47.394113 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:47.394083 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phflc\" (UniqueName: \"kubernetes.io/projected/011e5c94-e091-4cf4-a86e-cdc878bd49c6-kube-api-access-phflc\") pod \"test-curl-7356-140124767536560\" (UID: \"011e5c94-e091-4cf4-a86e-cdc878bd49c6\") " pod="openshift-ingress/test-curl-7356-140124767536560" Jul 13 19:18:47.527588 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:47.527495 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7356-140124767536560" Jul 13 19:18:47.672628 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:47.667090 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7356-140124767536560"] Jul 13 19:18:47.676132 ip-10-0-129-191 kubenswrapper[2582]: W0713 19:18:47.676094 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod011e5c94_e091_4cf4_a86e_cdc878bd49c6.slice/crio-20bb6e504cade735beb76545b5d0b8e37044b5b9d1d8f263c549c01c99b556cb WatchSource:0}: Error finding container 20bb6e504cade735beb76545b5d0b8e37044b5b9d1d8f263c549c01c99b556cb: Status 404 returned error can't find the container with id 20bb6e504cade735beb76545b5d0b8e37044b5b9d1d8f263c549c01c99b556cb Jul 13 19:18:47.677508 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:47.677486 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 19:18:48.570502 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:48.570469 2582 generic.go:356] "Generic (PLEG): container finished" podID="011e5c94-e091-4cf4-a86e-cdc878bd49c6" containerID="5e8463354cff7343755c95ce21b410bd69f1869d96a1837272303c85835c5c60" exitCode=0 Jul 13 19:18:48.570917 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:48.570535 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7356-140124767536560" event={"ID":"011e5c94-e091-4cf4-a86e-cdc878bd49c6","Type":"ContainerDied","Data":"5e8463354cff7343755c95ce21b410bd69f1869d96a1837272303c85835c5c60"} Jul 13 19:18:48.570917 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:48.570567 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7356-140124767536560" event={"ID":"011e5c94-e091-4cf4-a86e-cdc878bd49c6","Type":"ContainerStarted","Data":"20bb6e504cade735beb76545b5d0b8e37044b5b9d1d8f263c549c01c99b556cb"} Jul 13 19:18:49.721510 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:49.721482 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7356-140124767536560" Jul 13 19:18:49.742788 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:49.742762 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7356-140124767536560_011e5c94-e091-4cf4-a86e-cdc878bd49c6/test-curl-7356-140124767536560/0.log" Jul 13 19:18:49.770364 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:49.770326 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7356-140124767536560"] Jul 13 19:18:49.774653 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:49.774630 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7356-140124767536560"] Jul 13 19:18:49.807725 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:49.807697 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phflc\" (UniqueName: \"kubernetes.io/projected/011e5c94-e091-4cf4-a86e-cdc878bd49c6-kube-api-access-phflc\") pod \"011e5c94-e091-4cf4-a86e-cdc878bd49c6\" (UID: \"011e5c94-e091-4cf4-a86e-cdc878bd49c6\") " Jul 13 19:18:49.809901 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:49.809872 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/011e5c94-e091-4cf4-a86e-cdc878bd49c6-kube-api-access-phflc" (OuterVolumeSpecName: "kube-api-access-phflc") pod "011e5c94-e091-4cf4-a86e-cdc878bd49c6" (UID: "011e5c94-e091-4cf4-a86e-cdc878bd49c6"). InnerVolumeSpecName "kube-api-access-phflc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:18:49.909433 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:49.909339 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phflc\" (UniqueName: \"kubernetes.io/projected/011e5c94-e091-4cf4-a86e-cdc878bd49c6-kube-api-access-phflc\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:18:49.911646 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:49.911621 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7356-140124767986608"] Jul 13 19:18:49.912034 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:49.912022 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="011e5c94-e091-4cf4-a86e-cdc878bd49c6" containerName="test-curl-7356-140124767536560" Jul 13 19:18:49.912076 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:49.912036 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="011e5c94-e091-4cf4-a86e-cdc878bd49c6" containerName="test-curl-7356-140124767536560" Jul 13 19:18:49.912109 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:49.912103 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="011e5c94-e091-4cf4-a86e-cdc878bd49c6" containerName="test-curl-7356-140124767536560" Jul 13 19:18:49.915207 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:49.915191 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7356-140124767986608" Jul 13 19:18:49.924651 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:49.924629 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7356-140124767986608"] Jul 13 19:18:50.010274 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:50.010238 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgkkj\" (UniqueName: \"kubernetes.io/projected/1a2357e1-51e1-48c0-8e4e-c554c682c0d2-kube-api-access-hgkkj\") pod \"test-curl-7356-140124767986608\" (UID: \"1a2357e1-51e1-48c0-8e4e-c554c682c0d2\") " pod="openshift-ingress/test-curl-7356-140124767986608" Jul 13 19:18:50.110887 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:50.110848 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgkkj\" (UniqueName: \"kubernetes.io/projected/1a2357e1-51e1-48c0-8e4e-c554c682c0d2-kube-api-access-hgkkj\") pod \"test-curl-7356-140124767986608\" (UID: \"1a2357e1-51e1-48c0-8e4e-c554c682c0d2\") " pod="openshift-ingress/test-curl-7356-140124767986608" Jul 13 19:18:50.119932 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:50.119899 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgkkj\" (UniqueName: \"kubernetes.io/projected/1a2357e1-51e1-48c0-8e4e-c554c682c0d2-kube-api-access-hgkkj\") pod \"test-curl-7356-140124767986608\" (UID: \"1a2357e1-51e1-48c0-8e4e-c554c682c0d2\") " pod="openshift-ingress/test-curl-7356-140124767986608" Jul 13 19:18:50.224759 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:50.224720 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7356-140124767986608" Jul 13 19:18:50.284665 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:50.284591 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="011e5c94-e091-4cf4-a86e-cdc878bd49c6" path="/var/lib/kubelet/pods/011e5c94-e091-4cf4-a86e-cdc878bd49c6/volumes" Jul 13 19:18:50.360748 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:50.360710 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7356-140124767986608"] Jul 13 19:18:50.362751 ip-10-0-129-191 kubenswrapper[2582]: W0713 19:18:50.362730 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1a2357e1_51e1_48c0_8e4e_c554c682c0d2.slice/crio-f14833e86a0bfe7015c6119bd6f3012134c6f5b582e5e697faca8a0e24635e2d WatchSource:0}: Error finding container f14833e86a0bfe7015c6119bd6f3012134c6f5b582e5e697faca8a0e24635e2d: Status 404 returned error can't find the container with id f14833e86a0bfe7015c6119bd6f3012134c6f5b582e5e697faca8a0e24635e2d Jul 13 19:18:50.583164 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:50.583080 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7356-140124767986608" event={"ID":"1a2357e1-51e1-48c0-8e4e-c554c682c0d2","Type":"ContainerStarted","Data":"f14833e86a0bfe7015c6119bd6f3012134c6f5b582e5e697faca8a0e24635e2d"} Jul 13 19:18:50.584442 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:50.584422 2582 scope.go:117] "RemoveContainer" containerID="5e8463354cff7343755c95ce21b410bd69f1869d96a1837272303c85835c5c60" Jul 13 19:18:50.584442 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:50.584433 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7356-140124767536560" Jul 13 19:18:51.592684 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:51.592647 2582 generic.go:356] "Generic (PLEG): container finished" podID="1a2357e1-51e1-48c0-8e4e-c554c682c0d2" containerID="3bbf5ec3d73861b943f5408f727741386889d9d2ce5be2be07c8ba36546bec2c" exitCode=0 Jul 13 19:18:51.593067 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:51.592681 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7356-140124767986608" event={"ID":"1a2357e1-51e1-48c0-8e4e-c554c682c0d2","Type":"ContainerDied","Data":"3bbf5ec3d73861b943f5408f727741386889d9d2ce5be2be07c8ba36546bec2c"} Jul 13 19:18:52.735707 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:52.735681 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7356-140124767986608" Jul 13 19:18:52.759572 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:52.759549 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7356-140124767986608_1a2357e1-51e1-48c0-8e4e-c554c682c0d2/test-curl-7356-140124767986608/0.log" Jul 13 19:18:52.786071 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:52.786040 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7356-140124767986608"] Jul 13 19:18:52.790195 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:52.790170 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7356-140124767986608"] Jul 13 19:18:52.835704 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:52.835674 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgkkj\" (UniqueName: \"kubernetes.io/projected/1a2357e1-51e1-48c0-8e4e-c554c682c0d2-kube-api-access-hgkkj\") pod \"1a2357e1-51e1-48c0-8e4e-c554c682c0d2\" (UID: \"1a2357e1-51e1-48c0-8e4e-c554c682c0d2\") " Jul 13 19:18:52.837924 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:52.837899 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a2357e1-51e1-48c0-8e4e-c554c682c0d2-kube-api-access-hgkkj" (OuterVolumeSpecName: "kube-api-access-hgkkj") pod "1a2357e1-51e1-48c0-8e4e-c554c682c0d2" (UID: "1a2357e1-51e1-48c0-8e4e-c554c682c0d2"). InnerVolumeSpecName "kube-api-access-hgkkj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:18:52.936834 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:52.936749 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgkkj\" (UniqueName: \"kubernetes.io/projected/1a2357e1-51e1-48c0-8e4e-c554c682c0d2-kube-api-access-hgkkj\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:18:52.953696 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:52.953662 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7356-140124767772720"] Jul 13 19:18:52.954068 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:52.954056 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a2357e1-51e1-48c0-8e4e-c554c682c0d2" containerName="test-curl-7356-140124767986608" Jul 13 19:18:52.954108 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:52.954070 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a2357e1-51e1-48c0-8e4e-c554c682c0d2" containerName="test-curl-7356-140124767986608" Jul 13 19:18:52.954155 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:52.954142 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a2357e1-51e1-48c0-8e4e-c554c682c0d2" containerName="test-curl-7356-140124767986608" Jul 13 19:18:52.957102 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:52.957087 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7356-140124767772720" Jul 13 19:18:52.964618 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:52.964573 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7356-140124767772720"] Jul 13 19:18:53.037757 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:53.037712 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpj5x\" (UniqueName: \"kubernetes.io/projected/a2db9ab7-07ba-4124-bd3a-27bce4f26cf3-kube-api-access-hpj5x\") pod \"test-curl-7356-140124767772720\" (UID: \"a2db9ab7-07ba-4124-bd3a-27bce4f26cf3\") " pod="openshift-ingress/test-curl-7356-140124767772720" Jul 13 19:18:53.139222 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:53.139180 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpj5x\" (UniqueName: \"kubernetes.io/projected/a2db9ab7-07ba-4124-bd3a-27bce4f26cf3-kube-api-access-hpj5x\") pod \"test-curl-7356-140124767772720\" (UID: \"a2db9ab7-07ba-4124-bd3a-27bce4f26cf3\") " pod="openshift-ingress/test-curl-7356-140124767772720" Jul 13 19:18:53.147864 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:53.147835 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpj5x\" (UniqueName: \"kubernetes.io/projected/a2db9ab7-07ba-4124-bd3a-27bce4f26cf3-kube-api-access-hpj5x\") pod \"test-curl-7356-140124767772720\" (UID: \"a2db9ab7-07ba-4124-bd3a-27bce4f26cf3\") " pod="openshift-ingress/test-curl-7356-140124767772720" Jul 13 19:18:53.267866 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:53.267832 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7356-140124767772720" Jul 13 19:18:53.407088 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:53.407064 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7356-140124767772720"] Jul 13 19:18:53.408071 ip-10-0-129-191 kubenswrapper[2582]: W0713 19:18:53.408048 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda2db9ab7_07ba_4124_bd3a_27bce4f26cf3.slice/crio-89358e033ae1560584f1a1f1cafe0346204b82faf5e5d95139423a9b3472a8b7 WatchSource:0}: Error finding container 89358e033ae1560584f1a1f1cafe0346204b82faf5e5d95139423a9b3472a8b7: Status 404 returned error can't find the container with id 89358e033ae1560584f1a1f1cafe0346204b82faf5e5d95139423a9b3472a8b7 Jul 13 19:18:53.603904 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:53.603795 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7356-140124767772720" event={"ID":"a2db9ab7-07ba-4124-bd3a-27bce4f26cf3","Type":"ContainerStarted","Data":"89358e033ae1560584f1a1f1cafe0346204b82faf5e5d95139423a9b3472a8b7"} Jul 13 19:18:53.604958 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:53.604939 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7356-140124767986608" Jul 13 19:18:53.605061 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:53.604941 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f14833e86a0bfe7015c6119bd6f3012134c6f5b582e5e697faca8a0e24635e2d" Jul 13 19:18:53.615412 ip-10-0-129-191 kubenswrapper[2582]: E0713 19:18:53.615385 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7356-140124767986608\" is forbidden: User \"system:node:ip-10-0-129-191.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-129-191.ec2.internal' and this object" podUID="1a2357e1-51e1-48c0-8e4e-c554c682c0d2" pod="openshift-ingress/test-curl-7356-140124767986608" Jul 13 19:18:54.279784 ip-10-0-129-191 kubenswrapper[2582]: E0713 19:18:54.279744 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7356-140124767986608\" is forbidden: User \"system:node:ip-10-0-129-191.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-129-191.ec2.internal' and this object" podUID="1a2357e1-51e1-48c0-8e4e-c554c682c0d2" pod="openshift-ingress/test-curl-7356-140124767986608" Jul 13 19:18:54.281107 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:54.281086 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a2357e1-51e1-48c0-8e4e-c554c682c0d2" path="/var/lib/kubelet/pods/1a2357e1-51e1-48c0-8e4e-c554c682c0d2/volumes" Jul 13 19:18:54.611348 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:54.611246 2582 generic.go:356] "Generic (PLEG): container finished" podID="a2db9ab7-07ba-4124-bd3a-27bce4f26cf3" containerID="0a76af4acdf11c18428be3cfec4b33648d0908f397c40488f4a3500808f3a66c" exitCode=0 Jul 13 19:18:54.611348 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:54.611327 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7356-140124767772720" event={"ID":"a2db9ab7-07ba-4124-bd3a-27bce4f26cf3","Type":"ContainerDied","Data":"0a76af4acdf11c18428be3cfec4b33648d0908f397c40488f4a3500808f3a66c"} Jul 13 19:18:55.762059 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:55.762037 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7356-140124767772720" Jul 13 19:18:55.782967 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:55.782941 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7356-140124767772720_a2db9ab7-07ba-4124-bd3a-27bce4f26cf3/test-curl-7356-140124767772720/0.log" Jul 13 19:18:55.807525 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:55.807492 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7356-140124767772720"] Jul 13 19:18:55.811711 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:55.811688 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7356-140124767772720"] Jul 13 19:18:55.865096 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:55.865058 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hpj5x\" (UniqueName: \"kubernetes.io/projected/a2db9ab7-07ba-4124-bd3a-27bce4f26cf3-kube-api-access-hpj5x\") pod \"a2db9ab7-07ba-4124-bd3a-27bce4f26cf3\" (UID: \"a2db9ab7-07ba-4124-bd3a-27bce4f26cf3\") " Jul 13 19:18:55.867301 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:55.867239 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2db9ab7-07ba-4124-bd3a-27bce4f26cf3-kube-api-access-hpj5x" (OuterVolumeSpecName: "kube-api-access-hpj5x") pod "a2db9ab7-07ba-4124-bd3a-27bce4f26cf3" (UID: "a2db9ab7-07ba-4124-bd3a-27bce4f26cf3"). InnerVolumeSpecName "kube-api-access-hpj5x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:18:55.940929 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:55.940892 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7356-140124767772720"] Jul 13 19:18:55.941326 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:55.941314 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2db9ab7-07ba-4124-bd3a-27bce4f26cf3" containerName="test-curl-7356-140124767772720" Jul 13 19:18:55.941369 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:55.941328 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2db9ab7-07ba-4124-bd3a-27bce4f26cf3" containerName="test-curl-7356-140124767772720" Jul 13 19:18:55.941414 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:55.941405 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2db9ab7-07ba-4124-bd3a-27bce4f26cf3" containerName="test-curl-7356-140124767772720" Jul 13 19:18:55.944463 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:55.944448 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7356-140124767772720" Jul 13 19:18:55.949311 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:55.949284 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7356-140124767772720"] Jul 13 19:18:55.966296 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:55.966268 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpj5x\" (UniqueName: \"kubernetes.io/projected/a2db9ab7-07ba-4124-bd3a-27bce4f26cf3-kube-api-access-hpj5x\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:18:56.067300 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:56.067267 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55962\" (UniqueName: \"kubernetes.io/projected/9cd55361-964c-4748-bb8f-aa5b4ce77c32-kube-api-access-55962\") pod \"test-curl-7356-140124767772720\" (UID: \"9cd55361-964c-4748-bb8f-aa5b4ce77c32\") " pod="openshift-ingress/test-curl-7356-140124767772720" Jul 13 19:18:56.168685 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:56.168564 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55962\" (UniqueName: \"kubernetes.io/projected/9cd55361-964c-4748-bb8f-aa5b4ce77c32-kube-api-access-55962\") pod \"test-curl-7356-140124767772720\" (UID: \"9cd55361-964c-4748-bb8f-aa5b4ce77c32\") " pod="openshift-ingress/test-curl-7356-140124767772720" Jul 13 19:18:56.178289 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:56.178265 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55962\" (UniqueName: \"kubernetes.io/projected/9cd55361-964c-4748-bb8f-aa5b4ce77c32-kube-api-access-55962\") pod \"test-curl-7356-140124767772720\" (UID: \"9cd55361-964c-4748-bb8f-aa5b4ce77c32\") " pod="openshift-ingress/test-curl-7356-140124767772720" Jul 13 19:18:56.255332 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:56.255293 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7356-140124767772720" Jul 13 19:18:56.281896 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:56.281863 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2db9ab7-07ba-4124-bd3a-27bce4f26cf3" path="/var/lib/kubelet/pods/a2db9ab7-07ba-4124-bd3a-27bce4f26cf3/volumes" Jul 13 19:18:56.394818 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:56.394790 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7356-140124767772720"] Jul 13 19:18:56.396591 ip-10-0-129-191 kubenswrapper[2582]: W0713 19:18:56.396565 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9cd55361_964c_4748_bb8f_aa5b4ce77c32.slice/crio-90995ae100fc101a41ad657fe7bea18aea08e7c66a0ffd1a1fe77c42dba876e2 WatchSource:0}: Error finding container 90995ae100fc101a41ad657fe7bea18aea08e7c66a0ffd1a1fe77c42dba876e2: Status 404 returned error can't find the container with id 90995ae100fc101a41ad657fe7bea18aea08e7c66a0ffd1a1fe77c42dba876e2 Jul 13 19:18:56.623221 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:56.623191 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7356-140124767772720" Jul 13 19:18:56.623422 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:56.623243 2582 scope.go:117] "RemoveContainer" containerID="0a76af4acdf11c18428be3cfec4b33648d0908f397c40488f4a3500808f3a66c" Jul 13 19:18:56.624492 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:56.624394 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7356-140124767772720" event={"ID":"9cd55361-964c-4748-bb8f-aa5b4ce77c32","Type":"ContainerStarted","Data":"90995ae100fc101a41ad657fe7bea18aea08e7c66a0ffd1a1fe77c42dba876e2"} Jul 13 19:18:57.632170 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:57.632133 2582 generic.go:356] "Generic (PLEG): container finished" podID="9cd55361-964c-4748-bb8f-aa5b4ce77c32" containerID="3a02ac6f98ce94cde6db090946ca6f9513102c6bd150460f379f1f2eddb6b286" exitCode=0 Jul 13 19:18:57.632549 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:57.632225 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7356-140124767772720" event={"ID":"9cd55361-964c-4748-bb8f-aa5b4ce77c32","Type":"ContainerDied","Data":"3a02ac6f98ce94cde6db090946ca6f9513102c6bd150460f379f1f2eddb6b286"} Jul 13 19:18:58.780502 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:58.780477 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7356-140124767772720" Jul 13 19:18:58.805221 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:58.805199 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7356-140124767772720_9cd55361-964c-4748-bb8f-aa5b4ce77c32/test-curl-7356-140124767772720/0.log" Jul 13 19:18:58.834867 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:58.834831 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7356-140124767772720"] Jul 13 19:18:58.839590 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:58.839561 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7356-140124767772720"] Jul 13 19:18:58.893925 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:58.893835 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-55962\" (UniqueName: \"kubernetes.io/projected/9cd55361-964c-4748-bb8f-aa5b4ce77c32-kube-api-access-55962\") pod \"9cd55361-964c-4748-bb8f-aa5b4ce77c32\" (UID: \"9cd55361-964c-4748-bb8f-aa5b4ce77c32\") " Jul 13 19:18:58.895984 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:58.895961 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cd55361-964c-4748-bb8f-aa5b4ce77c32-kube-api-access-55962" (OuterVolumeSpecName: "kube-api-access-55962") pod "9cd55361-964c-4748-bb8f-aa5b4ce77c32" (UID: "9cd55361-964c-4748-bb8f-aa5b4ce77c32"). InnerVolumeSpecName "kube-api-access-55962". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:18:58.968201 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:58.968168 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7356-140124767773728"] Jul 13 19:18:58.968581 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:58.968568 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cd55361-964c-4748-bb8f-aa5b4ce77c32" containerName="test-curl-7356-140124767772720" Jul 13 19:18:58.968661 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:58.968583 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cd55361-964c-4748-bb8f-aa5b4ce77c32" containerName="test-curl-7356-140124767772720" Jul 13 19:18:58.968708 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:58.968679 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cd55361-964c-4748-bb8f-aa5b4ce77c32" containerName="test-curl-7356-140124767772720" Jul 13 19:18:58.971769 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:58.971755 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7356-140124767773728" Jul 13 19:18:58.983232 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:58.983209 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7356-140124767773728"] Jul 13 19:18:58.995201 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:58.995180 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55962\" (UniqueName: \"kubernetes.io/projected/9cd55361-964c-4748-bb8f-aa5b4ce77c32-kube-api-access-55962\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:18:59.096296 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:59.096264 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzhj5\" (UniqueName: \"kubernetes.io/projected/739dace0-042e-43ed-975f-0460443e7eaa-kube-api-access-xzhj5\") pod \"test-curl-7356-140124767773728\" (UID: \"739dace0-042e-43ed-975f-0460443e7eaa\") " pod="openshift-ingress/test-curl-7356-140124767773728" Jul 13 19:18:59.197221 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:59.197178 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzhj5\" (UniqueName: \"kubernetes.io/projected/739dace0-042e-43ed-975f-0460443e7eaa-kube-api-access-xzhj5\") pod \"test-curl-7356-140124767773728\" (UID: \"739dace0-042e-43ed-975f-0460443e7eaa\") " pod="openshift-ingress/test-curl-7356-140124767773728" Jul 13 19:18:59.205970 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:59.205942 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzhj5\" (UniqueName: \"kubernetes.io/projected/739dace0-042e-43ed-975f-0460443e7eaa-kube-api-access-xzhj5\") pod \"test-curl-7356-140124767773728\" (UID: \"739dace0-042e-43ed-975f-0460443e7eaa\") " pod="openshift-ingress/test-curl-7356-140124767773728" Jul 13 19:18:59.280995 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:59.280963 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7356-140124767773728" Jul 13 19:18:59.418896 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:59.418874 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7356-140124767773728"] Jul 13 19:18:59.645128 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:59.645097 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7356-140124767772720" Jul 13 19:18:59.645306 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:59.645136 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="90995ae100fc101a41ad657fe7bea18aea08e7c66a0ffd1a1fe77c42dba876e2" Jul 13 19:18:59.646501 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:18:59.646467 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7356-140124767773728" event={"ID":"739dace0-042e-43ed-975f-0460443e7eaa","Type":"ContainerStarted","Data":"ef5c893d803dc8d32dbd267910a4ff9f048cc565830e2f1e277d6c1617d33752"} Jul 13 19:18:59.656274 ip-10-0-129-191 kubenswrapper[2582]: E0713 19:18:59.656248 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7356-140124767772720\" is forbidden: User \"system:node:ip-10-0-129-191.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-129-191.ec2.internal' and this object" podUID="9cd55361-964c-4748-bb8f-aa5b4ce77c32" pod="openshift-ingress/test-curl-7356-140124767772720" Jul 13 19:19:00.281573 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:00.281532 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cd55361-964c-4748-bb8f-aa5b4ce77c32" path="/var/lib/kubelet/pods/9cd55361-964c-4748-bb8f-aa5b4ce77c32/volumes" Jul 13 19:19:00.284781 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:00.284735 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-7b883e/rate-test-model-7b883e-kserve-7776bcb76c-2mptq" podUID="ed65f12d-f80a-4986-9101-44ce26fc95f8" containerName="main" containerID="cri-o://c89a3cc4d99f607551359bc0a13a38000e3f5aa65b213828236bbd52a49a0a07" gracePeriod=45 Jul 13 19:19:00.537573 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:00.537501 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7b883e/rate-test-model-7b883e-kserve-7776bcb76c-2mptq" Jul 13 19:19:00.611852 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:00.611810 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed65f12d-f80a-4986-9101-44ce26fc95f8-dshm\") pod \"ed65f12d-f80a-4986-9101-44ce26fc95f8\" (UID: \"ed65f12d-f80a-4986-9101-44ce26fc95f8\") " Jul 13 19:19:00.611852 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:00.611851 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed65f12d-f80a-4986-9101-44ce26fc95f8-tls-certs\") pod \"ed65f12d-f80a-4986-9101-44ce26fc95f8\" (UID: \"ed65f12d-f80a-4986-9101-44ce26fc95f8\") " Jul 13 19:19:00.612098 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:00.611891 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed65f12d-f80a-4986-9101-44ce26fc95f8-tmp-dir\") pod \"ed65f12d-f80a-4986-9101-44ce26fc95f8\" (UID: \"ed65f12d-f80a-4986-9101-44ce26fc95f8\") " Jul 13 19:19:00.612098 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:00.611915 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed65f12d-f80a-4986-9101-44ce26fc95f8-home\") pod \"ed65f12d-f80a-4986-9101-44ce26fc95f8\" (UID: \"ed65f12d-f80a-4986-9101-44ce26fc95f8\") " Jul 13 19:19:00.612098 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:00.611953 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed65f12d-f80a-4986-9101-44ce26fc95f8-model-cache\") pod \"ed65f12d-f80a-4986-9101-44ce26fc95f8\" (UID: \"ed65f12d-f80a-4986-9101-44ce26fc95f8\") " Jul 13 19:19:00.612098 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:00.612013 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mwqmt\" (UniqueName: \"kubernetes.io/projected/ed65f12d-f80a-4986-9101-44ce26fc95f8-kube-api-access-mwqmt\") pod \"ed65f12d-f80a-4986-9101-44ce26fc95f8\" (UID: \"ed65f12d-f80a-4986-9101-44ce26fc95f8\") " Jul 13 19:19:00.612303 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:00.612161 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed65f12d-f80a-4986-9101-44ce26fc95f8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ed65f12d-f80a-4986-9101-44ce26fc95f8" (UID: "ed65f12d-f80a-4986-9101-44ce26fc95f8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:19:00.612460 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:00.612434 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed65f12d-f80a-4986-9101-44ce26fc95f8-model-cache" (OuterVolumeSpecName: "model-cache") pod "ed65f12d-f80a-4986-9101-44ce26fc95f8" (UID: "ed65f12d-f80a-4986-9101-44ce26fc95f8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:19:00.612460 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:00.612451 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed65f12d-f80a-4986-9101-44ce26fc95f8-home" (OuterVolumeSpecName: "home") pod "ed65f12d-f80a-4986-9101-44ce26fc95f8" (UID: "ed65f12d-f80a-4986-9101-44ce26fc95f8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:19:00.612789 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:00.612441 2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed65f12d-f80a-4986-9101-44ce26fc95f8-tmp-dir\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:19:00.614104 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:00.614080 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed65f12d-f80a-4986-9101-44ce26fc95f8-dshm" (OuterVolumeSpecName: "dshm") pod "ed65f12d-f80a-4986-9101-44ce26fc95f8" (UID: "ed65f12d-f80a-4986-9101-44ce26fc95f8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:19:00.614415 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:00.614386 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed65f12d-f80a-4986-9101-44ce26fc95f8-kube-api-access-mwqmt" (OuterVolumeSpecName: "kube-api-access-mwqmt") pod "ed65f12d-f80a-4986-9101-44ce26fc95f8" (UID: "ed65f12d-f80a-4986-9101-44ce26fc95f8"). InnerVolumeSpecName "kube-api-access-mwqmt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:19:00.614475 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:00.614414 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed65f12d-f80a-4986-9101-44ce26fc95f8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ed65f12d-f80a-4986-9101-44ce26fc95f8" (UID: "ed65f12d-f80a-4986-9101-44ce26fc95f8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:19:00.654041 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:00.654010 2582 generic.go:356] "Generic (PLEG): container finished" podID="739dace0-042e-43ed-975f-0460443e7eaa" containerID="ea3c6f4d1c1ae2a81040ba4af08f3ca504ba931f97746c37bed5e4adcfb9be10" exitCode=0 Jul 13 19:19:00.654218 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:00.654096 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7356-140124767773728" event={"ID":"739dace0-042e-43ed-975f-0460443e7eaa","Type":"ContainerDied","Data":"ea3c6f4d1c1ae2a81040ba4af08f3ca504ba931f97746c37bed5e4adcfb9be10"} Jul 13 19:19:00.655303 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:00.655280 2582 generic.go:356] "Generic (PLEG): container finished" podID="ed65f12d-f80a-4986-9101-44ce26fc95f8" containerID="c89a3cc4d99f607551359bc0a13a38000e3f5aa65b213828236bbd52a49a0a07" exitCode=0 Jul 13 19:19:00.655454 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:00.655322 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7b883e/rate-test-model-7b883e-kserve-7776bcb76c-2mptq" Jul 13 19:19:00.655454 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:00.655347 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7b883e/rate-test-model-7b883e-kserve-7776bcb76c-2mptq" event={"ID":"ed65f12d-f80a-4986-9101-44ce26fc95f8","Type":"ContainerDied","Data":"c89a3cc4d99f607551359bc0a13a38000e3f5aa65b213828236bbd52a49a0a07"} Jul 13 19:19:00.655454 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:00.655378 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7b883e/rate-test-model-7b883e-kserve-7776bcb76c-2mptq" event={"ID":"ed65f12d-f80a-4986-9101-44ce26fc95f8","Type":"ContainerDied","Data":"35b2d4bee7f067942ff61b3dc9869b04b72cf203ac697aa34a40525a1fc08fb3"} Jul 13 19:19:00.655454 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:00.655397 2582 scope.go:117] "RemoveContainer" containerID="c89a3cc4d99f607551359bc0a13a38000e3f5aa65b213828236bbd52a49a0a07" Jul 13 19:19:00.664459 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:00.664442 2582 scope.go:117] "RemoveContainer" containerID="c89a3cc4d99f607551359bc0a13a38000e3f5aa65b213828236bbd52a49a0a07" Jul 13 19:19:00.664761 ip-10-0-129-191 kubenswrapper[2582]: E0713 19:19:00.664738 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c89a3cc4d99f607551359bc0a13a38000e3f5aa65b213828236bbd52a49a0a07\": container with ID starting with c89a3cc4d99f607551359bc0a13a38000e3f5aa65b213828236bbd52a49a0a07 not found: ID does not exist" containerID="c89a3cc4d99f607551359bc0a13a38000e3f5aa65b213828236bbd52a49a0a07" Jul 13 19:19:00.664826 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:00.664777 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c89a3cc4d99f607551359bc0a13a38000e3f5aa65b213828236bbd52a49a0a07"} err="failed to get container status \"c89a3cc4d99f607551359bc0a13a38000e3f5aa65b213828236bbd52a49a0a07\": rpc error: code = NotFound desc = could not find container \"c89a3cc4d99f607551359bc0a13a38000e3f5aa65b213828236bbd52a49a0a07\": container with ID starting with c89a3cc4d99f607551359bc0a13a38000e3f5aa65b213828236bbd52a49a0a07 not found: ID does not exist" Jul 13 19:19:00.703663 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:00.703631 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7b883e/rate-test-model-7b883e-kserve-7776bcb76c-2mptq"] Jul 13 19:19:00.705508 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:00.705487 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-7b883e/rate-test-model-7b883e-kserve-7776bcb76c-2mptq"] Jul 13 19:19:00.712956 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:00.712935 2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed65f12d-f80a-4986-9101-44ce26fc95f8-dshm\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:19:00.712956 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:00.712956 2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed65f12d-f80a-4986-9101-44ce26fc95f8-tls-certs\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:19:00.713097 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:00.712967 2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed65f12d-f80a-4986-9101-44ce26fc95f8-home\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:19:00.713097 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:00.712978 2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed65f12d-f80a-4986-9101-44ce26fc95f8-model-cache\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:19:00.713097 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:00.712990 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mwqmt\" (UniqueName: \"kubernetes.io/projected/ed65f12d-f80a-4986-9101-44ce26fc95f8-kube-api-access-mwqmt\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:19:01.805277 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:01.805254 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7356-140124767773728" Jul 13 19:19:01.827687 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:01.827663 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7356-140124767773728_739dace0-042e-43ed-975f-0460443e7eaa/test-curl-7356-140124767773728/0.log" Jul 13 19:19:01.852707 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:01.852671 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7356-140124767773728"] Jul 13 19:19:01.856780 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:01.856756 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7356-140124767773728"] Jul 13 19:19:01.925065 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:01.924985 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xzhj5\" (UniqueName: \"kubernetes.io/projected/739dace0-042e-43ed-975f-0460443e7eaa-kube-api-access-xzhj5\") pod \"739dace0-042e-43ed-975f-0460443e7eaa\" (UID: \"739dace0-042e-43ed-975f-0460443e7eaa\") " Jul 13 19:19:01.927153 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:01.927131 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/739dace0-042e-43ed-975f-0460443e7eaa-kube-api-access-xzhj5" (OuterVolumeSpecName: "kube-api-access-xzhj5") pod "739dace0-042e-43ed-975f-0460443e7eaa" (UID: "739dace0-042e-43ed-975f-0460443e7eaa"). InnerVolumeSpecName "kube-api-access-xzhj5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:19:01.984919 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:01.984883 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7356-140124767773728"] Jul 13 19:19:01.985287 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:01.985274 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="739dace0-042e-43ed-975f-0460443e7eaa" containerName="test-curl-7356-140124767773728" Jul 13 19:19:01.985344 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:01.985289 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="739dace0-042e-43ed-975f-0460443e7eaa" containerName="test-curl-7356-140124767773728" Jul 13 19:19:01.985344 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:01.985321 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed65f12d-f80a-4986-9101-44ce26fc95f8" containerName="main" Jul 13 19:19:01.985344 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:01.985330 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed65f12d-f80a-4986-9101-44ce26fc95f8" containerName="main" Jul 13 19:19:01.985452 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:01.985385 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed65f12d-f80a-4986-9101-44ce26fc95f8" containerName="main" Jul 13 19:19:01.985452 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:01.985397 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="739dace0-042e-43ed-975f-0460443e7eaa" containerName="test-curl-7356-140124767773728" Jul 13 19:19:01.988374 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:01.988357 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7356-140124767773728" Jul 13 19:19:01.995862 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:01.995841 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7356-140124767773728"] Jul 13 19:19:02.032382 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:02.027212 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzhj5\" (UniqueName: \"kubernetes.io/projected/739dace0-042e-43ed-975f-0460443e7eaa-kube-api-access-xzhj5\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:19:02.128331 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:02.128295 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjj5d\" (UniqueName: \"kubernetes.io/projected/f361d4b9-ee58-4783-b2c0-957e73228514-kube-api-access-tjj5d\") pod \"test-curl-7356-140124767773728\" (UID: \"f361d4b9-ee58-4783-b2c0-957e73228514\") " pod="openshift-ingress/test-curl-7356-140124767773728" Jul 13 19:19:02.229734 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:02.229696 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjj5d\" (UniqueName: \"kubernetes.io/projected/f361d4b9-ee58-4783-b2c0-957e73228514-kube-api-access-tjj5d\") pod \"test-curl-7356-140124767773728\" (UID: \"f361d4b9-ee58-4783-b2c0-957e73228514\") " pod="openshift-ingress/test-curl-7356-140124767773728" Jul 13 19:19:02.239317 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:02.239282 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjj5d\" (UniqueName: \"kubernetes.io/projected/f361d4b9-ee58-4783-b2c0-957e73228514-kube-api-access-tjj5d\") pod \"test-curl-7356-140124767773728\" (UID: \"f361d4b9-ee58-4783-b2c0-957e73228514\") " pod="openshift-ingress/test-curl-7356-140124767773728" Jul 13 19:19:02.282029 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:02.281996 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="739dace0-042e-43ed-975f-0460443e7eaa" path="/var/lib/kubelet/pods/739dace0-042e-43ed-975f-0460443e7eaa/volumes" Jul 13 19:19:02.282406 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:02.282394 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed65f12d-f80a-4986-9101-44ce26fc95f8" path="/var/lib/kubelet/pods/ed65f12d-f80a-4986-9101-44ce26fc95f8/volumes" Jul 13 19:19:02.298540 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:02.298512 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7356-140124767773728" Jul 13 19:19:02.437765 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:02.437732 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7356-140124767773728"] Jul 13 19:19:02.441164 ip-10-0-129-191 kubenswrapper[2582]: W0713 19:19:02.441128 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf361d4b9_ee58_4783_b2c0_957e73228514.slice/crio-8508bd3cdfda4fdfefbbec3f647d4a2263d24f9ac270f2a685b410a37b27426c WatchSource:0}: Error finding container 8508bd3cdfda4fdfefbbec3f647d4a2263d24f9ac270f2a685b410a37b27426c: Status 404 returned error can't find the container with id 8508bd3cdfda4fdfefbbec3f647d4a2263d24f9ac270f2a685b410a37b27426c Jul 13 19:19:02.669379 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:02.669294 2582 scope.go:117] "RemoveContainer" containerID="ea3c6f4d1c1ae2a81040ba4af08f3ca504ba931f97746c37bed5e4adcfb9be10" Jul 13 19:19:02.669538 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:02.669307 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7356-140124767773728" Jul 13 19:19:02.670809 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:02.670553 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7356-140124767773728" event={"ID":"f361d4b9-ee58-4783-b2c0-957e73228514","Type":"ContainerStarted","Data":"8508bd3cdfda4fdfefbbec3f647d4a2263d24f9ac270f2a685b410a37b27426c"} Jul 13 19:19:03.677691 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:03.677657 2582 generic.go:356] "Generic (PLEG): container finished" podID="f361d4b9-ee58-4783-b2c0-957e73228514" containerID="94a17d124eaf08d756292923a18bc66bcffc50610afe503eb2b6635f4bb400c0" exitCode=0 Jul 13 19:19:03.678083 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:03.677747 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7356-140124767773728" event={"ID":"f361d4b9-ee58-4783-b2c0-957e73228514","Type":"ContainerDied","Data":"94a17d124eaf08d756292923a18bc66bcffc50610afe503eb2b6635f4bb400c0"} Jul 13 19:19:04.836543 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:04.836516 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7356-140124767773728" Jul 13 19:19:04.857491 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:04.857458 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7356-140124767773728_f361d4b9-ee58-4783-b2c0-957e73228514/test-curl-7356-140124767773728/0.log" Jul 13 19:19:04.881806 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:04.881771 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7356-140124767773728"] Jul 13 19:19:04.886727 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:04.886701 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7356-140124767773728"] Jul 13 19:19:04.953900 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:04.953872 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tjj5d\" (UniqueName: \"kubernetes.io/projected/f361d4b9-ee58-4783-b2c0-957e73228514-kube-api-access-tjj5d\") pod \"f361d4b9-ee58-4783-b2c0-957e73228514\" (UID: \"f361d4b9-ee58-4783-b2c0-957e73228514\") " Jul 13 19:19:04.956386 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:04.956351 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f361d4b9-ee58-4783-b2c0-957e73228514-kube-api-access-tjj5d" (OuterVolumeSpecName: "kube-api-access-tjj5d") pod "f361d4b9-ee58-4783-b2c0-957e73228514" (UID: "f361d4b9-ee58-4783-b2c0-957e73228514"). InnerVolumeSpecName "kube-api-access-tjj5d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:19:05.054797 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:05.054763 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjj5d\" (UniqueName: \"kubernetes.io/projected/f361d4b9-ee58-4783-b2c0-957e73228514-kube-api-access-tjj5d\") on node \"ip-10-0-129-191.ec2.internal\" DevicePath \"\"" Jul 13 19:19:05.691731 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:05.691698 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7356-140124767773728" Jul 13 19:19:05.691935 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:05.691743 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8508bd3cdfda4fdfefbbec3f647d4a2263d24f9ac270f2a685b410a37b27426c" Jul 13 19:19:05.701397 ip-10-0-129-191 kubenswrapper[2582]: E0713 19:19:05.701368 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7356-140124767773728\" is forbidden: User \"system:node:ip-10-0-129-191.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-129-191.ec2.internal' and this object" podUID="f361d4b9-ee58-4783-b2c0-957e73228514" pod="openshift-ingress/test-curl-7356-140124767773728" Jul 13 19:19:05.724904 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:05.724873 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:19:05.882500 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:05.882467 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:19:06.281447 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:06.281333 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f361d4b9-ee58-4783-b2c0-957e73228514" path="/var/lib/kubelet/pods/f361d4b9-ee58-4783-b2c0-957e73228514/volumes" Jul 13 19:19:15.628985 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:15.628958 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:19:20.241709 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:20.241671 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:19:26.414951 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:26.414915 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:19:26.485704 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:26.485659 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:19:30.944728 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:30.944679 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:19:31.179178 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:31.179150 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:19:32.005357 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:32.005324 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:19:32.137709 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:32.137674 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:19:35.204105 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:35.204054 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:19:41.791443 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:41.791408 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:19:46.531616 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:46.531570 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:19:52.922755 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:52.922708 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:19:53.022927 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:53.022895 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:19:57.134856 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:57.134822 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:19:57.364515 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:57.364463 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:19:57.846657 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:19:57.846623 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7466b976c5-65jm2_bf8799c3-1a9e-44b3-9ea8-821ce4309b84/authorino/0.log" Jul 13 19:20:03.769103 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:03.769071 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b4789cfbf-tpzzr_e9fbcb61-a250-427c-9ba9-4d5ca449c193/copy-manifests/0.log" Jul 13 19:20:03.776208 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:03.776186 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b4789cfbf-tpzzr_e9fbcb61-a250-427c-9ba9-4d5ca449c193/manager/0.log" Jul 13 19:20:04.140309 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:04.140225 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5648755864-phxl4_36417751-3059-46a2-9b61-d8e1583fd7f8/maas-api/0.log" Jul 13 19:20:04.257290 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:04.257260 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29732805-tnpdt_fa557cf2-3943-470e-99fb-40bc4620162d/cleanup/0.log" Jul 13 19:20:04.379014 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:04.378988 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29732820-cw7x6_2c8f49e3-4619-4e18-a1c3-81abbf2b8115/cleanup/0.log" Jul 13 19:20:04.614884 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:04.614854 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-258c642c-7stgk_6eb5f51c-4adf-4c95-b794-ba8ef7e3ab7c/revoke-keys/0.log" Jul 13 19:20:04.733666 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:04.733638 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-f7c6e726-8x98p_dc6c6f41-6d9a-46d6-ac6f-1a50c3fbdab1/revoke-keys/0.log" Jul 13 19:20:05.868870 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:05.868844 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-srgb8_0474b7cd-3f14-4f74-bdad-60283629dc07/postgres/0.log" Jul 13 19:20:07.167494 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:07.167464 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7466b976c5-65jm2_bf8799c3-1a9e-44b3-9ea8-821ce4309b84/authorino/0.log" Jul 13 19:20:07.403091 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:07.403060 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-5n7cf_ad280742-6869-4e53-8348-b4f3e578d870/manager/0.log" Jul 13 19:20:07.515808 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:07.515786 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-cnls6_2f436467-347d-4bf3-9194-7bf0a7acab6e/kuadrant-console-plugin/0.log" Jul 13 19:20:08.617215 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:08.617188 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66f8768797-j4bdx_cc526317-542a-4c45-81c4-fac064ee4661/kube-auth-proxy/0.log" Jul 13 19:20:09.713883 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:09.713854 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-vt48k_48b1a3bd-df4a-4886-b7ab-3beae9539f93/main/0.log" Jul 13 19:20:09.829428 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:09.829395 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-jmzbd_92869120-8a9e-4054-92f3-e9baa9bd383b/main/0.log" Jul 13 19:20:14.719733 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:14.719642 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:20:14.834718 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:14.834677 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" limit=1024 Jul 13 19:20:17.542555 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:17.542524 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-pg9xz_cfcad9fb-77bc-460d-9ee0-fbfb5573ba07/global-pull-secret-syncer/0.log" Jul 13 19:20:17.602076 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:17.602046 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-7x978_a25a1123-769c-46d1-90a7-ceaf4dc36537/konnectivity-agent/0.log" Jul 13 19:20:17.697488 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:17.697459 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-191.ec2.internal_717de0a249da5500e718fed7ba33a501/haproxy/0.log" Jul 13 19:20:21.742855 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:21.742823 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7466b976c5-65jm2_bf8799c3-1a9e-44b3-9ea8-821ce4309b84/authorino/0.log" Jul 13 19:20:21.836520 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:21.836492 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-5n7cf_ad280742-6869-4e53-8348-b4f3e578d870/manager/0.log" Jul 13 19:20:21.863725 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:21.863698 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-cnls6_2f436467-347d-4bf3-9194-7bf0a7acab6e/kuadrant-console-plugin/0.log" Jul 13 19:20:23.853350 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:23.853240 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_975ed946-1a3d-46ea-82c6-f2f35d35721e/alertmanager/0.log" Jul 13 19:20:23.879946 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:23.879923 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_975ed946-1a3d-46ea-82c6-f2f35d35721e/config-reloader/0.log" Jul 13 19:20:23.908133 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:23.908107 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_975ed946-1a3d-46ea-82c6-f2f35d35721e/kube-rbac-proxy-web/0.log" Jul 13 19:20:23.937734 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:23.937709 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_975ed946-1a3d-46ea-82c6-f2f35d35721e/kube-rbac-proxy/0.log" Jul 13 19:20:23.966433 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:23.966403 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_975ed946-1a3d-46ea-82c6-f2f35d35721e/kube-rbac-proxy-metric/0.log" Jul 13 19:20:23.986844 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:23.986818 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_975ed946-1a3d-46ea-82c6-f2f35d35721e/prom-label-proxy/0.log" Jul 13 19:20:24.010468 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:24.010442 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_975ed946-1a3d-46ea-82c6-f2f35d35721e/init-config-reloader/0.log" Jul 13 19:20:24.121806 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:24.121718 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-8zgr7_4fa44170-99b2-4f9a-be6e-415cc519a3f5/kube-state-metrics/0.log" Jul 13 19:20:24.142180 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:24.142152 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-8zgr7_4fa44170-99b2-4f9a-be6e-415cc519a3f5/kube-rbac-proxy-main/0.log" Jul 13 19:20:24.170707 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:24.170666 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-8zgr7_4fa44170-99b2-4f9a-be6e-415cc519a3f5/kube-rbac-proxy-self/0.log" Jul 13 19:20:24.200572 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:24.200545 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7b5c78f7b5-j2kwg_2fe534d4-9744-4412-a2c1-b58d4db2a2a6/metrics-server/0.log" Jul 13 19:20:24.227732 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:24.227708 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-d2cqx_fb20fd4a-4270-4106-8c3a-330dd18bd732/monitoring-plugin/0.log" Jul 13 19:20:24.431898 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:24.431824 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zd5xm_58ca72a5-ee9b-44f0-9cf2-c503f37d929b/node-exporter/0.log" Jul 13 19:20:24.458675 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:24.458644 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zd5xm_58ca72a5-ee9b-44f0-9cf2-c503f37d929b/kube-rbac-proxy/0.log" Jul 13 19:20:24.483486 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:24.483462 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zd5xm_58ca72a5-ee9b-44f0-9cf2-c503f37d929b/init-textfile/0.log" Jul 13 19:20:24.517030 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:24.517002 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-vrgqz_b6d0ba1f-a0cd-490b-9a6a-1c573bdffc5e/kube-rbac-proxy-main/0.log" Jul 13 19:20:24.543525 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:24.543496 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-vrgqz_b6d0ba1f-a0cd-490b-9a6a-1c573bdffc5e/kube-rbac-proxy-self/0.log" Jul 13 19:20:24.579234 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:24.579205 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-vrgqz_b6d0ba1f-a0cd-490b-9a6a-1c573bdffc5e/openshift-state-metrics/0.log" Jul 13 19:20:24.620597 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:24.620573 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8f9f41cf-286b-4bc6-9523-b364128375aa/prometheus/0.log" Jul 13 19:20:24.645596 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:24.645572 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8f9f41cf-286b-4bc6-9523-b364128375aa/config-reloader/0.log" Jul 13 19:20:24.674385 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:24.674348 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8f9f41cf-286b-4bc6-9523-b364128375aa/thanos-sidecar/0.log" Jul 13 19:20:24.702317 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:24.702292 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8f9f41cf-286b-4bc6-9523-b364128375aa/kube-rbac-proxy-web/0.log" Jul 13 19:20:24.734807 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:24.734779 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8f9f41cf-286b-4bc6-9523-b364128375aa/kube-rbac-proxy/0.log" Jul 13 19:20:24.761776 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:24.761745 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8f9f41cf-286b-4bc6-9523-b364128375aa/kube-rbac-proxy-thanos/0.log" Jul 13 19:20:24.790307 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:24.790279 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8f9f41cf-286b-4bc6-9523-b364128375aa/init-config-reloader/0.log" Jul 13 19:20:24.859507 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:24.859479 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-qc4ml_4923aacc-d13a-4892-9cde-a2425d9f36b4/prometheus-operator/0.log" Jul 13 19:20:24.884572 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:24.884540 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-qc4ml_4923aacc-d13a-4892-9cde-a2425d9f36b4/kube-rbac-proxy/0.log" Jul 13 19:20:24.938051 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:24.938023 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-bzv49_f3baa39d-9895-4c3e-afe0-e3b1633cd340/prometheus-operator-admission-webhook/0.log" Jul 13 19:20:24.998761 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:24.998682 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-856878f6c6-mclbd_b524ba2c-fcaa-437a-9cd5-64512e35cf7e/telemeter-client/0.log" Jul 13 19:20:25.020863 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:25.020837 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-856878f6c6-mclbd_b524ba2c-fcaa-437a-9cd5-64512e35cf7e/reload/0.log" Jul 13 19:20:25.048935 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:25.048907 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-856878f6c6-mclbd_b524ba2c-fcaa-437a-9cd5-64512e35cf7e/kube-rbac-proxy/0.log" Jul 13 19:20:25.124951 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:25.124924 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-77f95c75bb-4mxf7_572dea7e-ab24-47f1-9e10-e099d9e0316e/thanos-query/0.log" Jul 13 19:20:25.153082 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:25.153051 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-77f95c75bb-4mxf7_572dea7e-ab24-47f1-9e10-e099d9e0316e/kube-rbac-proxy-web/0.log" Jul 13 19:20:25.185475 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:25.185449 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-77f95c75bb-4mxf7_572dea7e-ab24-47f1-9e10-e099d9e0316e/kube-rbac-proxy/0.log" Jul 13 19:20:25.231562 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:25.231538 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-77f95c75bb-4mxf7_572dea7e-ab24-47f1-9e10-e099d9e0316e/prom-label-proxy/0.log" Jul 13 19:20:25.269511 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:25.269493 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-77f95c75bb-4mxf7_572dea7e-ab24-47f1-9e10-e099d9e0316e/kube-rbac-proxy-rules/0.log" Jul 13 19:20:25.303019 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:25.302966 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-77f95c75bb-4mxf7_572dea7e-ab24-47f1-9e10-e099d9e0316e/kube-rbac-proxy-metrics/0.log" Jul 13 19:20:27.819942 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:27.819911 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-msvxh/perf-node-gather-daemonset-pjpv7"] Jul 13 19:20:27.820328 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:27.820312 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f361d4b9-ee58-4783-b2c0-957e73228514" containerName="test-curl-7356-140124767773728" Jul 13 19:20:27.820328 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:27.820329 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f361d4b9-ee58-4783-b2c0-957e73228514" containerName="test-curl-7356-140124767773728" Jul 13 19:20:27.820411 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:27.820402 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f361d4b9-ee58-4783-b2c0-957e73228514" containerName="test-curl-7356-140124767773728" Jul 13 19:20:27.823893 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:27.823879 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-pjpv7" Jul 13 19:20:27.828210 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:27.828189 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-msvxh\"/\"default-dockercfg-tkvs5\"" Jul 13 19:20:27.829353 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:27.829335 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-msvxh\"/\"kube-root-ca.crt\"" Jul 13 19:20:27.829574 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:27.829345 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-msvxh\"/\"openshift-service-ca.crt\"" Jul 13 19:20:27.834336 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:27.834316 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-msvxh/perf-node-gather-daemonset-pjpv7"] Jul 13 19:20:27.927581 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:27.927535 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d4179d0c-7055-407e-a378-138637daffed-lib-modules\") pod \"perf-node-gather-daemonset-pjpv7\" (UID: \"d4179d0c-7055-407e-a378-138637daffed\") " pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-pjpv7" Jul 13 19:20:27.927786 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:27.927732 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d4179d0c-7055-407e-a378-138637daffed-podres\") pod \"perf-node-gather-daemonset-pjpv7\" (UID: \"d4179d0c-7055-407e-a378-138637daffed\") " pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-pjpv7" Jul 13 19:20:27.927786 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:27.927779 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6p8b5\" (UniqueName: \"kubernetes.io/projected/d4179d0c-7055-407e-a378-138637daffed-kube-api-access-6p8b5\") pod \"perf-node-gather-daemonset-pjpv7\" (UID: \"d4179d0c-7055-407e-a378-138637daffed\") " pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-pjpv7" Jul 13 19:20:27.927876 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:27.927803 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d4179d0c-7055-407e-a378-138637daffed-proc\") pod \"perf-node-gather-daemonset-pjpv7\" (UID: \"d4179d0c-7055-407e-a378-138637daffed\") " pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-pjpv7" Jul 13 19:20:27.927876 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:27.927829 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d4179d0c-7055-407e-a378-138637daffed-sys\") pod \"perf-node-gather-daemonset-pjpv7\" (UID: \"d4179d0c-7055-407e-a378-138637daffed\") " pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-pjpv7" Jul 13 19:20:28.012641 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:28.012588 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-bggrq_e657c3c1-fb9d-4da0-b88f-6db7f2510f95/download-server/0.log" Jul 13 19:20:28.028363 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:28.028331 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d4179d0c-7055-407e-a378-138637daffed-podres\") pod \"perf-node-gather-daemonset-pjpv7\" (UID: \"d4179d0c-7055-407e-a378-138637daffed\") " pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-pjpv7" Jul 13 19:20:28.028546 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:28.028383 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6p8b5\" (UniqueName: \"kubernetes.io/projected/d4179d0c-7055-407e-a378-138637daffed-kube-api-access-6p8b5\") pod \"perf-node-gather-daemonset-pjpv7\" (UID: \"d4179d0c-7055-407e-a378-138637daffed\") " pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-pjpv7" Jul 13 19:20:28.028546 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:28.028509 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d4179d0c-7055-407e-a378-138637daffed-podres\") pod \"perf-node-gather-daemonset-pjpv7\" (UID: \"d4179d0c-7055-407e-a378-138637daffed\") " pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-pjpv7" Jul 13 19:20:28.028546 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:28.028515 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d4179d0c-7055-407e-a378-138637daffed-proc\") pod \"perf-node-gather-daemonset-pjpv7\" (UID: \"d4179d0c-7055-407e-a378-138637daffed\") " pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-pjpv7" Jul 13 19:20:28.028750 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:28.028558 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d4179d0c-7055-407e-a378-138637daffed-sys\") pod \"perf-node-gather-daemonset-pjpv7\" (UID: \"d4179d0c-7055-407e-a378-138637daffed\") " pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-pjpv7" Jul 13 19:20:28.028750 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:28.028633 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d4179d0c-7055-407e-a378-138637daffed-lib-modules\") pod \"perf-node-gather-daemonset-pjpv7\" (UID: \"d4179d0c-7055-407e-a378-138637daffed\") " pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-pjpv7" Jul 13 19:20:28.028750 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:28.028639 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d4179d0c-7055-407e-a378-138637daffed-proc\") pod \"perf-node-gather-daemonset-pjpv7\" (UID: \"d4179d0c-7055-407e-a378-138637daffed\") " pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-pjpv7" Jul 13 19:20:28.028750 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:28.028709 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d4179d0c-7055-407e-a378-138637daffed-sys\") pod \"perf-node-gather-daemonset-pjpv7\" (UID: \"d4179d0c-7055-407e-a378-138637daffed\") " pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-pjpv7" Jul 13 19:20:28.028885 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:28.028753 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d4179d0c-7055-407e-a378-138637daffed-lib-modules\") pod \"perf-node-gather-daemonset-pjpv7\" (UID: \"d4179d0c-7055-407e-a378-138637daffed\") " pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-pjpv7" Jul 13 19:20:28.037694 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:28.037669 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6p8b5\" (UniqueName: \"kubernetes.io/projected/d4179d0c-7055-407e-a378-138637daffed-kube-api-access-6p8b5\") pod \"perf-node-gather-daemonset-pjpv7\" (UID: \"d4179d0c-7055-407e-a378-138637daffed\") " pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-pjpv7" Jul 13 19:20:28.134550 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:28.134462 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-pjpv7" Jul 13 19:20:28.279534 ip-10-0-129-191 kubenswrapper[2582]: W0713 19:20:28.279507 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd4179d0c_7055_407e_a378_138637daffed.slice/crio-c8d2c5d5f965f710b4dabbc6bd5f12e4a1a38c4a996eedf31af6da8055d735db WatchSource:0}: Error finding container c8d2c5d5f965f710b4dabbc6bd5f12e4a1a38c4a996eedf31af6da8055d735db: Status 404 returned error can't find the container with id c8d2c5d5f965f710b4dabbc6bd5f12e4a1a38c4a996eedf31af6da8055d735db Jul 13 19:20:28.281725 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:28.281700 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-msvxh/perf-node-gather-daemonset-pjpv7"] Jul 13 19:20:29.135533 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:29.135499 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-pjpv7" event={"ID":"d4179d0c-7055-407e-a378-138637daffed","Type":"ContainerStarted","Data":"5d888863c4ba5335f167757467163f9ddcb1d379a7b240b5e7dfd08d60b5b22d"} Jul 13 19:20:29.135952 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:29.135548 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-pjpv7" event={"ID":"d4179d0c-7055-407e-a378-138637daffed","Type":"ContainerStarted","Data":"c8d2c5d5f965f710b4dabbc6bd5f12e4a1a38c4a996eedf31af6da8055d735db"} Jul 13 19:20:29.135952 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:29.135625 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-pjpv7" Jul 13 19:20:29.169590 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:29.169543 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-pjpv7" podStartSLOduration=2.169526006 podStartE2EDuration="2.169526006s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:20:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:20:29.168870254 +0000 UTC m=+2885.452298855" watchObservedRunningTime="2026-07-13 19:20:29.169526006 +0000 UTC m=+2885.452954607" Jul 13 19:20:29.888060 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:29.888022 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-v5z9m_8fcf3f48-d491-4064-a7b6-b860ab39e829/dns/0.log" Jul 13 19:20:29.914551 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:29.914523 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-v5z9m_8fcf3f48-d491-4064-a7b6-b860ab39e829/kube-rbac-proxy/0.log" Jul 13 19:20:30.087691 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:30.087665 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nt9r9_36baa7e8-9898-491d-a961-c364c6e6af25/dns-node-resolver/0.log" Jul 13 19:20:30.808333 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:30.808305 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5qgnw_5fee7e65-59ed-4225-b607-67c9a5218be4/node-ca/0.log" Jul 13 19:20:32.052376 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:32.052344 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66f8768797-j4bdx_cc526317-542a-4c45-81c4-fac064ee4661/kube-auth-proxy/0.log" Jul 13 19:20:33.141515 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:33.141489 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vl6xp_051cc321-d24b-4f45-b2e5-477428647fe3/serve-healthcheck-canary/0.log" Jul 13 19:20:33.804883 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:33.804846 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zc5rl_a8906f23-73c2-4ee7-9202-ec4564ef44d3/kube-rbac-proxy/0.log" Jul 13 19:20:33.827595 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:33.827571 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zc5rl_a8906f23-73c2-4ee7-9202-ec4564ef44d3/exporter/0.log" Jul 13 19:20:33.852783 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:33.852757 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zc5rl_a8906f23-73c2-4ee7-9202-ec4564ef44d3/extractor/0.log" Jul 13 19:20:35.151716 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:35.151693 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-pjpv7" Jul 13 19:20:36.720043 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:36.720017 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b4789cfbf-tpzzr_e9fbcb61-a250-427c-9ba9-4d5ca449c193/manager/0.log" Jul 13 19:20:36.743136 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:36.743118 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b4789cfbf-tpzzr_e9fbcb61-a250-427c-9ba9-4d5ca449c193/copy-manifests/0.log" Jul 13 19:20:38.154093 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:38.154012 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66f8768797-j4bdx_cc526317-542a-4c45-81c4-fac064ee4661/kube-auth-proxy/0.log" Jul 13 19:20:38.165584 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:38.165561 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7f7b869d75-hgcsz_06386055-ce8e-465f-950a-ab247881f51b/manager/0.log" Jul 13 19:20:38.254941 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:38.254903 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-9nwgn_908fd811-9587-407d-92c6-c1d37f9f5163/openshift-lws-operator/0.log" Jul 13 19:20:39.713461 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:39.713431 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66f8768797-j4bdx_cc526317-542a-4c45-81c4-fac064ee4661/kube-auth-proxy/0.log" Jul 13 19:20:44.936652 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:44.936592 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f7lnn_c402e9df-0605-412f-89d2-7aeaa8cb1277/kube-multus-additional-cni-plugins/0.log" Jul 13 19:20:44.958507 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:44.958476 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f7lnn_c402e9df-0605-412f-89d2-7aeaa8cb1277/egress-router-binary-copy/0.log" Jul 13 19:20:44.977986 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:44.977936 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f7lnn_c402e9df-0605-412f-89d2-7aeaa8cb1277/cni-plugins/0.log" Jul 13 19:20:44.998533 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:44.998509 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f7lnn_c402e9df-0605-412f-89d2-7aeaa8cb1277/bond-cni-plugin/0.log" Jul 13 19:20:45.017809 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:45.017779 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f7lnn_c402e9df-0605-412f-89d2-7aeaa8cb1277/routeoverride-cni/0.log" Jul 13 19:20:45.037290 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:45.037268 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f7lnn_c402e9df-0605-412f-89d2-7aeaa8cb1277/whereabouts-cni-bincopy/0.log" Jul 13 19:20:45.056336 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:45.056314 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f7lnn_c402e9df-0605-412f-89d2-7aeaa8cb1277/whereabouts-cni/0.log" Jul 13 19:20:45.292005 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:45.291981 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-w59t6_518044c4-7583-4f6b-b75c-710afb3ca571/kube-multus/0.log" Jul 13 19:20:45.315726 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:45.315691 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-hvwvw_63f8372e-78ca-4580-8d1e-a57593c38dac/network-metrics-daemon/0.log" Jul 13 19:20:45.335220 ip-10-0-129-191 kubenswrapper[2582]: I0713 19:20:45.335201 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-hvwvw_63f8372e-78ca-4580-8d1e-a57593c38dac/kube-rbac-proxy/0.log"