Jul 09 15:00:18.578001 ip-10-0-130-114 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 09 15:00:18.578014 ip-10-0-130-114 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 09 15:00:18.578025 ip-10-0-130-114 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 09 15:00:18.578384 ip-10-0-130-114 systemd[1]: Failed to start Kubernetes Kubelet. Jul 09 15:00:28.675192 ip-10-0-130-114 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 09 15:00:28.675213 ip-10-0-130-114 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 4a8be972e73f418a9b0eefc70ad58ab8 -- Jul 09 15:02:31.794197 ip-10-0-130-114 systemd[1]: Starting Kubernetes Kubelet... Jul 09 15:02:32.205794 ip-10-0-130-114 kubenswrapper[2567]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 15:02:32.205794 ip-10-0-130-114 kubenswrapper[2567]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 09 15:02:32.205794 ip-10-0-130-114 kubenswrapper[2567]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 15:02:32.205794 ip-10-0-130-114 kubenswrapper[2567]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 09 15:02:32.205794 ip-10-0-130-114 kubenswrapper[2567]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 15:02:32.206497 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.205844 2567 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 09 15:02:32.210751 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210733 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 15:02:32.210751 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210750 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 15:02:32.210820 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210754 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 15:02:32.210820 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210758 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 15:02:32.210820 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210761 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 15:02:32.210820 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210765 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 15:02:32.210820 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210768 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 15:02:32.210820 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210773 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 15:02:32.210820 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210777 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 15:02:32.210820 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210781 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 15:02:32.210820 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210785 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 15:02:32.210820 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210788 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 15:02:32.210820 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210791 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 15:02:32.210820 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210794 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 15:02:32.210820 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210797 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 15:02:32.210820 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210800 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 15:02:32.210820 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210804 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 15:02:32.210820 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210809 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 15:02:32.210820 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210812 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 15:02:32.210820 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210815 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 15:02:32.210820 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210818 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 15:02:32.211543 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210821 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 15:02:32.211543 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210824 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 15:02:32.211543 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210827 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 15:02:32.211543 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210830 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 15:02:32.211543 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210833 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 15:02:32.211543 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210836 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 15:02:32.211543 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210838 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 15:02:32.211543 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210841 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 15:02:32.211543 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210843 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 15:02:32.211543 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210846 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 15:02:32.211543 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210849 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 15:02:32.211543 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210851 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 15:02:32.211543 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210854 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 15:02:32.211543 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210856 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 15:02:32.211543 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210859 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 15:02:32.211543 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210861 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 15:02:32.211543 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210865 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 15:02:32.211543 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210867 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 15:02:32.211543 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210870 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 15:02:32.212305 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210872 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 15:02:32.212305 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210885 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 15:02:32.212305 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210890 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 15:02:32.212305 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210894 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 15:02:32.212305 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210898 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 15:02:32.212305 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210902 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 15:02:32.212305 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210906 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 15:02:32.212305 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210911 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 15:02:32.212305 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210914 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 15:02:32.212305 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210917 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 15:02:32.212305 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210920 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 15:02:32.212305 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210922 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 15:02:32.212305 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210925 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 15:02:32.212305 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210928 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 15:02:32.212305 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210931 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 09 15:02:32.212305 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210933 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 15:02:32.212305 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210936 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 15:02:32.212305 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210939 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 15:02:32.212305 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210941 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 15:02:32.212305 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210944 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 15:02:32.213009 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210946 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 15:02:32.213009 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210949 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 15:02:32.213009 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210952 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 15:02:32.213009 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210954 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 15:02:32.213009 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210963 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 15:02:32.213009 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210975 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 15:02:32.213009 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210979 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 15:02:32.213009 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210982 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 15:02:32.213009 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210985 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 15:02:32.213009 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210987 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 15:02:32.213009 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210990 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 15:02:32.213009 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210994 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 15:02:32.213009 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210996 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 15:02:32.213009 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.210999 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 15:02:32.213009 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.211002 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 15:02:32.213009 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.211004 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 15:02:32.213009 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.211007 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 15:02:32.213009 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.211010 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 15:02:32.213009 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.211014 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 15:02:32.213009 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.211017 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 15:02:32.213493 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.211019 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 15:02:32.213493 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.211022 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 15:02:32.213493 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.211026 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 15:02:32.213493 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.211030 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 15:02:32.213493 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.211034 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 15:02:32.213493 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.211036 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 15:02:32.213493 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.211040 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 15:02:32.213493 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.211043 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 15:02:32.213493 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.211047 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 15:02:32.213493 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.211050 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 15:02:32.213493 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.211053 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 15:02:32.213493 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.211056 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 15:02:32.213493 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.211059 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 15:02:32.213493 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.211061 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 15:02:32.213493 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.211064 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 15:02:32.213493 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.211066 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 15:02:32.213493 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.211068 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 15:02:32.213493 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.211071 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 15:02:32.213493 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.211074 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 15:02:32.213493 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213407 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 15:02:32.213994 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213420 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 15:02:32.213994 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213424 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 15:02:32.213994 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213427 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 15:02:32.213994 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213430 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 15:02:32.213994 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213433 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 15:02:32.213994 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213436 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 15:02:32.213994 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213439 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 15:02:32.213994 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213442 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 15:02:32.213994 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213445 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 15:02:32.213994 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213450 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 15:02:32.213994 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213455 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 15:02:32.213994 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213460 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 15:02:32.213994 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213464 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 15:02:32.213994 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213467 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 15:02:32.213994 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213470 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 15:02:32.213994 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213472 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 15:02:32.213994 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213475 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 15:02:32.213994 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213478 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 15:02:32.213994 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213480 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 15:02:32.213994 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213483 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 15:02:32.214477 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213485 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 15:02:32.214477 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213488 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 15:02:32.214477 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213492 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 15:02:32.214477 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213497 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 15:02:32.214477 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213500 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 15:02:32.214477 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213503 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 15:02:32.214477 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213506 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 15:02:32.214477 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213508 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 15:02:32.214477 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213511 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 15:02:32.214477 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213514 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 15:02:32.214477 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213520 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 15:02:32.214477 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213523 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 15:02:32.214477 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213527 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 15:02:32.214477 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213535 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 15:02:32.214477 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213540 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 09 15:02:32.214477 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213543 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 15:02:32.214477 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213546 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 15:02:32.214477 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213549 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 15:02:32.214477 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213552 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 15:02:32.214477 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213554 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 15:02:32.214974 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213557 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 15:02:32.214974 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213560 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 15:02:32.214974 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213562 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 15:02:32.214974 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213565 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 15:02:32.214974 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213568 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 15:02:32.214974 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213570 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 15:02:32.214974 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213574 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 15:02:32.214974 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213576 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 15:02:32.214974 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213579 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 15:02:32.214974 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213581 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 15:02:32.214974 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213584 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 15:02:32.214974 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213587 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 15:02:32.214974 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213589 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 15:02:32.214974 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213592 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 15:02:32.214974 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213595 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 15:02:32.214974 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213597 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 15:02:32.214974 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213600 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 15:02:32.214974 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213602 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 15:02:32.214974 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213606 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 15:02:32.215437 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213610 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 15:02:32.215437 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213615 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 15:02:32.215437 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213619 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 15:02:32.215437 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213622 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 15:02:32.215437 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213625 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 15:02:32.215437 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213627 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 15:02:32.215437 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213632 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 15:02:32.215437 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213635 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 15:02:32.215437 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213638 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 15:02:32.215437 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213641 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 15:02:32.215437 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213643 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 15:02:32.215437 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213646 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 15:02:32.215437 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213648 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 15:02:32.215437 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213652 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 15:02:32.215437 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213657 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 15:02:32.215437 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213660 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 15:02:32.215437 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213663 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 15:02:32.215437 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213666 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 15:02:32.215437 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213669 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 15:02:32.215946 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213672 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 15:02:32.215946 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213675 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 15:02:32.215946 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213677 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 15:02:32.215946 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213681 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 15:02:32.215946 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213685 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 15:02:32.215946 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213690 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 15:02:32.215946 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213694 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 15:02:32.215946 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213698 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 15:02:32.215946 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213702 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 15:02:32.215946 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213704 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 15:02:32.215946 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213707 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 15:02:32.215946 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213709 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 15:02:32.215946 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213712 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 15:02:32.215946 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213715 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 15:02:32.215946 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213718 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 15:02:32.215946 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213720 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 15:02:32.215946 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213724 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 15:02:32.215946 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213726 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 15:02:32.215946 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213729 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 15:02:32.216415 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.213733 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 15:02:32.216415 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214175 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 15:02:32.216415 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214181 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 15:02:32.216415 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214184 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 15:02:32.216415 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214187 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 15:02:32.216415 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214190 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 15:02:32.216415 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214193 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 15:02:32.216415 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214196 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 15:02:32.216415 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214198 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 15:02:32.216415 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214201 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 15:02:32.216415 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214203 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 15:02:32.216415 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214206 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 15:02:32.216415 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214209 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 15:02:32.216415 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214211 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 15:02:32.216415 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214214 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 15:02:32.216415 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214216 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 15:02:32.216415 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214219 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 15:02:32.216415 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214223 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 15:02:32.216415 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214226 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 15:02:32.216415 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214229 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 15:02:32.216924 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214231 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 15:02:32.216924 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214234 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 15:02:32.216924 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214236 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 15:02:32.216924 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214240 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 15:02:32.216924 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214244 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 15:02:32.216924 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214249 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 15:02:32.216924 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214253 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 15:02:32.216924 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214256 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 15:02:32.216924 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214259 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 15:02:32.216924 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214262 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 15:02:32.216924 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214266 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 15:02:32.216924 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214269 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 15:02:32.216924 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214273 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 15:02:32.216924 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214275 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 15:02:32.216924 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214278 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 15:02:32.216924 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214280 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 15:02:32.216924 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214283 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 15:02:32.216924 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214285 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 15:02:32.216924 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214288 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 15:02:32.216924 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214290 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 15:02:32.217445 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214293 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 15:02:32.217445 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214295 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 15:02:32.217445 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214298 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 15:02:32.217445 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214300 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 15:02:32.217445 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214303 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 15:02:32.217445 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214305 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 15:02:32.217445 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214308 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 15:02:32.217445 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214311 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 15:02:32.217445 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214314 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 15:02:32.217445 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214316 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 15:02:32.217445 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214321 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 15:02:32.217445 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214324 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 15:02:32.217445 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214328 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 15:02:32.217445 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214333 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 15:02:32.217445 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214336 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 15:02:32.217445 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214339 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 15:02:32.217445 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214341 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 15:02:32.217445 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214344 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 15:02:32.217445 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214346 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 15:02:32.217445 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214349 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 15:02:32.217943 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214351 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 15:02:32.217943 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214354 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 15:02:32.217943 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214356 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 15:02:32.217943 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214359 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 15:02:32.217943 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214362 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 15:02:32.217943 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214364 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 15:02:32.217943 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214369 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 15:02:32.217943 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214373 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 15:02:32.217943 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214376 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 15:02:32.217943 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214378 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 15:02:32.217943 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214381 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 15:02:32.217943 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214383 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 15:02:32.217943 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214386 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 15:02:32.217943 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214389 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 15:02:32.217943 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214392 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 15:02:32.217943 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214395 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 15:02:32.217943 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214398 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 15:02:32.217943 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214402 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 15:02:32.217943 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214406 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 15:02:32.218409 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214410 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 15:02:32.218409 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214414 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 15:02:32.218409 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214417 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 15:02:32.218409 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214419 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 15:02:32.218409 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214423 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 15:02:32.218409 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214426 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 15:02:32.218409 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214429 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 15:02:32.218409 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214433 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 15:02:32.218409 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214436 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 09 15:02:32.218409 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214439 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 15:02:32.218409 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214442 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 15:02:32.218409 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214445 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 15:02:32.218409 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214448 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 15:02:32.218409 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214450 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 15:02:32.218409 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214453 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 15:02:32.218409 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214455 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 15:02:32.218409 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214458 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 15:02:32.218409 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214461 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 15:02:32.218409 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214463 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 15:02:32.218904 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214466 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 15:02:32.218904 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.214468 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 15:02:32.218904 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214546 2567 flags.go:64] FLAG: --address="0.0.0.0" Jul 09 15:02:32.218904 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214554 2567 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 09 15:02:32.218904 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214565 2567 flags.go:64] FLAG: --anonymous-auth="true" Jul 09 15:02:32.218904 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214573 2567 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 09 15:02:32.218904 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214577 2567 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 09 15:02:32.218904 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214581 2567 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 09 15:02:32.218904 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214585 2567 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 09 15:02:32.218904 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214590 2567 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 09 15:02:32.218904 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214593 2567 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 09 15:02:32.218904 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214596 2567 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 09 15:02:32.218904 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214599 2567 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 09 15:02:32.218904 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214603 2567 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 09 15:02:32.218904 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214606 2567 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 09 15:02:32.218904 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214609 2567 flags.go:64] FLAG: --cgroup-root="" Jul 09 15:02:32.218904 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214613 2567 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 09 15:02:32.218904 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214616 2567 flags.go:64] FLAG: --client-ca-file="" Jul 09 15:02:32.218904 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214620 2567 flags.go:64] FLAG: --cloud-provider="external" Jul 09 15:02:32.218904 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214625 2567 flags.go:64] FLAG: --cluster-dns="[]" Jul 09 15:02:32.218904 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214630 2567 flags.go:64] FLAG: --cluster-domain="" Jul 09 15:02:32.218904 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214633 2567 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 09 15:02:32.218904 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214636 2567 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 09 15:02:32.218904 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214641 2567 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 09 15:02:32.219487 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214647 2567 flags.go:64] FLAG: --container-log-max-files="5" Jul 09 15:02:32.219487 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214653 2567 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 09 15:02:32.219487 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214656 2567 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 09 15:02:32.219487 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214659 2567 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 09 15:02:32.219487 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214663 2567 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 09 15:02:32.219487 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214666 2567 flags.go:64] FLAG: --contention-profiling="false" Jul 09 15:02:32.219487 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214670 2567 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 09 15:02:32.219487 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214673 2567 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 09 15:02:32.219487 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214676 2567 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 09 15:02:32.219487 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214679 2567 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 09 15:02:32.219487 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214683 2567 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 09 15:02:32.219487 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214686 2567 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 09 15:02:32.219487 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214689 2567 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 09 15:02:32.219487 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214692 2567 flags.go:64] FLAG: --enable-load-reader="false" Jul 09 15:02:32.219487 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214695 2567 flags.go:64] FLAG: --enable-server="true" Jul 09 15:02:32.219487 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214697 2567 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 09 15:02:32.219487 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214703 2567 flags.go:64] FLAG: --event-burst="100" Jul 09 15:02:32.219487 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214706 2567 flags.go:64] FLAG: --event-qps="50" Jul 09 15:02:32.219487 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214708 2567 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 09 15:02:32.219487 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214711 2567 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 09 15:02:32.219487 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214714 2567 flags.go:64] FLAG: --eviction-hard="" Jul 09 15:02:32.219487 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214720 2567 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 09 15:02:32.219487 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214725 2567 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 09 15:02:32.219487 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214730 2567 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 09 15:02:32.219487 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214734 2567 flags.go:64] FLAG: --eviction-soft="" Jul 09 15:02:32.220110 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214737 2567 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 09 15:02:32.220110 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214740 2567 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 09 15:02:32.220110 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214745 2567 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 09 15:02:32.220110 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214748 2567 flags.go:64] FLAG: --experimental-mounter-path="" Jul 09 15:02:32.220110 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214751 2567 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 09 15:02:32.220110 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214755 2567 flags.go:64] FLAG: --fail-swap-on="true" Jul 09 15:02:32.220110 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214758 2567 flags.go:64] FLAG: --feature-gates="" Jul 09 15:02:32.220110 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214762 2567 flags.go:64] FLAG: --file-check-frequency="20s" Jul 09 15:02:32.220110 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214765 2567 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 09 15:02:32.220110 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214768 2567 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 09 15:02:32.220110 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214771 2567 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 09 15:02:32.220110 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214774 2567 flags.go:64] FLAG: --healthz-port="10248" Jul 09 15:02:32.220110 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214777 2567 flags.go:64] FLAG: --help="false" Jul 09 15:02:32.220110 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214780 2567 flags.go:64] FLAG: --hostname-override="ip-10-0-130-114.ec2.internal" Jul 09 15:02:32.220110 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214783 2567 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 09 15:02:32.220110 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214786 2567 flags.go:64] FLAG: --http-check-frequency="20s" Jul 09 15:02:32.220110 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214789 2567 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 09 15:02:32.220110 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214792 2567 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 09 15:02:32.220110 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214796 2567 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 09 15:02:32.220110 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214801 2567 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 09 15:02:32.220110 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214806 2567 flags.go:64] FLAG: --image-service-endpoint="" Jul 09 15:02:32.220110 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214810 2567 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 09 15:02:32.220110 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214814 2567 flags.go:64] FLAG: --kube-api-burst="100" Jul 09 15:02:32.220692 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214817 2567 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 09 15:02:32.220692 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214820 2567 flags.go:64] FLAG: --kube-api-qps="50" Jul 09 15:02:32.220692 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214823 2567 flags.go:64] FLAG: --kube-reserved="" Jul 09 15:02:32.220692 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214826 2567 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 09 15:02:32.220692 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214828 2567 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 09 15:02:32.220692 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214831 2567 flags.go:64] FLAG: --kubelet-cgroups="" Jul 09 15:02:32.220692 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214834 2567 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 09 15:02:32.220692 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214837 2567 flags.go:64] FLAG: --lock-file="" Jul 09 15:02:32.220692 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214840 2567 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 09 15:02:32.220692 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214842 2567 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 09 15:02:32.220692 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214846 2567 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 09 15:02:32.220692 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214854 2567 flags.go:64] FLAG: --log-json-split-stream="false" Jul 09 15:02:32.220692 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214857 2567 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 09 15:02:32.220692 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214860 2567 flags.go:64] FLAG: --log-text-split-stream="false" Jul 09 15:02:32.220692 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214864 2567 flags.go:64] FLAG: --logging-format="text" Jul 09 15:02:32.220692 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214867 2567 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 09 15:02:32.220692 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214871 2567 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 09 15:02:32.220692 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214889 2567 flags.go:64] FLAG: --manifest-url="" Jul 09 15:02:32.220692 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214894 2567 flags.go:64] FLAG: --manifest-url-header="" Jul 09 15:02:32.220692 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214899 2567 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 09 15:02:32.220692 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214902 2567 flags.go:64] FLAG: --max-open-files="1000000" Jul 09 15:02:32.220692 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214906 2567 flags.go:64] FLAG: --max-pods="110" Jul 09 15:02:32.220692 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214909 2567 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 09 15:02:32.220692 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214913 2567 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 09 15:02:32.220692 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214915 2567 flags.go:64] FLAG: --memory-manager-policy="None" Jul 09 15:02:32.221315 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214918 2567 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 09 15:02:32.221315 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214921 2567 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 09 15:02:32.221315 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214924 2567 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 09 15:02:32.221315 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214927 2567 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 09 15:02:32.221315 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214934 2567 flags.go:64] FLAG: --node-status-max-images="50" Jul 09 15:02:32.221315 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214937 2567 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 09 15:02:32.221315 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214940 2567 flags.go:64] FLAG: --oom-score-adj="-999" Jul 09 15:02:32.221315 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214943 2567 flags.go:64] FLAG: --pod-cidr="" Jul 09 15:02:32.221315 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214946 2567 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 09 15:02:32.221315 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214952 2567 flags.go:64] FLAG: --pod-manifest-path="" Jul 09 15:02:32.221315 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214956 2567 flags.go:64] FLAG: --pod-max-pids="-1" Jul 09 15:02:32.221315 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214961 2567 flags.go:64] FLAG: --pods-per-core="0" Jul 09 15:02:32.221315 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214966 2567 flags.go:64] FLAG: --port="10250" Jul 09 15:02:32.221315 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214971 2567 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 09 15:02:32.221315 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214974 2567 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0eeb8eaae8837c1e0" Jul 09 15:02:32.221315 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214977 2567 flags.go:64] FLAG: --qos-reserved="" Jul 09 15:02:32.221315 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214980 2567 flags.go:64] FLAG: --read-only-port="10255" Jul 09 15:02:32.221315 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214984 2567 flags.go:64] FLAG: --register-node="true" Jul 09 15:02:32.221315 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214987 2567 flags.go:64] FLAG: --register-with-taints="" Jul 09 15:02:32.221315 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214992 2567 flags.go:64] FLAG: --registry-burst="10" Jul 09 15:02:32.221315 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.214997 2567 flags.go:64] FLAG: --registry-qps="5" Jul 09 15:02:32.221315 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.215001 2567 flags.go:64] FLAG: --reserved-cpus="" Jul 09 15:02:32.221315 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.215004 2567 flags.go:64] FLAG: --reserved-memory="" Jul 09 15:02:32.221315 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.215008 2567 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 09 15:02:32.221315 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.215011 2567 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 09 15:02:32.222029 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.215014 2567 flags.go:64] FLAG: --rotate-certificates="false" Jul 09 15:02:32.222029 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.215017 2567 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 09 15:02:32.222029 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.215020 2567 flags.go:64] FLAG: --runonce="false" Jul 09 15:02:32.222029 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.215022 2567 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 09 15:02:32.222029 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.215026 2567 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 09 15:02:32.222029 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.215029 2567 flags.go:64] FLAG: --seccomp-default="false" Jul 09 15:02:32.222029 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.215032 2567 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 09 15:02:32.222029 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.215036 2567 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 09 15:02:32.222029 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.215041 2567 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 09 15:02:32.222029 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.215046 2567 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 09 15:02:32.222029 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.215051 2567 flags.go:64] FLAG: --storage-driver-password="root" Jul 09 15:02:32.222029 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.215054 2567 flags.go:64] FLAG: --storage-driver-secure="false" Jul 09 15:02:32.222029 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.215057 2567 flags.go:64] FLAG: --storage-driver-table="stats" Jul 09 15:02:32.222029 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.215060 2567 flags.go:64] FLAG: --storage-driver-user="root" Jul 09 15:02:32.222029 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.215063 2567 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 09 15:02:32.222029 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.215066 2567 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 09 15:02:32.222029 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.215069 2567 flags.go:64] FLAG: --system-cgroups="" Jul 09 15:02:32.222029 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.215072 2567 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 09 15:02:32.222029 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.215077 2567 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 09 15:02:32.222029 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.215080 2567 flags.go:64] FLAG: --tls-cert-file="" Jul 09 15:02:32.222029 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.215082 2567 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 09 15:02:32.222029 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.215086 2567 flags.go:64] FLAG: --tls-min-version="" Jul 09 15:02:32.222029 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.215089 2567 flags.go:64] FLAG: --tls-private-key-file="" Jul 09 15:02:32.222029 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.215092 2567 flags.go:64] FLAG: --topology-manager-policy="none" Jul 09 15:02:32.222029 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.215095 2567 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 09 15:02:32.222679 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.215097 2567 flags.go:64] FLAG: --topology-manager-scope="container" Jul 09 15:02:32.222679 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.215102 2567 flags.go:64] FLAG: --v="2" Jul 09 15:02:32.222679 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.215106 2567 flags.go:64] FLAG: --version="false" Jul 09 15:02:32.222679 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.215113 2567 flags.go:64] FLAG: --vmodule="" Jul 09 15:02:32.222679 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.215119 2567 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 09 15:02:32.222679 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.215124 2567 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 09 15:02:32.222679 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215251 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 15:02:32.222679 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215255 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 15:02:32.222679 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215258 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 15:02:32.222679 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215261 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 15:02:32.222679 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215263 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 15:02:32.222679 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215267 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 15:02:32.222679 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215270 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 15:02:32.222679 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215275 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 15:02:32.222679 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215279 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 15:02:32.222679 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215283 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 15:02:32.222679 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215287 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 15:02:32.222679 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215290 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 15:02:32.222679 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215293 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 15:02:32.222679 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215296 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 15:02:32.222679 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215298 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 15:02:32.223209 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215301 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 15:02:32.223209 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215305 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 15:02:32.223209 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215309 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 15:02:32.223209 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215312 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 15:02:32.223209 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215316 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 15:02:32.223209 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215319 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 15:02:32.223209 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215323 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 15:02:32.223209 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215326 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 15:02:32.223209 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215329 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 15:02:32.223209 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215332 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 15:02:32.223209 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215334 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 15:02:32.223209 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215337 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 15:02:32.223209 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215341 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 15:02:32.223209 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215344 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 15:02:32.223209 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215348 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 15:02:32.223209 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215353 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 15:02:32.223209 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215357 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 15:02:32.223209 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215362 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 15:02:32.223209 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215366 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 15:02:32.223672 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215369 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 15:02:32.223672 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215371 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 15:02:32.223672 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215374 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 15:02:32.223672 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215376 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 15:02:32.223672 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215379 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 15:02:32.223672 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215381 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 15:02:32.223672 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215384 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 15:02:32.223672 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215387 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 15:02:32.223672 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215389 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 15:02:32.223672 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215392 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 15:02:32.223672 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215396 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 15:02:32.223672 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215399 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 15:02:32.223672 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215402 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 15:02:32.223672 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215405 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 15:02:32.223672 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215407 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 15:02:32.223672 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215410 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 15:02:32.223672 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215413 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 15:02:32.223672 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215415 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 15:02:32.223672 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215418 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 15:02:32.224166 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215420 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 15:02:32.224166 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215423 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 15:02:32.224166 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215425 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 15:02:32.224166 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215428 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 15:02:32.224166 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215432 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 15:02:32.224166 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215436 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 15:02:32.224166 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215442 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 15:02:32.224166 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215447 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 15:02:32.224166 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215451 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 15:02:32.224166 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215454 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 15:02:32.224166 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215457 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 15:02:32.224166 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215460 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 15:02:32.224166 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215463 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 15:02:32.224166 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215465 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 15:02:32.224166 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215468 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 15:02:32.224166 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215470 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 15:02:32.224166 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215473 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 15:02:32.224166 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215476 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 15:02:32.224166 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215478 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 15:02:32.224166 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215480 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 15:02:32.224644 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215483 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 15:02:32.224644 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215485 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 15:02:32.224644 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215488 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 15:02:32.224644 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215491 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 15:02:32.224644 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215493 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 15:02:32.224644 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215495 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 15:02:32.224644 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215498 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 15:02:32.224644 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215501 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 15:02:32.224644 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215504 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 15:02:32.224644 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215506 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 15:02:32.224644 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215509 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 15:02:32.224644 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215514 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 09 15:02:32.224644 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215518 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 15:02:32.224644 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215522 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 15:02:32.224644 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215527 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 15:02:32.224644 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215530 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 15:02:32.224644 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215532 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 15:02:32.224644 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215535 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 15:02:32.224644 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215538 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 15:02:32.224644 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215541 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 15:02:32.225147 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215545 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 15:02:32.225147 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215547 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 15:02:32.225147 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215550 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 15:02:32.225147 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215553 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 15:02:32.225147 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215556 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 15:02:32.225147 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.215558 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 15:02:32.225147 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.216077 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 15:02:32.225147 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.222949 2567 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 09 15:02:32.225147 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.222969 2567 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 09 15:02:32.225147 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.223814 2567 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 09 15:02:32.225147 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.223821 2567 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 09 15:02:32.225147 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.223895 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 15:02:32.225147 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.223902 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 15:02:32.225147 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.223905 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 15:02:32.225147 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.223908 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 15:02:32.225533 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.223911 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 15:02:32.225533 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.223914 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 15:02:32.225533 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.223917 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 15:02:32.225533 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.223920 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 15:02:32.225533 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.223922 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 15:02:32.225533 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.223925 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 15:02:32.225533 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.223928 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 15:02:32.225533 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.223930 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 15:02:32.225533 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.223935 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 15:02:32.225533 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.223938 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 15:02:32.225533 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.223942 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 15:02:32.225533 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.223945 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 15:02:32.225533 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.223948 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 15:02:32.225533 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.223951 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 15:02:32.225533 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.223954 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 15:02:32.225533 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.223957 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 15:02:32.225533 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.223959 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 15:02:32.225533 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.223962 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 15:02:32.225533 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.223964 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 15:02:32.225533 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.223967 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 15:02:32.226025 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.223970 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 15:02:32.226025 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.223972 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 15:02:32.226025 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.223975 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 15:02:32.226025 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.223977 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 15:02:32.226025 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.223980 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 15:02:32.226025 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.223984 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 15:02:32.226025 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.223988 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 15:02:32.226025 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.223990 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 15:02:32.226025 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.223993 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 15:02:32.226025 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.223996 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 15:02:32.226025 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.223998 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 15:02:32.226025 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224001 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 15:02:32.226025 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224003 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 15:02:32.226025 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224006 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 15:02:32.226025 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224009 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 15:02:32.226025 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224011 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 15:02:32.226025 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224014 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 15:02:32.226025 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224017 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 15:02:32.226025 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224019 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 15:02:32.226474 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224022 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 15:02:32.226474 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224025 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 15:02:32.226474 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224028 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 15:02:32.226474 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224031 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 15:02:32.226474 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224033 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 15:02:32.226474 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224036 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 15:02:32.226474 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224038 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 15:02:32.226474 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224041 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 15:02:32.226474 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224044 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 15:02:32.226474 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224047 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 15:02:32.226474 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224049 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 15:02:32.226474 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224052 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 15:02:32.226474 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224054 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 15:02:32.226474 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224057 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 15:02:32.226474 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224059 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 15:02:32.226474 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224062 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 15:02:32.226474 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224064 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 15:02:32.226474 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224067 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 15:02:32.226474 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224070 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 15:02:32.226474 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224074 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 15:02:32.227001 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224077 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 15:02:32.227001 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224080 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 15:02:32.227001 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224083 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 15:02:32.227001 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224086 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 15:02:32.227001 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224091 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 15:02:32.227001 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224093 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 09 15:02:32.227001 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224096 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 15:02:32.227001 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224098 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 15:02:32.227001 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224101 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 15:02:32.227001 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224103 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 15:02:32.227001 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224106 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 15:02:32.227001 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224108 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 15:02:32.227001 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224111 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 15:02:32.227001 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224114 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 15:02:32.227001 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224116 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 15:02:32.227001 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224119 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 15:02:32.227001 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224121 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 15:02:32.227001 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224124 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 15:02:32.227001 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224127 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 15:02:32.227001 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224129 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 15:02:32.227532 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224132 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 15:02:32.227532 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224134 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 15:02:32.227532 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224137 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 15:02:32.227532 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224141 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 15:02:32.227532 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224144 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 15:02:32.227532 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224147 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 15:02:32.227532 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224150 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 15:02:32.227532 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224153 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 15:02:32.227532 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224155 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 15:02:32.227532 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224159 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 15:02:32.227532 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224163 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 15:02:32.227532 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224166 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 15:02:32.227532 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224169 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 15:02:32.227532 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224172 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 15:02:32.227532 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224175 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 15:02:32.227532 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224178 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 15:02:32.227936 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.224183 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 15:02:32.227936 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224304 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 15:02:32.227936 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224309 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 15:02:32.227936 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224312 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 15:02:32.227936 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224315 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 15:02:32.227936 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224318 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 15:02:32.227936 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224321 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 15:02:32.227936 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224324 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 15:02:32.227936 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224327 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 15:02:32.227936 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224330 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 15:02:32.227936 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224332 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 15:02:32.227936 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224335 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 15:02:32.227936 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224338 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 15:02:32.227936 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224341 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 15:02:32.227936 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224343 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 15:02:32.228328 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224346 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 15:02:32.228328 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224349 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 15:02:32.228328 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224351 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 15:02:32.228328 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224354 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 15:02:32.228328 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224356 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 15:02:32.228328 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224359 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 15:02:32.228328 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224362 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 15:02:32.228328 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224364 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 15:02:32.228328 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224367 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 15:02:32.228328 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224369 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 15:02:32.228328 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224372 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 15:02:32.228328 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224374 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 15:02:32.228328 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224379 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 15:02:32.228328 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224382 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 15:02:32.228328 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224386 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 15:02:32.228328 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224388 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 15:02:32.228328 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224392 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 15:02:32.228328 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224394 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 15:02:32.228328 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224397 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 15:02:32.228328 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224399 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 15:02:32.229164 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224402 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 15:02:32.229164 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224405 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 15:02:32.229164 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224407 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 15:02:32.229164 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224410 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 15:02:32.229164 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224412 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 15:02:32.229164 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224415 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 15:02:32.229164 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224417 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 15:02:32.229164 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224420 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 15:02:32.229164 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224422 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 15:02:32.229164 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224425 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 15:02:32.229164 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224427 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 15:02:32.229164 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224430 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 15:02:32.229164 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224433 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 15:02:32.229164 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224435 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 15:02:32.229164 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224438 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 15:02:32.229164 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224440 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 15:02:32.229164 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224443 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 09 15:02:32.229164 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224445 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 15:02:32.229164 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224448 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 15:02:32.229164 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224450 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 15:02:32.229829 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224453 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 15:02:32.229829 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224455 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 15:02:32.229829 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224458 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 15:02:32.229829 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224461 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 15:02:32.229829 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224464 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 15:02:32.229829 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224466 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 15:02:32.229829 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224469 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 15:02:32.229829 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224472 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 15:02:32.229829 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224475 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 15:02:32.229829 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224478 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 15:02:32.229829 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224481 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 15:02:32.229829 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224483 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 15:02:32.229829 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224487 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 15:02:32.229829 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224490 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 15:02:32.229829 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224493 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 15:02:32.229829 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224496 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 15:02:32.229829 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224499 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 15:02:32.229829 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224501 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 15:02:32.229829 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224503 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 15:02:32.230412 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224506 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 15:02:32.230412 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224508 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 15:02:32.230412 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224511 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 15:02:32.230412 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224513 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 15:02:32.230412 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224516 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 15:02:32.230412 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224518 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 15:02:32.230412 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224521 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 15:02:32.230412 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224524 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 15:02:32.230412 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224526 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 15:02:32.230412 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224530 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 15:02:32.230412 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224533 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 15:02:32.230412 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224537 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 15:02:32.230412 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224539 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 15:02:32.230412 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224542 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 15:02:32.230412 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224545 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 15:02:32.230412 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224548 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 15:02:32.230412 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224550 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 15:02:32.230412 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224553 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 15:02:32.230412 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224556 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 15:02:32.230412 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224558 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 15:02:32.230971 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224561 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 15:02:32.230971 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224565 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 15:02:32.230971 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224567 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 15:02:32.230971 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224570 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 15:02:32.230971 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224572 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 15:02:32.230971 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:32.224575 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 15:02:32.230971 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.224580 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 15:02:32.230971 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.224735 2567 server.go:962] "Client rotation is on, will bootstrap in background" Jul 09 15:02:32.230971 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.226870 2567 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 09 15:02:32.230971 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.228265 2567 server.go:1019] "Starting client certificate rotation" Jul 09 15:02:32.230971 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.228379 2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 15:02:32.230971 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.228422 2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 15:02:32.230971 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.230594 2567 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 15:02:32.231757 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.231736 2567 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 15:02:32.240957 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.240937 2567 log.go:25] "Validated CRI v1 runtime API" Jul 09 15:02:32.245019 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.244997 2567 log.go:25] "Validated CRI v1 image API" Jul 09 15:02:32.245837 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.245818 2567 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 09 15:02:32.248407 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.248388 2567 fs.go:135] Filesystem UUIDs: map[6118d0ee-f8cd-4cfa-8cc0-890e30dd7ce2:/dev/nvme0n1p4 68e84098-6e68-4c89-9af9-aaf688f811bd:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 09 15:02:32.248475 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.248407 2567 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 09 15:02:32.254391 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.254278 2567 manager.go:216] Machine: {Timestamp:2026-07-09 15:02:32.252906469 +0000 UTC m=+0.344350670 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101098 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2369df5f4d6a3143f4036a4f9b58d9 SystemUUID:ec2369df-5f4d-6a31-43f4-036a4f9b58d9 BootID:4a8be972-e73f-418a-9b0e-efc70ad58ab8 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:a6:7d:84:d6:95 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:a6:7d:84:d6:95 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:52:4d:bb:1f:ab:ff Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 09 15:02:32.254391 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.254389 2567 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 09 15:02:32.254526 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.254479 2567 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 09 15:02:32.257016 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.256997 2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 15:02:32.257903 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.257856 2567 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 09 15:02:32.258081 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.257909 2567 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-114.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 09 15:02:32.258635 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.258624 2567 topology_manager.go:138] "Creating topology manager with none policy" Jul 09 15:02:32.258666 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.258637 2567 container_manager_linux.go:309] "Creating device plugin manager" Jul 09 15:02:32.258666 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.258644 2567 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 15:02:32.259331 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.259319 2567 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 15:02:32.260570 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.260560 2567 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 09 15:02:32.263641 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.263629 2567 state_mem.go:36] "Initialized new in-memory state store" Jul 09 15:02:32.263992 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.263982 2567 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 09 15:02:32.264029 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.264022 2567 kubelet.go:486] "Attempting to sync node with API server" Jul 09 15:02:32.264056 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.264033 2567 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 09 15:02:32.264056 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.264047 2567 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 09 15:02:32.264056 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.264056 2567 kubelet.go:398] "Adding apiserver pod source" Jul 09 15:02:32.264160 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.264070 2567 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 09 15:02:32.264486 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.264473 2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 15:02:32.264526 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.264501 2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 15:02:32.266936 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.266919 2567 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 09 15:02:32.270106 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.270059 2567 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 09 15:02:32.270203 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.270132 2567 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 09 15:02:32.270443 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.270426 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 09 15:02:32.270500 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.270452 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 09 15:02:32.270500 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.270465 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 09 15:02:32.270500 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.270477 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 09 15:02:32.270500 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.270491 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 09 15:02:32.270660 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.270503 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 09 15:02:32.270968 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.270952 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 09 15:02:32.271015 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.270969 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 09 15:02:32.271015 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.270983 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 09 15:02:32.271015 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.270993 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 09 15:02:32.271015 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.271006 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 09 15:02:32.271015 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.271016 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 09 15:02:32.272478 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.272462 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 09 15:02:32.272537 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.272481 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 09 15:02:32.275472 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:32.275445 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-114.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 09 15:02:32.275578 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:32.275545 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 09 15:02:32.275675 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.275658 2567 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-114.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 09 15:02:32.276373 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.276362 2567 server.go:1268] "Started kubelet" Jul 09 15:02:32.276453 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.276426 2567 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 09 15:02:32.276504 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.276445 2567 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 09 15:02:32.276553 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.276521 2567 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 09 15:02:32.277342 ip-10-0-130-114 systemd[1]: Started Kubernetes Kubelet. Jul 09 15:02:32.277908 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.277804 2567 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 09 15:02:32.278232 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.278213 2567 server.go:310] "Adding debug handlers to kubelet server" Jul 09 15:02:32.282247 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:32.282223 2567 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 09 15:02:32.282707 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.282689 2567 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 09 15:02:32.282788 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.282691 2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 09 15:02:32.282897 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:32.282791 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-114.ec2.internal\" not found" Jul 09 15:02:32.283438 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.283420 2567 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 09 15:02:32.283521 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.283438 2567 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 09 15:02:32.283575 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.283541 2567 reconstruct.go:97] "Volume reconstruction finished" Jul 09 15:02:32.283575 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.283553 2567 reconciler.go:29] "Reconciler: start to sync state" Jul 09 15:02:32.283668 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.283584 2567 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 09 15:02:32.283957 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.283942 2567 factory.go:153] Registering CRI-O factory Jul 09 15:02:32.284046 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.283960 2567 factory.go:223] Registration of the crio container factory successfully Jul 09 15:02:32.284046 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.284027 2567 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 09 15:02:32.284046 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.284038 2567 factory.go:55] Registering systemd factory Jul 09 15:02:32.284046 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.284046 2567 factory.go:223] Registration of the systemd container factory successfully Jul 09 15:02:32.284245 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.284068 2567 factory.go:103] Registering Raw factory Jul 09 15:02:32.284245 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.284081 2567 manager.go:1196] Started watching for new ooms in manager Jul 09 15:02:32.284352 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:32.283425 2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-114.ec2.internal.18c0a741847ecf4a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-114.ec2.internal,UID:ip-10-0-130-114.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-114.ec2.internal,},FirstTimestamp:2026-07-09 15:02:32.276332362 +0000 UTC m=+0.367776562,LastTimestamp:2026-07-09 15:02:32.276332362 +0000 UTC m=+0.367776562,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-114.ec2.internal,}" Jul 09 15:02:32.284592 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.284571 2567 manager.go:349] Starting recovery of all containers Jul 09 15:02:32.285403 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.285360 2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wpfwz" Jul 09 15:02:32.292204 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:32.292033 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 09 15:02:32.293977 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.293956 2567 manager.go:354] Recovery completed Jul 09 15:02:32.295224 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:32.295192 2567 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-114.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 09 15:02:32.296147 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.296123 2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wpfwz" Jul 09 15:02:32.300606 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.300589 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 15:02:32.302511 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.302496 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasSufficientMemory" Jul 09 15:02:32.302574 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.302524 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 15:02:32.302574 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.302536 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasSufficientPID" Jul 09 15:02:32.303128 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.303113 2567 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 09 15:02:32.303128 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.303124 2567 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 09 15:02:32.303128 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.303128 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 15:02:32.303296 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.303140 2567 state_mem.go:36] "Initialized new in-memory state store" Jul 09 15:02:32.303799 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.303785 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasSufficientMemory" Jul 09 15:02:32.303861 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.303812 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 15:02:32.303861 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.303822 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasSufficientPID" Jul 09 15:02:32.304475 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.304463 2567 policy_none.go:49] "None policy: Start" Jul 09 15:02:32.304514 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.304480 2567 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 09 15:02:32.304514 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.304491 2567 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 09 15:02:32.304985 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:32.304861 2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-114.ec2.internal.18c0a741860e460e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-114.ec2.internal,UID:ip-10-0-130-114.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-114.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-114.ec2.internal,},FirstTimestamp:2026-07-09 15:02:32.30251163 +0000 UTC m=+0.393955833,LastTimestamp:2026-07-09 15:02:32.30251163 +0000 UTC m=+0.393955833,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-114.ec2.internal,}" Jul 09 15:02:32.305742 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.305730 2567 policy_none.go:47] "Start" Jul 09 15:02:32.340544 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.340513 2567 manager.go:337] "Starting Device Plugin manager" Jul 09 15:02:32.340672 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:32.340565 2567 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 09 15:02:32.340672 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.340580 2567 server.go:87] "Starting device plugin registration server" Jul 09 15:02:32.340947 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.340932 2567 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 09 15:02:32.341009 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.340947 2567 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 09 15:02:32.341053 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.341020 2567 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 09 15:02:32.341130 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.341117 2567 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 09 15:02:32.341192 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.341132 2567 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 09 15:02:32.341779 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:32.341756 2567 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 09 15:02:32.341897 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:32.341804 2567 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-114.ec2.internal\" not found" Jul 09 15:02:32.441461 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.441414 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 15:02:32.442979 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.442565 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasSufficientMemory" Jul 09 15:02:32.442979 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.442596 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 15:02:32.442979 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.442606 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasSufficientPID" Jul 09 15:02:32.442979 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.442634 2567 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-114.ec2.internal" Jul 09 15:02:32.451039 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.451020 2567 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-114.ec2.internal" Jul 09 15:02:32.451098 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:32.451044 2567 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-114.ec2.internal\": node \"ip-10-0-130-114.ec2.internal\" not found" Jul 09 15:02:32.459815 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.459755 2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 09 15:02:32.460968 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.460944 2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 09 15:02:32.461028 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.460986 2567 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 09 15:02:32.461028 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.461022 2567 kubelet.go:2448] "Starting kubelet main sync loop" Jul 09 15:02:32.461127 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:32.461058 2567 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 09 15:02:32.463239 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.463212 2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 15:02:32.467305 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:32.467289 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-114.ec2.internal\" not found" Jul 09 15:02:32.561477 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.561420 2567 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-114.ec2.internal"] Jul 09 15:02:32.561629 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.561540 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 15:02:32.563298 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.563281 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasSufficientMemory" Jul 09 15:02:32.563365 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.563315 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 15:02:32.563365 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.563331 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasSufficientPID" Jul 09 15:02:32.565171 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.565159 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 15:02:32.565299 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.565273 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal" Jul 09 15:02:32.565334 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.565313 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 15:02:32.565989 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.565972 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasSufficientMemory" Jul 09 15:02:32.566071 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.566001 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 15:02:32.566071 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.566016 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasSufficientMemory" Jul 09 15:02:32.566071 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.566023 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasSufficientPID" Jul 09 15:02:32.566071 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.566038 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 15:02:32.566071 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.566048 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasSufficientPID" Jul 09 15:02:32.567863 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:32.567848 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-114.ec2.internal\" not found" Jul 09 15:02:32.567991 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.567975 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-114.ec2.internal" Jul 09 15:02:32.568048 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.568001 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 15:02:32.568686 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.568667 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasSufficientMemory" Jul 09 15:02:32.568786 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.568699 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 15:02:32.568786 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.568712 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasSufficientPID" Jul 09 15:02:32.585095 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.585070 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c3942009bd02b28965ce62ce1b271d1a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal\" (UID: \"c3942009bd02b28965ce62ce1b271d1a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal" Jul 09 15:02:32.585225 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.585099 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3942009bd02b28965ce62ce1b271d1a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal\" (UID: \"c3942009bd02b28965ce62ce1b271d1a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal" Jul 09 15:02:32.585225 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.585116 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b4114c320ef9f8839a18178df34d18c8-config\") pod \"kube-apiserver-proxy-ip-10-0-130-114.ec2.internal\" (UID: \"b4114c320ef9f8839a18178df34d18c8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-114.ec2.internal" Jul 09 15:02:32.591668 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:32.591649 2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-114.ec2.internal\" not found" node="ip-10-0-130-114.ec2.internal" Jul 09 15:02:32.596265 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:32.596246 2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-114.ec2.internal\" not found" node="ip-10-0-130-114.ec2.internal" Jul 09 15:02:32.668753 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:32.668717 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-114.ec2.internal\" not found" Jul 09 15:02:32.685714 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.685685 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3942009bd02b28965ce62ce1b271d1a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal\" (UID: \"c3942009bd02b28965ce62ce1b271d1a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal" Jul 09 15:02:32.685714 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.685715 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b4114c320ef9f8839a18178df34d18c8-config\") pod \"kube-apiserver-proxy-ip-10-0-130-114.ec2.internal\" (UID: \"b4114c320ef9f8839a18178df34d18c8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-114.ec2.internal" Jul 09 15:02:32.685863 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.685732 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c3942009bd02b28965ce62ce1b271d1a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal\" (UID: \"c3942009bd02b28965ce62ce1b271d1a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal" Jul 09 15:02:32.685863 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.685759 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c3942009bd02b28965ce62ce1b271d1a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal\" (UID: \"c3942009bd02b28965ce62ce1b271d1a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal" Jul 09 15:02:32.685863 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.685775 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3942009bd02b28965ce62ce1b271d1a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal\" (UID: \"c3942009bd02b28965ce62ce1b271d1a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal" Jul 09 15:02:32.685863 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.685798 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b4114c320ef9f8839a18178df34d18c8-config\") pod \"kube-apiserver-proxy-ip-10-0-130-114.ec2.internal\" (UID: \"b4114c320ef9f8839a18178df34d18c8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-114.ec2.internal" Jul 09 15:02:32.769945 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:32.769822 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-114.ec2.internal\" not found" Jul 09 15:02:32.870417 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:32.870377 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-114.ec2.internal\" not found" Jul 09 15:02:32.893958 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.893927 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal" Jul 09 15:02:32.898846 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:32.898830 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-114.ec2.internal" Jul 09 15:02:32.970652 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:32.970612 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-114.ec2.internal\" not found" Jul 09 15:02:33.071158 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:33.071094 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-114.ec2.internal\" not found" Jul 09 15:02:33.169189 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.169155 2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 15:02:33.171917 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:33.171896 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-114.ec2.internal\" not found" Jul 09 15:02:33.225361 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.225333 2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 15:02:33.227336 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.227320 2567 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 09 15:02:33.227480 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.227461 2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 15:02:33.227534 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.227493 2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 15:02:33.264682 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.264637 2567 apiserver.go:52] "Watching apiserver" Jul 09 15:02:33.283095 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.283065 2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 09 15:02:33.283276 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.283213 2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal" Jul 09 15:02:33.285721 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.284418 2567 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 09 15:02:33.285721 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.285192 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-p8vsh","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9hc8","openshift-cluster-node-tuning-operator/tuned-6r75j","openshift-multus/multus-additional-cni-plugins-b9zn8","openshift-multus/network-metrics-daemon-jcctg","openshift-network-diagnostics/network-check-target-96fg2","openshift-ovn-kubernetes/ovnkube-node-hml7m","kube-system/konnectivity-agent-9647m","openshift-dns/node-resolver-k9ppv","openshift-image-registry/node-ca-w9rr7","openshift-multus/multus-54jgr","openshift-network-operator/iptables-alerter-gnc2d"] Jul 09 15:02:33.286674 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.286654 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:02:33.286767 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:33.286727 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8vsh" podUID="fce18369-7aaf-4358-b5f5-46b0218924fd" Jul 09 15:02:33.287870 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.287852 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9hc8" Jul 09 15:02:33.289085 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.289060 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.290094 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.290076 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 09 15:02:33.290237 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.290221 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gncsm\"" Jul 09 15:02:33.290323 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.290308 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-b9zn8" Jul 09 15:02:33.290378 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.290334 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 09 15:02:33.290431 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.290334 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 09 15:02:33.291238 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.291225 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 09 15:02:33.291581 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.291567 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:02:33.291656 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:33.291621 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jcctg" podUID="37399ba7-ce43-46e3-b78e-73b29c2196be" Jul 09 15:02:33.292634 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.292617 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:02:33.292715 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:33.292665 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-96fg2" podUID="d469078c-492e-4dbd-843f-7d1699fa1ce9" Jul 09 15:02:33.293678 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.293660 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 09 15:02:33.293800 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.293782 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 09 15:02:33.293800 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.293792 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 09 15:02:33.293920 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.293799 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 09 15:02:33.293920 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.293811 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 09 15:02:33.293920 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.293820 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-n2jr2\"" Jul 09 15:02:33.294066 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.293922 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 09 15:02:33.294066 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.293954 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4fm9c\"" Jul 09 15:02:33.294271 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.294255 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.295663 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.295646 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9647m" Jul 09 15:02:33.296735 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.296719 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 09 15:02:33.296925 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.296912 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-k9ppv" Jul 09 15:02:33.297356 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.297317 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 09 15:02:33.297356 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.297329 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 09 15:02:33.297356 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.297345 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 09 15:02:33.297521 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.297361 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 09 15:02:33.297521 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.297370 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 09 15:02:33.297521 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.297331 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-b64nd\"" Jul 09 15:02:33.297817 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.297802 2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 15:02:33.297931 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.297871 2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-114.ec2.internal" Jul 09 15:02:33.298578 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.298563 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-w9rr7" Jul 09 15:02:33.299402 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.299378 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 14:57:33 +0000 UTC" deadline="2028-03-24 21:00:18.348327918 +0000 UTC" Jul 09 15:02:33.299402 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.299399 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14981h57m45.048931006s" Jul 09 15:02:33.299692 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.299673 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 09 15:02:33.299762 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.299722 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 09 15:02:33.299815 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.299760 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 09 15:02:33.299815 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.299806 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 09 15:02:33.300044 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.299956 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.300044 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.300017 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-rkj5l\"" Jul 09 15:02:33.300419 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.300394 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-h2xsm\"" Jul 09 15:02:33.300750 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.300731 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 09 15:02:33.300908 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.300893 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 09 15:02:33.300982 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.300950 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-88kj7\"" Jul 09 15:02:33.301165 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.301152 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 09 15:02:33.301495 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.301477 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal"] Jul 09 15:02:33.301576 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.301531 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-gnc2d" Jul 09 15:02:33.301626 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.301602 2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 15:02:33.304254 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.304239 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 09 15:02:33.304254 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.304250 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-zk69p\"" Jul 09 15:02:33.304557 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.304545 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 09 15:02:33.304704 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.304690 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 09 15:02:33.305110 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.304948 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rxzcb\"" Jul 09 15:02:33.305946 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.305931 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 09 15:02:33.310452 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.310437 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-130-114.ec2.internal"] Jul 09 15:02:33.310699 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.310684 2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 15:02:33.324634 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.324570 2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rjgh9" Jul 09 15:02:33.340226 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.340207 2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rjgh9" Jul 09 15:02:33.384767 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.384748 2567 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 09 15:02:33.388549 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.388504 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24e2d8f2-c395-4f83-9b5b-c71affe096ea-tmp\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.388627 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.388542 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67924f4e-f69d-48b8-ba23-e4c110bb24e5-tmp-dir\") pod \"node-resolver-k9ppv\" (UID: \"67924f4e-f69d-48b8-ba23-e4c110bb24e5\") " pod="openshift-dns/node-resolver-k9ppv" Jul 09 15:02:33.388627 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.388572 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-multus-cni-dir\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.388627 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.388595 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/51611d68-7db8-44af-81b2-87c97ab134cc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h9hc8\" (UID: \"51611d68-7db8-44af-81b2-87c97ab134cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9hc8" Jul 09 15:02:33.388627 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.388618 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/24e2d8f2-c395-4f83-9b5b-c71affe096ea-var-lib-kubelet\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.388796 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.388640 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/24e2d8f2-c395-4f83-9b5b-c71affe096ea-host\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.388796 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.388661 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-host-cni-netd\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.388796 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.388674 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-system-cni-dir\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.388796 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.388692 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/daad1095-91b7-47c6-bddf-dd0b6f01f8a9-konnectivity-ca\") pod \"konnectivity-agent-9647m\" (UID: \"daad1095-91b7-47c6-bddf-dd0b6f01f8a9\") " pod="kube-system/konnectivity-agent-9647m" Jul 09 15:02:33.388796 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.388717 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37399ba7-ce43-46e3-b78e-73b29c2196be-metrics-certs\") pod \"network-metrics-daemon-jcctg\" (UID: \"37399ba7-ce43-46e3-b78e-73b29c2196be\") " pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:02:33.388796 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.388738 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-host-kubelet\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.388796 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.388778 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-host-run-netns\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.389125 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.388807 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-run-ovn\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.389125 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.388831 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-node-log\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.389125 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.388950 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-host-run-ovn-kubernetes\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.389125 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.388996 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-host-var-lib-cni-multus\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.389125 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.389025 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/51611d68-7db8-44af-81b2-87c97ab134cc-sys-fs\") pod \"aws-ebs-csi-driver-node-h9hc8\" (UID: \"51611d68-7db8-44af-81b2-87c97ab134cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9hc8" Jul 09 15:02:33.389125 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.389054 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/24e2d8f2-c395-4f83-9b5b-c71affe096ea-etc-sysconfig\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.389125 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.389078 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/24e2d8f2-c395-4f83-9b5b-c71affe096ea-etc-systemd\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.389125 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.389105 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.389516 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.389129 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/84a10578-4913-456b-9fb7-e6440914b677-ovnkube-config\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.389516 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.389153 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwnrb\" (UniqueName: \"kubernetes.io/projected/84a10578-4913-456b-9fb7-e6440914b677-kube-api-access-mwnrb\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.389516 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.389179 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-cnibin\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.389516 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.389201 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ba00eb64-f303-4c0c-9292-88611e2bf9c0-cni-binary-copy\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.389516 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.389255 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fce18369-7aaf-4358-b5f5-46b0218924fd-kubelet-config\") pod \"global-pull-secret-syncer-p8vsh\" (UID: \"fce18369-7aaf-4358-b5f5-46b0218924fd\") " pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:02:33.389516 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.389297 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fce18369-7aaf-4358-b5f5-46b0218924fd-dbus\") pod \"global-pull-secret-syncer-p8vsh\" (UID: \"fce18369-7aaf-4358-b5f5-46b0218924fd\") " pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:02:33.389516 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.389325 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-host-slash\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.389516 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.389350 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-var-lib-openvswitch\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.389516 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.389372 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-etc-openvswitch\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.389516 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.389424 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/67924f4e-f69d-48b8-ba23-e4c110bb24e5-hosts-file\") pod \"node-resolver-k9ppv\" (UID: \"67924f4e-f69d-48b8-ba23-e4c110bb24e5\") " pod="openshift-dns/node-resolver-k9ppv" Jul 09 15:02:33.389516 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.389448 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzmj8\" (UniqueName: \"kubernetes.io/projected/6cf97264-90f5-4d6e-9e44-50e1431776a8-kube-api-access-gzmj8\") pod \"multus-additional-cni-plugins-b9zn8\" (UID: \"6cf97264-90f5-4d6e-9e44-50e1431776a8\") " pod="openshift-multus/multus-additional-cni-plugins-b9zn8" Jul 09 15:02:33.389516 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.389472 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d7e1ab55-7472-458f-9078-fe9ca998cbd9-host-slash\") pod \"iptables-alerter-gnc2d\" (UID: \"d7e1ab55-7472-458f-9078-fe9ca998cbd9\") " pod="openshift-network-operator/iptables-alerter-gnc2d" Jul 09 15:02:33.389516 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.389495 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nh8c\" (UniqueName: \"kubernetes.io/projected/51611d68-7db8-44af-81b2-87c97ab134cc-kube-api-access-7nh8c\") pod \"aws-ebs-csi-driver-node-h9hc8\" (UID: \"51611d68-7db8-44af-81b2-87c97ab134cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9hc8" Jul 09 15:02:33.389516 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.389515 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ffeceed9-e851-4ef7-8ad8-603243ee5157-host\") pod \"node-ca-w9rr7\" (UID: \"ffeceed9-e851-4ef7-8ad8-603243ee5157\") " pod="openshift-image-registry/node-ca-w9rr7" Jul 09 15:02:33.390128 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.389529 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-hostroot\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.390128 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.389544 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/daad1095-91b7-47c6-bddf-dd0b6f01f8a9-agent-certs\") pod \"konnectivity-agent-9647m\" (UID: \"daad1095-91b7-47c6-bddf-dd0b6f01f8a9\") " pod="kube-system/konnectivity-agent-9647m" Jul 09 15:02:33.390128 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.389566 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/51611d68-7db8-44af-81b2-87c97ab134cc-registration-dir\") pod \"aws-ebs-csi-driver-node-h9hc8\" (UID: \"51611d68-7db8-44af-81b2-87c97ab134cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9hc8" Jul 09 15:02:33.390128 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.389610 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/51611d68-7db8-44af-81b2-87c97ab134cc-etc-selinux\") pod \"aws-ebs-csi-driver-node-h9hc8\" (UID: \"51611d68-7db8-44af-81b2-87c97ab134cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9hc8" Jul 09 15:02:33.390128 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.389658 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/24e2d8f2-c395-4f83-9b5b-c71affe096ea-run\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.390128 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.389682 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9bfc\" (UniqueName: \"kubernetes.io/projected/24e2d8f2-c395-4f83-9b5b-c71affe096ea-kube-api-access-v9bfc\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.390128 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.389700 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gqbc\" (UniqueName: \"kubernetes.io/projected/37399ba7-ce43-46e3-b78e-73b29c2196be-kube-api-access-9gqbc\") pod \"network-metrics-daemon-jcctg\" (UID: \"37399ba7-ce43-46e3-b78e-73b29c2196be\") " pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:02:33.390128 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.389715 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-host-cni-bin\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.390128 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.389731 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ffeceed9-e851-4ef7-8ad8-603243ee5157-serviceca\") pod \"node-ca-w9rr7\" (UID: \"ffeceed9-e851-4ef7-8ad8-603243ee5157\") " pod="openshift-image-registry/node-ca-w9rr7" Jul 09 15:02:33.390128 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.389776 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/24e2d8f2-c395-4f83-9b5b-c71affe096ea-etc-modprobe-d\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.390128 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.389802 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/24e2d8f2-c395-4f83-9b5b-c71affe096ea-etc-sysctl-d\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.390128 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.389817 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-systemd-units\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.390128 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.389836 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-run-systemd\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.390128 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.389858 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-os-release\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.390128 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.389902 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-host-run-netns\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.390128 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.389937 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6cf97264-90f5-4d6e-9e44-50e1431776a8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b9zn8\" (UID: \"6cf97264-90f5-4d6e-9e44-50e1431776a8\") " pod="openshift-multus/multus-additional-cni-plugins-b9zn8" Jul 09 15:02:33.390739 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.389962 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clg8w\" (UniqueName: \"kubernetes.io/projected/d7e1ab55-7472-458f-9078-fe9ca998cbd9-kube-api-access-clg8w\") pod \"iptables-alerter-gnc2d\" (UID: \"d7e1ab55-7472-458f-9078-fe9ca998cbd9\") " pod="openshift-network-operator/iptables-alerter-gnc2d" Jul 09 15:02:33.390739 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.389990 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24e2d8f2-c395-4f83-9b5b-c71affe096ea-etc-kubernetes\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.390739 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.390024 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/24e2d8f2-c395-4f83-9b5b-c71affe096ea-sys\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.390739 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.390050 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/84a10578-4913-456b-9fb7-e6440914b677-ovn-node-metrics-cert\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.390739 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.390071 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-multus-conf-dir\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.390739 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.390114 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6cf97264-90f5-4d6e-9e44-50e1431776a8-os-release\") pod \"multus-additional-cni-plugins-b9zn8\" (UID: \"6cf97264-90f5-4d6e-9e44-50e1431776a8\") " pod="openshift-multus/multus-additional-cni-plugins-b9zn8" Jul 09 15:02:33.390739 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.390148 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/84a10578-4913-456b-9fb7-e6440914b677-ovnkube-script-lib\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.390739 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.390175 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/24e2d8f2-c395-4f83-9b5b-c71affe096ea-lib-modules\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.390739 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.390198 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-host-run-k8s-cni-cncf-io\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.390739 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.390242 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ba00eb64-f303-4c0c-9292-88611e2bf9c0-multus-daemon-config\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.390739 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.390272 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvdl8\" (UniqueName: \"kubernetes.io/projected/ba00eb64-f303-4c0c-9292-88611e2bf9c0-kube-api-access-rvdl8\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.390739 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.390297 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6cf97264-90f5-4d6e-9e44-50e1431776a8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b9zn8\" (UID: \"6cf97264-90f5-4d6e-9e44-50e1431776a8\") " pod="openshift-multus/multus-additional-cni-plugins-b9zn8" Jul 09 15:02:33.390739 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.390319 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6cf97264-90f5-4d6e-9e44-50e1431776a8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b9zn8\" (UID: \"6cf97264-90f5-4d6e-9e44-50e1431776a8\") " pod="openshift-multus/multus-additional-cni-plugins-b9zn8" Jul 09 15:02:33.390739 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.390357 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/51611d68-7db8-44af-81b2-87c97ab134cc-device-dir\") pod \"aws-ebs-csi-driver-node-h9hc8\" (UID: \"51611d68-7db8-44af-81b2-87c97ab134cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9hc8" Jul 09 15:02:33.390739 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.390385 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/24e2d8f2-c395-4f83-9b5b-c71affe096ea-etc-sysctl-conf\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.390739 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.390405 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/24e2d8f2-c395-4f83-9b5b-c71affe096ea-etc-tuned\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.391286 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.390426 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6cf97264-90f5-4d6e-9e44-50e1431776a8-cnibin\") pod \"multus-additional-cni-plugins-b9zn8\" (UID: \"6cf97264-90f5-4d6e-9e44-50e1431776a8\") " pod="openshift-multus/multus-additional-cni-plugins-b9zn8" Jul 09 15:02:33.391286 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.390452 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4c4r\" (UniqueName: \"kubernetes.io/projected/d469078c-492e-4dbd-843f-7d1699fa1ce9-kube-api-access-r4c4r\") pod \"network-check-target-96fg2\" (UID: \"d469078c-492e-4dbd-843f-7d1699fa1ce9\") " pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:02:33.391286 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.390478 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/51611d68-7db8-44af-81b2-87c97ab134cc-socket-dir\") pod \"aws-ebs-csi-driver-node-h9hc8\" (UID: \"51611d68-7db8-44af-81b2-87c97ab134cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9hc8" Jul 09 15:02:33.391286 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.390512 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-host-var-lib-cni-bin\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.391286 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.390534 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/84a10578-4913-456b-9fb7-e6440914b677-env-overrides\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.391286 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.390555 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzf5c\" (UniqueName: \"kubernetes.io/projected/ffeceed9-e851-4ef7-8ad8-603243ee5157-kube-api-access-xzf5c\") pod \"node-ca-w9rr7\" (UID: \"ffeceed9-e851-4ef7-8ad8-603243ee5157\") " pod="openshift-image-registry/node-ca-w9rr7" Jul 09 15:02:33.391286 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.390582 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-etc-kubernetes\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.391286 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.390605 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6cf97264-90f5-4d6e-9e44-50e1431776a8-cni-binary-copy\") pod \"multus-additional-cni-plugins-b9zn8\" (UID: \"6cf97264-90f5-4d6e-9e44-50e1431776a8\") " pod="openshift-multus/multus-additional-cni-plugins-b9zn8" Jul 09 15:02:33.391286 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.390620 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-log-socket\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.391286 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.390653 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-828gn\" (UniqueName: \"kubernetes.io/projected/67924f4e-f69d-48b8-ba23-e4c110bb24e5-kube-api-access-828gn\") pod \"node-resolver-k9ppv\" (UID: \"67924f4e-f69d-48b8-ba23-e4c110bb24e5\") " pod="openshift-dns/node-resolver-k9ppv" Jul 09 15:02:33.391286 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.390676 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-multus-socket-dir-parent\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.391286 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.390690 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-host-var-lib-kubelet\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.391286 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.390710 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-host-run-multus-certs\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.391286 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.390733 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fce18369-7aaf-4358-b5f5-46b0218924fd-original-pull-secret\") pod \"global-pull-secret-syncer-p8vsh\" (UID: \"fce18369-7aaf-4358-b5f5-46b0218924fd\") " pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:02:33.391286 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.390748 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-run-openvswitch\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.391286 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.390763 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6cf97264-90f5-4d6e-9e44-50e1431776a8-system-cni-dir\") pod \"multus-additional-cni-plugins-b9zn8\" (UID: \"6cf97264-90f5-4d6e-9e44-50e1431776a8\") " pod="openshift-multus/multus-additional-cni-plugins-b9zn8" Jul 09 15:02:33.391757 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.390800 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d7e1ab55-7472-458f-9078-fe9ca998cbd9-iptables-alerter-script\") pod \"iptables-alerter-gnc2d\" (UID: \"d7e1ab55-7472-458f-9078-fe9ca998cbd9\") " pod="openshift-network-operator/iptables-alerter-gnc2d" Jul 09 15:02:33.480230 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:33.480200 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4114c320ef9f8839a18178df34d18c8.slice/crio-653b7a381c909ddeeb38d26e6fb15727ec52b6efd250518fbe243480f8601be3 WatchSource:0}: Error finding container 653b7a381c909ddeeb38d26e6fb15727ec52b6efd250518fbe243480f8601be3: Status 404 returned error can't find the container with id 653b7a381c909ddeeb38d26e6fb15727ec52b6efd250518fbe243480f8601be3 Jul 09 15:02:33.487179 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.487163 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 15:02:33.491578 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.491558 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-multus-conf-dir\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.491673 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.491591 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6cf97264-90f5-4d6e-9e44-50e1431776a8-os-release\") pod \"multus-additional-cni-plugins-b9zn8\" (UID: \"6cf97264-90f5-4d6e-9e44-50e1431776a8\") " pod="openshift-multus/multus-additional-cni-plugins-b9zn8" Jul 09 15:02:33.491673 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.491615 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/84a10578-4913-456b-9fb7-e6440914b677-ovnkube-script-lib\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.491673 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.491663 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-multus-conf-dir\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.491822 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.491684 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6cf97264-90f5-4d6e-9e44-50e1431776a8-os-release\") pod \"multus-additional-cni-plugins-b9zn8\" (UID: \"6cf97264-90f5-4d6e-9e44-50e1431776a8\") " pod="openshift-multus/multus-additional-cni-plugins-b9zn8" Jul 09 15:02:33.491822 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.491713 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/24e2d8f2-c395-4f83-9b5b-c71affe096ea-lib-modules\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.491822 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.491737 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-host-run-k8s-cni-cncf-io\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.491822 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.491759 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ba00eb64-f303-4c0c-9292-88611e2bf9c0-multus-daemon-config\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.491822 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.491793 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-host-run-k8s-cni-cncf-io\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.491822 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.491814 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvdl8\" (UniqueName: \"kubernetes.io/projected/ba00eb64-f303-4c0c-9292-88611e2bf9c0-kube-api-access-rvdl8\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.492132 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.491830 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6cf97264-90f5-4d6e-9e44-50e1431776a8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b9zn8\" (UID: \"6cf97264-90f5-4d6e-9e44-50e1431776a8\") " pod="openshift-multus/multus-additional-cni-plugins-b9zn8" Jul 09 15:02:33.492132 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.491848 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6cf97264-90f5-4d6e-9e44-50e1431776a8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b9zn8\" (UID: \"6cf97264-90f5-4d6e-9e44-50e1431776a8\") " pod="openshift-multus/multus-additional-cni-plugins-b9zn8" Jul 09 15:02:33.492132 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.491864 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/24e2d8f2-c395-4f83-9b5b-c71affe096ea-lib-modules\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.492132 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.491868 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/51611d68-7db8-44af-81b2-87c97ab134cc-device-dir\") pod \"aws-ebs-csi-driver-node-h9hc8\" (UID: \"51611d68-7db8-44af-81b2-87c97ab134cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9hc8" Jul 09 15:02:33.492132 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.491922 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/24e2d8f2-c395-4f83-9b5b-c71affe096ea-etc-sysctl-conf\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.492132 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.491948 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/24e2d8f2-c395-4f83-9b5b-c71affe096ea-etc-tuned\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.492132 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.491979 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6cf97264-90f5-4d6e-9e44-50e1431776a8-cnibin\") pod \"multus-additional-cni-plugins-b9zn8\" (UID: \"6cf97264-90f5-4d6e-9e44-50e1431776a8\") " pod="openshift-multus/multus-additional-cni-plugins-b9zn8" Jul 09 15:02:33.492132 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.491925 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/51611d68-7db8-44af-81b2-87c97ab134cc-device-dir\") pod \"aws-ebs-csi-driver-node-h9hc8\" (UID: \"51611d68-7db8-44af-81b2-87c97ab134cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9hc8" Jul 09 15:02:33.492132 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492004 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4c4r\" (UniqueName: \"kubernetes.io/projected/d469078c-492e-4dbd-843f-7d1699fa1ce9-kube-api-access-r4c4r\") pod \"network-check-target-96fg2\" (UID: \"d469078c-492e-4dbd-843f-7d1699fa1ce9\") " pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:02:33.492132 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492034 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/51611d68-7db8-44af-81b2-87c97ab134cc-socket-dir\") pod \"aws-ebs-csi-driver-node-h9hc8\" (UID: \"51611d68-7db8-44af-81b2-87c97ab134cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9hc8" Jul 09 15:02:33.492132 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492043 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6cf97264-90f5-4d6e-9e44-50e1431776a8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b9zn8\" (UID: \"6cf97264-90f5-4d6e-9e44-50e1431776a8\") " pod="openshift-multus/multus-additional-cni-plugins-b9zn8" Jul 09 15:02:33.492132 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492059 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-host-var-lib-cni-bin\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.492132 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492085 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/84a10578-4913-456b-9fb7-e6440914b677-env-overrides\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.492132 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492110 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzf5c\" (UniqueName: \"kubernetes.io/projected/ffeceed9-e851-4ef7-8ad8-603243ee5157-kube-api-access-xzf5c\") pod \"node-ca-w9rr7\" (UID: \"ffeceed9-e851-4ef7-8ad8-603243ee5157\") " pod="openshift-image-registry/node-ca-w9rr7" Jul 09 15:02:33.492132 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492134 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-etc-kubernetes\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.492920 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492164 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6cf97264-90f5-4d6e-9e44-50e1431776a8-cni-binary-copy\") pod \"multus-additional-cni-plugins-b9zn8\" (UID: \"6cf97264-90f5-4d6e-9e44-50e1431776a8\") " pod="openshift-multus/multus-additional-cni-plugins-b9zn8" Jul 09 15:02:33.492920 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492187 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6cf97264-90f5-4d6e-9e44-50e1431776a8-cnibin\") pod \"multus-additional-cni-plugins-b9zn8\" (UID: \"6cf97264-90f5-4d6e-9e44-50e1431776a8\") " pod="openshift-multus/multus-additional-cni-plugins-b9zn8" Jul 09 15:02:33.492920 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492192 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/24e2d8f2-c395-4f83-9b5b-c71affe096ea-etc-sysctl-conf\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.492920 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492191 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-log-socket\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.492920 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492233 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-828gn\" (UniqueName: \"kubernetes.io/projected/67924f4e-f69d-48b8-ba23-e4c110bb24e5-kube-api-access-828gn\") pod \"node-resolver-k9ppv\" (UID: \"67924f4e-f69d-48b8-ba23-e4c110bb24e5\") " pod="openshift-dns/node-resolver-k9ppv" Jul 09 15:02:33.492920 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492243 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-log-socket\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.492920 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492260 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-multus-socket-dir-parent\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.492920 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492286 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-host-var-lib-kubelet\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.492920 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492309 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-host-run-multus-certs\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.492920 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492314 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/84a10578-4913-456b-9fb7-e6440914b677-ovnkube-script-lib\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.492920 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492334 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fce18369-7aaf-4358-b5f5-46b0218924fd-original-pull-secret\") pod \"global-pull-secret-syncer-p8vsh\" (UID: \"fce18369-7aaf-4358-b5f5-46b0218924fd\") " pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:02:33.492920 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492351 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/51611d68-7db8-44af-81b2-87c97ab134cc-socket-dir\") pod \"aws-ebs-csi-driver-node-h9hc8\" (UID: \"51611d68-7db8-44af-81b2-87c97ab134cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9hc8" Jul 09 15:02:33.492920 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492355 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ba00eb64-f303-4c0c-9292-88611e2bf9c0-multus-daemon-config\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.492920 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492378 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-run-openvswitch\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.492920 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:33.492413 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 15:02:33.492920 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492429 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-host-var-lib-cni-bin\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.492920 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492444 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6cf97264-90f5-4d6e-9e44-50e1431776a8-system-cni-dir\") pod \"multus-additional-cni-plugins-b9zn8\" (UID: \"6cf97264-90f5-4d6e-9e44-50e1431776a8\") " pod="openshift-multus/multus-additional-cni-plugins-b9zn8" Jul 09 15:02:33.492920 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492449 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6cf97264-90f5-4d6e-9e44-50e1431776a8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b9zn8\" (UID: \"6cf97264-90f5-4d6e-9e44-50e1431776a8\") " pod="openshift-multus/multus-additional-cni-plugins-b9zn8" Jul 09 15:02:33.493745 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:33.492473 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fce18369-7aaf-4358-b5f5-46b0218924fd-original-pull-secret podName:fce18369-7aaf-4358-b5f5-46b0218924fd nodeName:}" failed. No retries permitted until 2026-07-09 15:02:33.992452793 +0000 UTC m=+2.083897208 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fce18369-7aaf-4358-b5f5-46b0218924fd-original-pull-secret") pod "global-pull-secret-syncer-p8vsh" (UID: "fce18369-7aaf-4358-b5f5-46b0218924fd") : object "kube-system"/"original-pull-secret" not registered Jul 09 15:02:33.493745 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492469 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-etc-kubernetes\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.493745 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492510 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-run-openvswitch\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.493745 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492523 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-host-var-lib-kubelet\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.493745 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492569 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-host-run-multus-certs\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.493745 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492412 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6cf97264-90f5-4d6e-9e44-50e1431776a8-system-cni-dir\") pod \"multus-additional-cni-plugins-b9zn8\" (UID: \"6cf97264-90f5-4d6e-9e44-50e1431776a8\") " pod="openshift-multus/multus-additional-cni-plugins-b9zn8" Jul 09 15:02:33.493745 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492597 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-multus-socket-dir-parent\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.493745 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492622 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d7e1ab55-7472-458f-9078-fe9ca998cbd9-iptables-alerter-script\") pod \"iptables-alerter-gnc2d\" (UID: \"d7e1ab55-7472-458f-9078-fe9ca998cbd9\") " pod="openshift-network-operator/iptables-alerter-gnc2d" Jul 09 15:02:33.493745 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492626 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6cf97264-90f5-4d6e-9e44-50e1431776a8-cni-binary-copy\") pod \"multus-additional-cni-plugins-b9zn8\" (UID: \"6cf97264-90f5-4d6e-9e44-50e1431776a8\") " pod="openshift-multus/multus-additional-cni-plugins-b9zn8" Jul 09 15:02:33.493745 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492710 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24e2d8f2-c395-4f83-9b5b-c71affe096ea-tmp\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.493745 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492715 2567 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 09 15:02:33.493745 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492740 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67924f4e-f69d-48b8-ba23-e4c110bb24e5-tmp-dir\") pod \"node-resolver-k9ppv\" (UID: \"67924f4e-f69d-48b8-ba23-e4c110bb24e5\") " pod="openshift-dns/node-resolver-k9ppv" Jul 09 15:02:33.493745 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492764 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-multus-cni-dir\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.493745 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492788 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/84a10578-4913-456b-9fb7-e6440914b677-env-overrides\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.493745 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492796 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/51611d68-7db8-44af-81b2-87c97ab134cc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h9hc8\" (UID: \"51611d68-7db8-44af-81b2-87c97ab134cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9hc8" Jul 09 15:02:33.493745 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492819 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/24e2d8f2-c395-4f83-9b5b-c71affe096ea-var-lib-kubelet\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.493745 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492843 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/24e2d8f2-c395-4f83-9b5b-c71affe096ea-host\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.494519 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492862 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-multus-cni-dir\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.494519 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492872 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/51611d68-7db8-44af-81b2-87c97ab134cc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h9hc8\" (UID: \"51611d68-7db8-44af-81b2-87c97ab134cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9hc8" Jul 09 15:02:33.494519 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492928 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-host-cni-netd\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.494519 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492936 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/24e2d8f2-c395-4f83-9b5b-c71affe096ea-var-lib-kubelet\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.494519 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492866 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-host-cni-netd\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.494519 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492975 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-system-cni-dir\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.494519 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.492979 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/24e2d8f2-c395-4f83-9b5b-c71affe096ea-host\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.494519 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493001 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/daad1095-91b7-47c6-bddf-dd0b6f01f8a9-konnectivity-ca\") pod \"konnectivity-agent-9647m\" (UID: \"daad1095-91b7-47c6-bddf-dd0b6f01f8a9\") " pod="kube-system/konnectivity-agent-9647m" Jul 09 15:02:33.494519 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493019 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37399ba7-ce43-46e3-b78e-73b29c2196be-metrics-certs\") pod \"network-metrics-daemon-jcctg\" (UID: \"37399ba7-ce43-46e3-b78e-73b29c2196be\") " pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:02:33.494519 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493028 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-system-cni-dir\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.494519 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493033 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-host-kubelet\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.494519 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493057 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-host-kubelet\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.494519 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493061 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-host-run-netns\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.494519 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493088 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-run-ovn\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.494519 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493114 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-node-log\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.494519 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493142 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-host-run-ovn-kubernetes\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.494519 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493168 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-host-var-lib-cni-multus\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.494519 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:33.493181 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 15:02:33.495294 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493186 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d7e1ab55-7472-458f-9078-fe9ca998cbd9-iptables-alerter-script\") pod \"iptables-alerter-gnc2d\" (UID: \"d7e1ab55-7472-458f-9078-fe9ca998cbd9\") " pod="openshift-network-operator/iptables-alerter-gnc2d" Jul 09 15:02:33.495294 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493192 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/51611d68-7db8-44af-81b2-87c97ab134cc-sys-fs\") pod \"aws-ebs-csi-driver-node-h9hc8\" (UID: \"51611d68-7db8-44af-81b2-87c97ab134cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9hc8" Jul 09 15:02:33.495294 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493203 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-run-ovn\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.495294 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493168 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67924f4e-f69d-48b8-ba23-e4c110bb24e5-tmp-dir\") pod \"node-resolver-k9ppv\" (UID: \"67924f4e-f69d-48b8-ba23-e4c110bb24e5\") " pod="openshift-dns/node-resolver-k9ppv" Jul 09 15:02:33.495294 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493221 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/24e2d8f2-c395-4f83-9b5b-c71affe096ea-etc-sysconfig\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.495294 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:33.493245 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37399ba7-ce43-46e3-b78e-73b29c2196be-metrics-certs podName:37399ba7-ce43-46e3-b78e-73b29c2196be nodeName:}" failed. No retries permitted until 2026-07-09 15:02:33.993224339 +0000 UTC m=+2.084668540 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37399ba7-ce43-46e3-b78e-73b29c2196be-metrics-certs") pod "network-metrics-daemon-jcctg" (UID: "37399ba7-ce43-46e3-b78e-73b29c2196be") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 15:02:33.495294 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493251 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-node-log\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.495294 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493275 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/24e2d8f2-c395-4f83-9b5b-c71affe096ea-etc-systemd\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.495294 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493282 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-host-run-netns\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.495294 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493286 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/24e2d8f2-c395-4f83-9b5b-c71affe096ea-etc-sysconfig\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.495294 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493304 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.495294 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493312 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/51611d68-7db8-44af-81b2-87c97ab134cc-sys-fs\") pod \"aws-ebs-csi-driver-node-h9hc8\" (UID: \"51611d68-7db8-44af-81b2-87c97ab134cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9hc8" Jul 09 15:02:33.495294 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493332 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/84a10578-4913-456b-9fb7-e6440914b677-ovnkube-config\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.495294 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493347 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-host-run-ovn-kubernetes\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.495294 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493341 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-host-var-lib-cni-multus\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.495294 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493362 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwnrb\" (UniqueName: \"kubernetes.io/projected/84a10578-4913-456b-9fb7-e6440914b677-kube-api-access-mwnrb\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.495294 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493352 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/24e2d8f2-c395-4f83-9b5b-c71affe096ea-etc-systemd\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.496167 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493388 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-cnibin\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.496167 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493392 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.496167 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493412 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ba00eb64-f303-4c0c-9292-88611e2bf9c0-cni-binary-copy\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.496167 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493436 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fce18369-7aaf-4358-b5f5-46b0218924fd-kubelet-config\") pod \"global-pull-secret-syncer-p8vsh\" (UID: \"fce18369-7aaf-4358-b5f5-46b0218924fd\") " pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:02:33.496167 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493459 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fce18369-7aaf-4358-b5f5-46b0218924fd-dbus\") pod \"global-pull-secret-syncer-p8vsh\" (UID: \"fce18369-7aaf-4358-b5f5-46b0218924fd\") " pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:02:33.496167 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493480 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-host-slash\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.496167 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493504 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-var-lib-openvswitch\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.496167 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493521 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fce18369-7aaf-4358-b5f5-46b0218924fd-kubelet-config\") pod \"global-pull-secret-syncer-p8vsh\" (UID: \"fce18369-7aaf-4358-b5f5-46b0218924fd\") " pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:02:33.496167 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493533 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-etc-openvswitch\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.496167 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493557 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/67924f4e-f69d-48b8-ba23-e4c110bb24e5-hosts-file\") pod \"node-resolver-k9ppv\" (UID: \"67924f4e-f69d-48b8-ba23-e4c110bb24e5\") " pod="openshift-dns/node-resolver-k9ppv" Jul 09 15:02:33.496167 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493479 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-cnibin\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.496167 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493582 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzmj8\" (UniqueName: \"kubernetes.io/projected/6cf97264-90f5-4d6e-9e44-50e1431776a8-kube-api-access-gzmj8\") pod \"multus-additional-cni-plugins-b9zn8\" (UID: \"6cf97264-90f5-4d6e-9e44-50e1431776a8\") " pod="openshift-multus/multus-additional-cni-plugins-b9zn8" Jul 09 15:02:33.496167 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493603 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/daad1095-91b7-47c6-bddf-dd0b6f01f8a9-konnectivity-ca\") pod \"konnectivity-agent-9647m\" (UID: \"daad1095-91b7-47c6-bddf-dd0b6f01f8a9\") " pod="kube-system/konnectivity-agent-9647m" Jul 09 15:02:33.496167 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493609 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d7e1ab55-7472-458f-9078-fe9ca998cbd9-host-slash\") pod \"iptables-alerter-gnc2d\" (UID: \"d7e1ab55-7472-458f-9078-fe9ca998cbd9\") " pod="openshift-network-operator/iptables-alerter-gnc2d" Jul 09 15:02:33.496167 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493634 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nh8c\" (UniqueName: \"kubernetes.io/projected/51611d68-7db8-44af-81b2-87c97ab134cc-kube-api-access-7nh8c\") pod \"aws-ebs-csi-driver-node-h9hc8\" (UID: \"51611d68-7db8-44af-81b2-87c97ab134cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9hc8" Jul 09 15:02:33.496167 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493656 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ffeceed9-e851-4ef7-8ad8-603243ee5157-host\") pod \"node-ca-w9rr7\" (UID: \"ffeceed9-e851-4ef7-8ad8-603243ee5157\") " pod="openshift-image-registry/node-ca-w9rr7" Jul 09 15:02:33.496167 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493682 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-hostroot\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.496868 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493687 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fce18369-7aaf-4358-b5f5-46b0218924fd-dbus\") pod \"global-pull-secret-syncer-p8vsh\" (UID: \"fce18369-7aaf-4358-b5f5-46b0218924fd\") " pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:02:33.496868 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493702 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/67924f4e-f69d-48b8-ba23-e4c110bb24e5-hosts-file\") pod \"node-resolver-k9ppv\" (UID: \"67924f4e-f69d-48b8-ba23-e4c110bb24e5\") " pod="openshift-dns/node-resolver-k9ppv" Jul 09 15:02:33.496868 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493704 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/daad1095-91b7-47c6-bddf-dd0b6f01f8a9-agent-certs\") pod \"konnectivity-agent-9647m\" (UID: \"daad1095-91b7-47c6-bddf-dd0b6f01f8a9\") " pod="kube-system/konnectivity-agent-9647m" Jul 09 15:02:33.496868 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493727 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-host-slash\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.496868 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493743 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/51611d68-7db8-44af-81b2-87c97ab134cc-registration-dir\") pod \"aws-ebs-csi-driver-node-h9hc8\" (UID: \"51611d68-7db8-44af-81b2-87c97ab134cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9hc8" Jul 09 15:02:33.496868 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493768 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/51611d68-7db8-44af-81b2-87c97ab134cc-etc-selinux\") pod \"aws-ebs-csi-driver-node-h9hc8\" (UID: \"51611d68-7db8-44af-81b2-87c97ab134cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9hc8" Jul 09 15:02:33.496868 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493793 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/24e2d8f2-c395-4f83-9b5b-c71affe096ea-run\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.496868 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493817 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9bfc\" (UniqueName: \"kubernetes.io/projected/24e2d8f2-c395-4f83-9b5b-c71affe096ea-kube-api-access-v9bfc\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.496868 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493840 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/84a10578-4913-456b-9fb7-e6440914b677-ovnkube-config\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.496868 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493842 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gqbc\" (UniqueName: \"kubernetes.io/projected/37399ba7-ce43-46e3-b78e-73b29c2196be-kube-api-access-9gqbc\") pod \"network-metrics-daemon-jcctg\" (UID: \"37399ba7-ce43-46e3-b78e-73b29c2196be\") " pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:02:33.496868 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493909 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-host-cni-bin\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.496868 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493935 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ffeceed9-e851-4ef7-8ad8-603243ee5157-serviceca\") pod \"node-ca-w9rr7\" (UID: \"ffeceed9-e851-4ef7-8ad8-603243ee5157\") " pod="openshift-image-registry/node-ca-w9rr7" Jul 09 15:02:33.496868 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493956 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/24e2d8f2-c395-4f83-9b5b-c71affe096ea-etc-modprobe-d\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.496868 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493977 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/24e2d8f2-c395-4f83-9b5b-c71affe096ea-etc-sysctl-d\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.496868 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493998 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-systemd-units\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.496868 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.494019 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-run-systemd\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.496868 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.494043 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d7e1ab55-7472-458f-9078-fe9ca998cbd9-host-slash\") pod \"iptables-alerter-gnc2d\" (UID: \"d7e1ab55-7472-458f-9078-fe9ca998cbd9\") " pod="openshift-network-operator/iptables-alerter-gnc2d" Jul 09 15:02:33.496868 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.494040 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-os-release\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.497416 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.494078 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/51611d68-7db8-44af-81b2-87c97ab134cc-registration-dir\") pod \"aws-ebs-csi-driver-node-h9hc8\" (UID: \"51611d68-7db8-44af-81b2-87c97ab134cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9hc8" Jul 09 15:02:33.497416 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.494082 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-host-run-netns\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.497416 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.494124 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-host-run-netns\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.497416 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.494128 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6cf97264-90f5-4d6e-9e44-50e1431776a8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b9zn8\" (UID: \"6cf97264-90f5-4d6e-9e44-50e1431776a8\") " pod="openshift-multus/multus-additional-cni-plugins-b9zn8" Jul 09 15:02:33.497416 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.494137 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-hostroot\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.497416 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.493996 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ba00eb64-f303-4c0c-9292-88611e2bf9c0-cni-binary-copy\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.497416 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.494099 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ffeceed9-e851-4ef7-8ad8-603243ee5157-host\") pod \"node-ca-w9rr7\" (UID: \"ffeceed9-e851-4ef7-8ad8-603243ee5157\") " pod="openshift-image-registry/node-ca-w9rr7" Jul 09 15:02:33.497416 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.494188 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-run-systemd\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.497416 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.494195 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-etc-openvswitch\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.497416 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.494233 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/51611d68-7db8-44af-81b2-87c97ab134cc-etc-selinux\") pod \"aws-ebs-csi-driver-node-h9hc8\" (UID: \"51611d68-7db8-44af-81b2-87c97ab134cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9hc8" Jul 09 15:02:33.497416 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.494246 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/24e2d8f2-c395-4f83-9b5b-c71affe096ea-run\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.497416 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.494247 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ba00eb64-f303-4c0c-9292-88611e2bf9c0-os-release\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.497416 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.494273 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-host-cni-bin\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.497416 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.494366 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/24e2d8f2-c395-4f83-9b5b-c71affe096ea-etc-sysctl-d\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.497416 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.494378 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-var-lib-openvswitch\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.497416 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.494411 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clg8w\" (UniqueName: \"kubernetes.io/projected/d7e1ab55-7472-458f-9078-fe9ca998cbd9-kube-api-access-clg8w\") pod \"iptables-alerter-gnc2d\" (UID: \"d7e1ab55-7472-458f-9078-fe9ca998cbd9\") " pod="openshift-network-operator/iptables-alerter-gnc2d" Jul 09 15:02:33.497416 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.494434 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/24e2d8f2-c395-4f83-9b5b-c71affe096ea-etc-modprobe-d\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.497416 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.494439 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24e2d8f2-c395-4f83-9b5b-c71affe096ea-etc-kubernetes\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.497922 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.494475 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/84a10578-4913-456b-9fb7-e6440914b677-systemd-units\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.497922 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.494485 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24e2d8f2-c395-4f83-9b5b-c71affe096ea-etc-kubernetes\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.497922 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.494500 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/24e2d8f2-c395-4f83-9b5b-c71affe096ea-sys\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.497922 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.494527 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/84a10578-4913-456b-9fb7-e6440914b677-ovn-node-metrics-cert\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.497922 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.494602 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/24e2d8f2-c395-4f83-9b5b-c71affe096ea-sys\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.497922 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.494682 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6cf97264-90f5-4d6e-9e44-50e1431776a8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b9zn8\" (UID: \"6cf97264-90f5-4d6e-9e44-50e1431776a8\") " pod="openshift-multus/multus-additional-cni-plugins-b9zn8" Jul 09 15:02:33.497922 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.495292 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ffeceed9-e851-4ef7-8ad8-603243ee5157-serviceca\") pod \"node-ca-w9rr7\" (UID: \"ffeceed9-e851-4ef7-8ad8-603243ee5157\") " pod="openshift-image-registry/node-ca-w9rr7" Jul 09 15:02:33.497922 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.496355 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24e2d8f2-c395-4f83-9b5b-c71affe096ea-tmp\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.497922 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.496375 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/24e2d8f2-c395-4f83-9b5b-c71affe096ea-etc-tuned\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:33.497922 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.496621 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/daad1095-91b7-47c6-bddf-dd0b6f01f8a9-agent-certs\") pod \"konnectivity-agent-9647m\" (UID: \"daad1095-91b7-47c6-bddf-dd0b6f01f8a9\") " pod="kube-system/konnectivity-agent-9647m" Jul 09 15:02:33.497922 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.496852 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/84a10578-4913-456b-9fb7-e6440914b677-ovn-node-metrics-cert\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.499715 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:33.499698 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 15:02:33.499757 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:33.499721 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 15:02:33.499757 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:33.499731 2567 projected.go:196] Error preparing data for projected volume kube-api-access-r4c4r for pod openshift-network-diagnostics/network-check-target-96fg2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 15:02:33.499819 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:33.499782 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d469078c-492e-4dbd-843f-7d1699fa1ce9-kube-api-access-r4c4r podName:d469078c-492e-4dbd-843f-7d1699fa1ce9 nodeName:}" failed. No retries permitted until 2026-07-09 15:02:33.999769207 +0000 UTC m=+2.091213394 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-r4c4r" (UniqueName: "kubernetes.io/projected/d469078c-492e-4dbd-843f-7d1699fa1ce9-kube-api-access-r4c4r") pod "network-check-target-96fg2" (UID: "d469078c-492e-4dbd-843f-7d1699fa1ce9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 15:02:33.501651 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.501633 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-828gn\" (UniqueName: \"kubernetes.io/projected/67924f4e-f69d-48b8-ba23-e4c110bb24e5-kube-api-access-828gn\") pod \"node-resolver-k9ppv\" (UID: \"67924f4e-f69d-48b8-ba23-e4c110bb24e5\") " pod="openshift-dns/node-resolver-k9ppv" Jul 09 15:02:33.502013 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.501995 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzf5c\" (UniqueName: \"kubernetes.io/projected/ffeceed9-e851-4ef7-8ad8-603243ee5157-kube-api-access-xzf5c\") pod \"node-ca-w9rr7\" (UID: \"ffeceed9-e851-4ef7-8ad8-603243ee5157\") " pod="openshift-image-registry/node-ca-w9rr7" Jul 09 15:02:33.502109 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.502021 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvdl8\" (UniqueName: \"kubernetes.io/projected/ba00eb64-f303-4c0c-9292-88611e2bf9c0-kube-api-access-rvdl8\") pod \"multus-54jgr\" (UID: \"ba00eb64-f303-4c0c-9292-88611e2bf9c0\") " pod="openshift-multus/multus-54jgr" Jul 09 15:02:33.504143 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.504117 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwnrb\" (UniqueName: \"kubernetes.io/projected/84a10578-4913-456b-9fb7-e6440914b677-kube-api-access-mwnrb\") pod \"ovnkube-node-hml7m\" (UID: \"84a10578-4913-456b-9fb7-e6440914b677\") " pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:33.505485 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.505453 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzmj8\" (UniqueName: \"kubernetes.io/projected/6cf97264-90f5-4d6e-9e44-50e1431776a8-kube-api-access-gzmj8\") pod \"multus-additional-cni-plugins-b9zn8\" (UID: \"6cf97264-90f5-4d6e-9e44-50e1431776a8\") " pod="openshift-multus/multus-additional-cni-plugins-b9zn8" Jul 09 15:02:33.505485 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.505470 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gqbc\" (UniqueName: \"kubernetes.io/projected/37399ba7-ce43-46e3-b78e-73b29c2196be-kube-api-access-9gqbc\") pod \"network-metrics-daemon-jcctg\" (UID: \"37399ba7-ce43-46e3-b78e-73b29c2196be\") " pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:02:33.505636 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.505590 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clg8w\" (UniqueName: \"kubernetes.io/projected/d7e1ab55-7472-458f-9078-fe9ca998cbd9-kube-api-access-clg8w\") pod \"iptables-alerter-gnc2d\" (UID: \"d7e1ab55-7472-458f-9078-fe9ca998cbd9\") " pod="openshift-network-operator/iptables-alerter-gnc2d" Jul 09 15:02:33.506146 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.506127 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nh8c\" (UniqueName: \"kubernetes.io/projected/51611d68-7db8-44af-81b2-87c97ab134cc-kube-api-access-7nh8c\") pod \"aws-ebs-csi-driver-node-h9hc8\" (UID: \"51611d68-7db8-44af-81b2-87c97ab134cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9hc8" Jul 09 15:02:33.506375 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:33.506353 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9bfc\" (UniqueName: \"kubernetes.io/projected/24e2d8f2-c395-4f83-9b5b-c71affe096ea-kube-api-access-v9bfc\") pod \"tuned-6r75j\" (UID: \"24e2d8f2-c395-4f83-9b5b-c71affe096ea\") " pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:34.952952 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:34.952866 2567 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 15:02:34.978268 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:34.978243 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9hc8" Jul 09 15:02:34.987951 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:34.987924 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51611d68_7db8_44af_81b2_87c97ab134cc.slice/crio-efed742a482b043a9f73423440459e28b28138610ee5cd7344f34f73ce5a404e WatchSource:0}: Error finding container efed742a482b043a9f73423440459e28b28138610ee5cd7344f34f73ce5a404e: Status 404 returned error can't find the container with id efed742a482b043a9f73423440459e28b28138610ee5cd7344f34f73ce5a404e Jul 09 15:02:34.993227 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:34.993207 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6r75j" Jul 09 15:02:34.999776 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:34.999757 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-b9zn8" Jul 09 15:02:35.000510 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:35.000490 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24e2d8f2_c395_4f83_9b5b_c71affe096ea.slice/crio-e00b53f257531238aaf766de1586a67c64e44831b40949c533b744c18ba96f6b WatchSource:0}: Error finding container e00b53f257531238aaf766de1586a67c64e44831b40949c533b744c18ba96f6b: Status 404 returned error can't find the container with id e00b53f257531238aaf766de1586a67c64e44831b40949c533b744c18ba96f6b Jul 09 15:02:35.008704 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:35.008684 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cf97264_90f5_4d6e_9e44_50e1431776a8.slice/crio-4aa372b63b59a89dc2991f5ac5a9ece0826a79dec4c9ed2f47e1f956c3a50ab0 WatchSource:0}: Error finding container 4aa372b63b59a89dc2991f5ac5a9ece0826a79dec4c9ed2f47e1f956c3a50ab0: Status 404 returned error can't find the container with id 4aa372b63b59a89dc2991f5ac5a9ece0826a79dec4c9ed2f47e1f956c3a50ab0 Jul 09 15:02:35.017444 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:35.017427 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:02:35.020259 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:35.020242 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9647m" Jul 09 15:02:35.025400 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:35.025381 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84a10578_4913_456b_9fb7_e6440914b677.slice/crio-1dbaa4b726298ef94ff13b86b6d4cb84a35d436d058d009cb0d6dc9fdb388116 WatchSource:0}: Error finding container 1dbaa4b726298ef94ff13b86b6d4cb84a35d436d058d009cb0d6dc9fdb388116: Status 404 returned error can't find the container with id 1dbaa4b726298ef94ff13b86b6d4cb84a35d436d058d009cb0d6dc9fdb388116 Jul 09 15:02:35.029123 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:35.029093 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaad1095_91b7_47c6_bddf_dd0b6f01f8a9.slice/crio-8af1cc705e5846cbacfda96a03e97ca355c24c955c4e30964469ec789317f332 WatchSource:0}: Error finding container 8af1cc705e5846cbacfda96a03e97ca355c24c955c4e30964469ec789317f332: Status 404 returned error can't find the container with id 8af1cc705e5846cbacfda96a03e97ca355c24c955c4e30964469ec789317f332 Jul 09 15:02:35.047591 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:35.047568 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-k9ppv" Jul 09 15:02:35.057672 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:35.057647 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67924f4e_f69d_48b8_ba23_e4c110bb24e5.slice/crio-da310659c7648cff8761986b8d5ed80ef62b2eacfd896be68421b846467f1974 WatchSource:0}: Error finding container da310659c7648cff8761986b8d5ed80ef62b2eacfd896be68421b846467f1974: Status 404 returned error can't find the container with id da310659c7648cff8761986b8d5ed80ef62b2eacfd896be68421b846467f1974 Jul 09 15:02:35.069873 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:35.069847 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-w9rr7" Jul 09 15:02:35.078077 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:35.078054 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffeceed9_e851_4ef7_8ad8_603243ee5157.slice/crio-c4cd41f0a4bda7124d525da373f5587c40ac37195628c879c9c1eb812f152062 WatchSource:0}: Error finding container c4cd41f0a4bda7124d525da373f5587c40ac37195628c879c9c1eb812f152062: Status 404 returned error can't find the container with id c4cd41f0a4bda7124d525da373f5587c40ac37195628c879c9c1eb812f152062 Jul 09 15:02:35.087945 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:35.087923 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-54jgr" Jul 09 15:02:35.095130 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:35.095103 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-gnc2d" Jul 09 15:02:35.095706 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:35.095684 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba00eb64_f303_4c0c_9292_88611e2bf9c0.slice/crio-fcb2ca8819283f63b3331937bcae92133455aa5212e62aa623b73dc6d4370f26 WatchSource:0}: Error finding container fcb2ca8819283f63b3331937bcae92133455aa5212e62aa623b73dc6d4370f26: Status 404 returned error can't find the container with id fcb2ca8819283f63b3331937bcae92133455aa5212e62aa623b73dc6d4370f26 Jul 09 15:02:35.103094 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:35.103068 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7e1ab55_7472_458f_9078_fe9ca998cbd9.slice/crio-c57a0b57747bdb201d8b5ec4fdccd356bd01d1ca3520dd1c1f125078e96e9950 WatchSource:0}: Error finding container c57a0b57747bdb201d8b5ec4fdccd356bd01d1ca3520dd1c1f125078e96e9950: Status 404 returned error can't find the container with id c57a0b57747bdb201d8b5ec4fdccd356bd01d1ca3520dd1c1f125078e96e9950 Jul 09 15:02:35.358253 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:35.358091 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fce18369-7aaf-4358-b5f5-46b0218924fd-original-pull-secret\") pod \"global-pull-secret-syncer-p8vsh\" (UID: \"fce18369-7aaf-4358-b5f5-46b0218924fd\") " pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:02:35.358253 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:35.358142 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37399ba7-ce43-46e3-b78e-73b29c2196be-metrics-certs\") pod \"network-metrics-daemon-jcctg\" (UID: \"37399ba7-ce43-46e3-b78e-73b29c2196be\") " pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:02:35.358253 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:35.358197 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 15:02:35.358506 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:35.358279 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fce18369-7aaf-4358-b5f5-46b0218924fd-original-pull-secret podName:fce18369-7aaf-4358-b5f5-46b0218924fd nodeName:}" failed. No retries permitted until 2026-07-09 15:02:36.358258724 +0000 UTC m=+3.088765198 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fce18369-7aaf-4358-b5f5-46b0218924fd-original-pull-secret") pod "global-pull-secret-syncer-p8vsh" (UID: "fce18369-7aaf-4358-b5f5-46b0218924fd") : object "kube-system"/"original-pull-secret" not registered Jul 09 15:02:35.358506 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:35.358290 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 15:02:35.358506 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:35.358356 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37399ba7-ce43-46e3-b78e-73b29c2196be-metrics-certs podName:37399ba7-ce43-46e3-b78e-73b29c2196be nodeName:}" failed. No retries permitted until 2026-07-09 15:02:36.358341311 +0000 UTC m=+3.088847784 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37399ba7-ce43-46e3-b78e-73b29c2196be-metrics-certs") pod "network-metrics-daemon-jcctg" (UID: "37399ba7-ce43-46e3-b78e-73b29c2196be") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 15:02:35.458627 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:35.458591 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4c4r\" (UniqueName: \"kubernetes.io/projected/d469078c-492e-4dbd-843f-7d1699fa1ce9-kube-api-access-r4c4r\") pod \"network-check-target-96fg2\" (UID: \"d469078c-492e-4dbd-843f-7d1699fa1ce9\") " pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:02:35.458849 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:35.458830 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 15:02:35.458924 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:35.458860 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 15:02:35.458924 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:35.458873 2567 projected.go:196] Error preparing data for projected volume kube-api-access-r4c4r for pod openshift-network-diagnostics/network-check-target-96fg2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 15:02:35.459053 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:35.458941 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d469078c-492e-4dbd-843f-7d1699fa1ce9-kube-api-access-r4c4r podName:d469078c-492e-4dbd-843f-7d1699fa1ce9 nodeName:}" failed. No retries permitted until 2026-07-09 15:02:36.458921559 +0000 UTC m=+3.189428027 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-r4c4r" (UniqueName: "kubernetes.io/projected/d469078c-492e-4dbd-843f-7d1699fa1ce9-kube-api-access-r4c4r") pod "network-check-target-96fg2" (UID: "d469078c-492e-4dbd-843f-7d1699fa1ce9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 15:02:35.703622 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:35.703439 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 14:57:34 +0000 UTC" deadline="2027-12-14 12:49:41.621861328 +0000 UTC" Jul 09 15:02:35.703622 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:35.703503 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12549h47m5.918362807s" Jul 09 15:02:35.825119 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:35.825058 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:02:35.825309 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:35.825197 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jcctg" podUID="37399ba7-ce43-46e3-b78e-73b29c2196be" Jul 09 15:02:35.831030 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:35.830966 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" event={"ID":"84a10578-4913-456b-9fb7-e6440914b677","Type":"ContainerStarted","Data":"1dbaa4b726298ef94ff13b86b6d4cb84a35d436d058d009cb0d6dc9fdb388116"} Jul 09 15:02:35.832277 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:35.832248 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b9zn8" event={"ID":"6cf97264-90f5-4d6e-9e44-50e1431776a8","Type":"ContainerStarted","Data":"4aa372b63b59a89dc2991f5ac5a9ece0826a79dec4c9ed2f47e1f956c3a50ab0"} Jul 09 15:02:35.834546 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:35.834518 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-114.ec2.internal" event={"ID":"b4114c320ef9f8839a18178df34d18c8","Type":"ContainerStarted","Data":"653b7a381c909ddeeb38d26e6fb15727ec52b6efd250518fbe243480f8601be3"} Jul 09 15:02:35.837105 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:35.837055 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9647m" event={"ID":"daad1095-91b7-47c6-bddf-dd0b6f01f8a9","Type":"ContainerStarted","Data":"8af1cc705e5846cbacfda96a03e97ca355c24c955c4e30964469ec789317f332"} Jul 09 15:02:35.840296 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:35.840270 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6r75j" event={"ID":"24e2d8f2-c395-4f83-9b5b-c71affe096ea","Type":"ContainerStarted","Data":"e00b53f257531238aaf766de1586a67c64e44831b40949c533b744c18ba96f6b"} Jul 09 15:02:35.844456 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:35.844383 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9hc8" event={"ID":"51611d68-7db8-44af-81b2-87c97ab134cc","Type":"ContainerStarted","Data":"efed742a482b043a9f73423440459e28b28138610ee5cd7344f34f73ce5a404e"} Jul 09 15:02:35.846542 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:35.846519 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-gnc2d" event={"ID":"d7e1ab55-7472-458f-9078-fe9ca998cbd9","Type":"ContainerStarted","Data":"c57a0b57747bdb201d8b5ec4fdccd356bd01d1ca3520dd1c1f125078e96e9950"} Jul 09 15:02:35.848313 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:35.848288 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-54jgr" event={"ID":"ba00eb64-f303-4c0c-9292-88611e2bf9c0","Type":"ContainerStarted","Data":"fcb2ca8819283f63b3331937bcae92133455aa5212e62aa623b73dc6d4370f26"} Jul 09 15:02:35.854254 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:35.854229 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-w9rr7" event={"ID":"ffeceed9-e851-4ef7-8ad8-603243ee5157","Type":"ContainerStarted","Data":"c4cd41f0a4bda7124d525da373f5587c40ac37195628c879c9c1eb812f152062"} Jul 09 15:02:35.863801 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:35.863761 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-k9ppv" event={"ID":"67924f4e-f69d-48b8-ba23-e4c110bb24e5","Type":"ContainerStarted","Data":"da310659c7648cff8761986b8d5ed80ef62b2eacfd896be68421b846467f1974"} Jul 09 15:02:35.998321 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:35.998241 2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 15:02:36.059311 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:02:36.059274 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3942009bd02b28965ce62ce1b271d1a.slice/crio-bfee9a0eb0cfd1b9e861182eba8b5f1a7fcc790c1bb4a21f8a102f1b8cb3d1f9 WatchSource:0}: Error finding container bfee9a0eb0cfd1b9e861182eba8b5f1a7fcc790c1bb4a21f8a102f1b8cb3d1f9: Status 404 returned error can't find the container with id bfee9a0eb0cfd1b9e861182eba8b5f1a7fcc790c1bb4a21f8a102f1b8cb3d1f9 Jul 09 15:02:36.367092 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:36.366966 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fce18369-7aaf-4358-b5f5-46b0218924fd-original-pull-secret\") pod \"global-pull-secret-syncer-p8vsh\" (UID: \"fce18369-7aaf-4358-b5f5-46b0218924fd\") " pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:02:36.367092 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:36.367027 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37399ba7-ce43-46e3-b78e-73b29c2196be-metrics-certs\") pod \"network-metrics-daemon-jcctg\" (UID: \"37399ba7-ce43-46e3-b78e-73b29c2196be\") " pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:02:36.367317 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:36.367196 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 15:02:36.367317 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:36.367256 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37399ba7-ce43-46e3-b78e-73b29c2196be-metrics-certs podName:37399ba7-ce43-46e3-b78e-73b29c2196be nodeName:}" failed. No retries permitted until 2026-07-09 15:02:38.367237375 +0000 UTC m=+5.097743861 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37399ba7-ce43-46e3-b78e-73b29c2196be-metrics-certs") pod "network-metrics-daemon-jcctg" (UID: "37399ba7-ce43-46e3-b78e-73b29c2196be") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 15:02:36.367642 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:36.367627 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 15:02:36.367724 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:36.367674 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fce18369-7aaf-4358-b5f5-46b0218924fd-original-pull-secret podName:fce18369-7aaf-4358-b5f5-46b0218924fd nodeName:}" failed. No retries permitted until 2026-07-09 15:02:38.367661102 +0000 UTC m=+5.098167576 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fce18369-7aaf-4358-b5f5-46b0218924fd-original-pull-secret") pod "global-pull-secret-syncer-p8vsh" (UID: "fce18369-7aaf-4358-b5f5-46b0218924fd") : object "kube-system"/"original-pull-secret" not registered Jul 09 15:02:36.467611 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:36.467569 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4c4r\" (UniqueName: \"kubernetes.io/projected/d469078c-492e-4dbd-843f-7d1699fa1ce9-kube-api-access-r4c4r\") pod \"network-check-target-96fg2\" (UID: \"d469078c-492e-4dbd-843f-7d1699fa1ce9\") " pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:02:36.467808 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:36.467782 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 15:02:36.467808 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:36.467807 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 15:02:36.468004 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:36.467838 2567 projected.go:196] Error preparing data for projected volume kube-api-access-r4c4r for pod openshift-network-diagnostics/network-check-target-96fg2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 15:02:36.468004 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:36.467906 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d469078c-492e-4dbd-843f-7d1699fa1ce9-kube-api-access-r4c4r podName:d469078c-492e-4dbd-843f-7d1699fa1ce9 nodeName:}" failed. No retries permitted until 2026-07-09 15:02:38.467888113 +0000 UTC m=+5.198394586 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-r4c4r" (UniqueName: "kubernetes.io/projected/d469078c-492e-4dbd-843f-7d1699fa1ce9-kube-api-access-r4c4r") pod "network-check-target-96fg2" (UID: "d469078c-492e-4dbd-843f-7d1699fa1ce9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 15:02:36.704695 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:36.704599 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 14:57:34 +0000 UTC" deadline="2028-04-13 10:20:06.762750556 +0000 UTC" Jul 09 15:02:36.704695 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:36.704644 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15451h17m30.058111243s" Jul 09 15:02:36.823539 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:36.822848 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:02:36.823539 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:36.823000 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8vsh" podUID="fce18369-7aaf-4358-b5f5-46b0218924fd" Jul 09 15:02:36.823539 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:36.823399 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:02:36.823539 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:36.823486 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-96fg2" podUID="d469078c-492e-4dbd-843f-7d1699fa1ce9" Jul 09 15:02:36.868627 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:36.868588 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal" event={"ID":"c3942009bd02b28965ce62ce1b271d1a","Type":"ContainerStarted","Data":"bfee9a0eb0cfd1b9e861182eba8b5f1a7fcc790c1bb4a21f8a102f1b8cb3d1f9"} Jul 09 15:02:37.460437 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:37.460133 2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 15:02:37.823373 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:37.822790 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:02:37.823373 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:37.822954 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jcctg" podUID="37399ba7-ce43-46e3-b78e-73b29c2196be" Jul 09 15:02:38.386097 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:38.386029 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fce18369-7aaf-4358-b5f5-46b0218924fd-original-pull-secret\") pod \"global-pull-secret-syncer-p8vsh\" (UID: \"fce18369-7aaf-4358-b5f5-46b0218924fd\") " pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:02:38.386287 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:38.386115 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37399ba7-ce43-46e3-b78e-73b29c2196be-metrics-certs\") pod \"network-metrics-daemon-jcctg\" (UID: \"37399ba7-ce43-46e3-b78e-73b29c2196be\") " pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:02:38.386287 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:38.386208 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 15:02:38.386287 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:38.386244 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 15:02:38.386287 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:38.386285 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fce18369-7aaf-4358-b5f5-46b0218924fd-original-pull-secret podName:fce18369-7aaf-4358-b5f5-46b0218924fd nodeName:}" failed. No retries permitted until 2026-07-09 15:02:42.38626574 +0000 UTC m=+9.116772221 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fce18369-7aaf-4358-b5f5-46b0218924fd-original-pull-secret") pod "global-pull-secret-syncer-p8vsh" (UID: "fce18369-7aaf-4358-b5f5-46b0218924fd") : object "kube-system"/"original-pull-secret" not registered Jul 09 15:02:38.386521 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:38.386306 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37399ba7-ce43-46e3-b78e-73b29c2196be-metrics-certs podName:37399ba7-ce43-46e3-b78e-73b29c2196be nodeName:}" failed. No retries permitted until 2026-07-09 15:02:42.38629558 +0000 UTC m=+9.116802052 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37399ba7-ce43-46e3-b78e-73b29c2196be-metrics-certs") pod "network-metrics-daemon-jcctg" (UID: "37399ba7-ce43-46e3-b78e-73b29c2196be") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 15:02:38.486713 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:38.486672 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4c4r\" (UniqueName: \"kubernetes.io/projected/d469078c-492e-4dbd-843f-7d1699fa1ce9-kube-api-access-r4c4r\") pod \"network-check-target-96fg2\" (UID: \"d469078c-492e-4dbd-843f-7d1699fa1ce9\") " pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:02:38.486926 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:38.486872 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 15:02:38.486926 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:38.486896 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 15:02:38.486926 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:38.486910 2567 projected.go:196] Error preparing data for projected volume kube-api-access-r4c4r for pod openshift-network-diagnostics/network-check-target-96fg2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 15:02:38.487110 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:38.486985 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d469078c-492e-4dbd-843f-7d1699fa1ce9-kube-api-access-r4c4r podName:d469078c-492e-4dbd-843f-7d1699fa1ce9 nodeName:}" failed. No retries permitted until 2026-07-09 15:02:42.486965196 +0000 UTC m=+9.217471669 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-r4c4r" (UniqueName: "kubernetes.io/projected/d469078c-492e-4dbd-843f-7d1699fa1ce9-kube-api-access-r4c4r") pod "network-check-target-96fg2" (UID: "d469078c-492e-4dbd-843f-7d1699fa1ce9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 15:02:38.823074 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:38.822967 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:02:38.823241 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:38.823104 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8vsh" podUID="fce18369-7aaf-4358-b5f5-46b0218924fd" Jul 09 15:02:38.823241 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:38.822967 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:02:38.823241 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:38.823206 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-96fg2" podUID="d469078c-492e-4dbd-843f-7d1699fa1ce9" Jul 09 15:02:39.822573 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:39.822529 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:02:39.823065 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:39.822691 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jcctg" podUID="37399ba7-ce43-46e3-b78e-73b29c2196be" Jul 09 15:02:40.823213 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:40.822913 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:02:40.823213 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:40.822915 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:02:40.823213 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:40.823072 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-96fg2" podUID="d469078c-492e-4dbd-843f-7d1699fa1ce9" Jul 09 15:02:40.823213 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:40.823128 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8vsh" podUID="fce18369-7aaf-4358-b5f5-46b0218924fd" Jul 09 15:02:41.823611 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:41.822999 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:02:41.823611 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:41.823153 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jcctg" podUID="37399ba7-ce43-46e3-b78e-73b29c2196be" Jul 09 15:02:42.420770 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:42.420730 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fce18369-7aaf-4358-b5f5-46b0218924fd-original-pull-secret\") pod \"global-pull-secret-syncer-p8vsh\" (UID: \"fce18369-7aaf-4358-b5f5-46b0218924fd\") " pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:02:42.420975 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:42.420792 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37399ba7-ce43-46e3-b78e-73b29c2196be-metrics-certs\") pod \"network-metrics-daemon-jcctg\" (UID: \"37399ba7-ce43-46e3-b78e-73b29c2196be\") " pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:02:42.420975 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:42.420926 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 15:02:42.421099 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:42.420998 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fce18369-7aaf-4358-b5f5-46b0218924fd-original-pull-secret podName:fce18369-7aaf-4358-b5f5-46b0218924fd nodeName:}" failed. No retries permitted until 2026-07-09 15:02:50.42097786 +0000 UTC m=+17.151484357 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fce18369-7aaf-4358-b5f5-46b0218924fd-original-pull-secret") pod "global-pull-secret-syncer-p8vsh" (UID: "fce18369-7aaf-4358-b5f5-46b0218924fd") : object "kube-system"/"original-pull-secret" not registered Jul 09 15:02:42.421099 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:42.420934 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 15:02:42.421099 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:42.421082 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37399ba7-ce43-46e3-b78e-73b29c2196be-metrics-certs podName:37399ba7-ce43-46e3-b78e-73b29c2196be nodeName:}" failed. No retries permitted until 2026-07-09 15:02:50.421063258 +0000 UTC m=+17.151569731 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37399ba7-ce43-46e3-b78e-73b29c2196be-metrics-certs") pod "network-metrics-daemon-jcctg" (UID: "37399ba7-ce43-46e3-b78e-73b29c2196be") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 15:02:42.522030 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:42.521992 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4c4r\" (UniqueName: \"kubernetes.io/projected/d469078c-492e-4dbd-843f-7d1699fa1ce9-kube-api-access-r4c4r\") pod \"network-check-target-96fg2\" (UID: \"d469078c-492e-4dbd-843f-7d1699fa1ce9\") " pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:02:42.522223 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:42.522187 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 15:02:42.522223 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:42.522218 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 15:02:42.522341 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:42.522232 2567 projected.go:196] Error preparing data for projected volume kube-api-access-r4c4r for pod openshift-network-diagnostics/network-check-target-96fg2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 15:02:42.522341 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:42.522301 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d469078c-492e-4dbd-843f-7d1699fa1ce9-kube-api-access-r4c4r podName:d469078c-492e-4dbd-843f-7d1699fa1ce9 nodeName:}" failed. No retries permitted until 2026-07-09 15:02:50.522281787 +0000 UTC m=+17.252788261 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-r4c4r" (UniqueName: "kubernetes.io/projected/d469078c-492e-4dbd-843f-7d1699fa1ce9-kube-api-access-r4c4r") pod "network-check-target-96fg2" (UID: "d469078c-492e-4dbd-843f-7d1699fa1ce9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 15:02:42.823153 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:42.823046 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:02:42.823305 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:42.823170 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8vsh" podUID="fce18369-7aaf-4358-b5f5-46b0218924fd" Jul 09 15:02:42.823532 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:42.823383 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:02:42.823532 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:42.823494 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-96fg2" podUID="d469078c-492e-4dbd-843f-7d1699fa1ce9" Jul 09 15:02:43.823778 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:43.823401 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:02:43.823778 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:43.823528 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jcctg" podUID="37399ba7-ce43-46e3-b78e-73b29c2196be" Jul 09 15:02:44.823162 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:44.823027 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:02:44.823162 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:44.823038 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:02:44.823381 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:44.823166 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8vsh" podUID="fce18369-7aaf-4358-b5f5-46b0218924fd" Jul 09 15:02:44.823381 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:44.823247 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-96fg2" podUID="d469078c-492e-4dbd-843f-7d1699fa1ce9" Jul 09 15:02:45.822336 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:45.822278 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:02:45.822779 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:45.822444 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jcctg" podUID="37399ba7-ce43-46e3-b78e-73b29c2196be" Jul 09 15:02:46.822364 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:46.822323 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:02:46.822364 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:46.822360 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:02:46.822846 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:46.822471 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8vsh" podUID="fce18369-7aaf-4358-b5f5-46b0218924fd" Jul 09 15:02:46.822846 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:46.822587 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-96fg2" podUID="d469078c-492e-4dbd-843f-7d1699fa1ce9" Jul 09 15:02:47.822829 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:47.822791 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:02:47.823298 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:47.822950 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jcctg" podUID="37399ba7-ce43-46e3-b78e-73b29c2196be" Jul 09 15:02:48.822254 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:48.822223 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:02:48.822438 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:48.822223 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:02:48.822438 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:48.822345 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8vsh" podUID="fce18369-7aaf-4358-b5f5-46b0218924fd" Jul 09 15:02:48.822555 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:48.822428 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-96fg2" podUID="d469078c-492e-4dbd-843f-7d1699fa1ce9" Jul 09 15:02:49.823127 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:49.823086 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:02:49.823583 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:49.823248 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jcctg" podUID="37399ba7-ce43-46e3-b78e-73b29c2196be" Jul 09 15:02:50.482873 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:50.482830 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fce18369-7aaf-4358-b5f5-46b0218924fd-original-pull-secret\") pod \"global-pull-secret-syncer-p8vsh\" (UID: \"fce18369-7aaf-4358-b5f5-46b0218924fd\") " pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:02:50.483087 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:50.482888 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37399ba7-ce43-46e3-b78e-73b29c2196be-metrics-certs\") pod \"network-metrics-daemon-jcctg\" (UID: \"37399ba7-ce43-46e3-b78e-73b29c2196be\") " pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:02:50.483087 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:50.482988 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 15:02:50.483087 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:50.483010 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 15:02:50.483087 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:50.483069 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37399ba7-ce43-46e3-b78e-73b29c2196be-metrics-certs podName:37399ba7-ce43-46e3-b78e-73b29c2196be nodeName:}" failed. No retries permitted until 2026-07-09 15:03:06.483050524 +0000 UTC m=+33.213557140 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37399ba7-ce43-46e3-b78e-73b29c2196be-metrics-certs") pod "network-metrics-daemon-jcctg" (UID: "37399ba7-ce43-46e3-b78e-73b29c2196be") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 15:02:50.483087 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:50.483087 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fce18369-7aaf-4358-b5f5-46b0218924fd-original-pull-secret podName:fce18369-7aaf-4358-b5f5-46b0218924fd nodeName:}" failed. No retries permitted until 2026-07-09 15:03:06.483078337 +0000 UTC m=+33.213584807 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fce18369-7aaf-4358-b5f5-46b0218924fd-original-pull-secret") pod "global-pull-secret-syncer-p8vsh" (UID: "fce18369-7aaf-4358-b5f5-46b0218924fd") : object "kube-system"/"original-pull-secret" not registered Jul 09 15:02:50.584166 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:50.584128 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4c4r\" (UniqueName: \"kubernetes.io/projected/d469078c-492e-4dbd-843f-7d1699fa1ce9-kube-api-access-r4c4r\") pod \"network-check-target-96fg2\" (UID: \"d469078c-492e-4dbd-843f-7d1699fa1ce9\") " pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:02:50.584379 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:50.584324 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 15:02:50.584379 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:50.584351 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 15:02:50.584379 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:50.584364 2567 projected.go:196] Error preparing data for projected volume kube-api-access-r4c4r for pod openshift-network-diagnostics/network-check-target-96fg2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 15:02:50.584536 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:50.584429 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d469078c-492e-4dbd-843f-7d1699fa1ce9-kube-api-access-r4c4r podName:d469078c-492e-4dbd-843f-7d1699fa1ce9 nodeName:}" failed. No retries permitted until 2026-07-09 15:03:06.584410359 +0000 UTC m=+33.314916846 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-r4c4r" (UniqueName: "kubernetes.io/projected/d469078c-492e-4dbd-843f-7d1699fa1ce9-kube-api-access-r4c4r") pod "network-check-target-96fg2" (UID: "d469078c-492e-4dbd-843f-7d1699fa1ce9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 15:02:50.822877 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:50.822776 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:02:50.822877 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:50.822846 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:02:50.823170 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:50.822932 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8vsh" podUID="fce18369-7aaf-4358-b5f5-46b0218924fd" Jul 09 15:02:50.823170 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:50.822968 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-96fg2" podUID="d469078c-492e-4dbd-843f-7d1699fa1ce9" Jul 09 15:02:51.822700 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:51.822659 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:02:51.822937 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:51.822811 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jcctg" podUID="37399ba7-ce43-46e3-b78e-73b29c2196be" Jul 09 15:02:52.823184 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:52.823146 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:02:52.823666 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:52.823154 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:02:52.823666 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:52.823283 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-96fg2" podUID="d469078c-492e-4dbd-843f-7d1699fa1ce9" Jul 09 15:02:52.823666 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:52.823396 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8vsh" podUID="fce18369-7aaf-4358-b5f5-46b0218924fd" Jul 09 15:02:53.824176 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:53.824138 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:02:53.824566 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:53.824270 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jcctg" podUID="37399ba7-ce43-46e3-b78e-73b29c2196be" Jul 09 15:02:54.822304 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:54.822276 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:02:54.822454 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:54.822276 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:02:54.822454 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:54.822369 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8vsh" podUID="fce18369-7aaf-4358-b5f5-46b0218924fd" Jul 09 15:02:54.822554 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:54.822468 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-96fg2" podUID="d469078c-492e-4dbd-843f-7d1699fa1ce9" Jul 09 15:02:54.910417 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:54.910081 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-54jgr" event={"ID":"ba00eb64-f303-4c0c-9292-88611e2bf9c0","Type":"ContainerStarted","Data":"15ddff6b9abc5c1a341708831d51a5355fc46efa0c57e4117e4d515c29d980c1"} Jul 09 15:02:54.911876 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:54.911772 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-w9rr7" event={"ID":"ffeceed9-e851-4ef7-8ad8-603243ee5157","Type":"ContainerStarted","Data":"4b2c55dee2603a72e2477ec91e894c5bdd8fdcaa35187157693c1a546f984f00"} Jul 09 15:02:54.913544 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:54.913522 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-k9ppv" event={"ID":"67924f4e-f69d-48b8-ba23-e4c110bb24e5","Type":"ContainerStarted","Data":"e87fa919c2099f92fc12111b15907025426b57a8f01ec9ab7f6bfe6bd42375ef"} Jul 09 15:02:54.915010 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:54.914992 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hml7m_84a10578-4913-456b-9fb7-e6440914b677/ovn-acl-logging/0.log" Jul 09 15:02:54.915307 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:54.915286 2567 generic.go:356] "Generic (PLEG): container finished" podID="84a10578-4913-456b-9fb7-e6440914b677" containerID="9b267100f369523ac917f6ad533ae8c207d8b785d2a9c562611b4e4d14f96ddf" exitCode=1 Jul 09 15:02:54.915395 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:54.915348 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" event={"ID":"84a10578-4913-456b-9fb7-e6440914b677","Type":"ContainerDied","Data":"9b267100f369523ac917f6ad533ae8c207d8b785d2a9c562611b4e4d14f96ddf"} Jul 09 15:02:54.915460 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:54.915403 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" event={"ID":"84a10578-4913-456b-9fb7-e6440914b677","Type":"ContainerStarted","Data":"01ec16d714f6e1f479c609c5f2889df78123859b9f090d95199e50a754089d7c"} Jul 09 15:02:54.916738 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:54.916717 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b9zn8" event={"ID":"6cf97264-90f5-4d6e-9e44-50e1431776a8","Type":"ContainerStarted","Data":"ffc741d94ea2393ee4e79a5cae939e82ce14e4219d355acfec1838e8388ee579"} Jul 09 15:02:54.918154 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:54.918106 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-114.ec2.internal" event={"ID":"b4114c320ef9f8839a18178df34d18c8","Type":"ContainerStarted","Data":"4795fbabb529761eee20bcaee1c407499539420f4297c8487590448926410eac"} Jul 09 15:02:54.919643 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:54.919618 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9647m" event={"ID":"daad1095-91b7-47c6-bddf-dd0b6f01f8a9","Type":"ContainerStarted","Data":"9bc4301ac9489dc624181c02f0e302254f1160bc54c0ac27d873de487387a2c1"} Jul 09 15:02:54.920956 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:54.920934 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6r75j" event={"ID":"24e2d8f2-c395-4f83-9b5b-c71affe096ea","Type":"ContainerStarted","Data":"c0cae181999b98d5c6ff2b180cd4422d9599916d4960660581f955cc0dcc73b8"} Jul 09 15:02:54.922222 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:54.922205 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9hc8" event={"ID":"51611d68-7db8-44af-81b2-87c97ab134cc","Type":"ContainerStarted","Data":"4749518002ad9c99a5d68f37fb6b24d85bbd382b3dbb7f57919385fbe9e89703"} Jul 09 15:02:54.940384 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:54.940238 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-54jgr" podStartSLOduration=2.939861053 podStartE2EDuration="21.940224443s" totalImagesPullingTime="19.00036339s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:02:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:02:54.939981661 +0000 UTC m=+21.670488155" watchObservedRunningTime="2026-07-09 15:02:54.940224443 +0000 UTC m=+21.670730933" Jul 09 15:02:54.988098 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:54.988045 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9647m" podStartSLOduration=7.434252564 podStartE2EDuration="21.988025185s" totalImagesPullingTime="14.553772621s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:02:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:02:54.960498481 +0000 UTC m=+21.691004972" watchObservedRunningTime="2026-07-09 15:02:54.988025185 +0000 UTC m=+21.718531670" Jul 09 15:02:55.032310 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:55.032253 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-k9ppv" podStartSLOduration=3.021493881 podStartE2EDuration="22.032235245s" totalImagesPullingTime="19.010741364s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:02:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:02:55.007728034 +0000 UTC m=+21.738234524" watchObservedRunningTime="2026-07-09 15:02:55.032235245 +0000 UTC m=+21.762741737" Jul 09 15:02:55.032450 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:55.032332 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-w9rr7" podStartSLOduration=3.061672077 podStartE2EDuration="22.032328573s" totalImagesPullingTime="18.970656496s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:02:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:02:55.032163003 +0000 UTC m=+21.762669494" watchObservedRunningTime="2026-07-09 15:02:55.032328573 +0000 UTC m=+21.762835063" Jul 09 15:02:55.047097 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:55.047047 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-114.ec2.internal" podStartSLOduration=21.04703257 podStartE2EDuration="21.04703257s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:02:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:02:55.046885672 +0000 UTC m=+21.777392163" watchObservedRunningTime="2026-07-09 15:02:55.04703257 +0000 UTC m=+21.777539060" Jul 09 15:02:55.089758 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:55.089711 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-6r75j" podStartSLOduration=7.507881181 podStartE2EDuration="22.089688435s" totalImagesPullingTime="14.581807254s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:02:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:02:55.089486744 +0000 UTC m=+21.819993239" watchObservedRunningTime="2026-07-09 15:02:55.089688435 +0000 UTC m=+21.820194926" Jul 09 15:02:55.118307 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:55.118265 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:02:55.226607 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:55.226581 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:02:55.822520 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:55.822485 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:02:55.822795 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:55.822606 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jcctg" podUID="37399ba7-ce43-46e3-b78e-73b29c2196be" Jul 09 15:02:55.928377 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:55.928349 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hml7m_84a10578-4913-456b-9fb7-e6440914b677/ovn-acl-logging/0.log" Jul 09 15:02:55.928946 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:55.928695 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" event={"ID":"84a10578-4913-456b-9fb7-e6440914b677","Type":"ContainerStarted","Data":"5c02ee3efc3ff4e1bc7cf58e5ef02819de02ae33239fd84f442d64890cecc5fa"} Jul 09 15:02:55.928946 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:55.928837 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" event={"ID":"84a10578-4913-456b-9fb7-e6440914b677","Type":"ContainerStarted","Data":"4f9df088c5951965ab1038bc53ffad7fa5b4fe78fb5f88e9e4bc1f210fec2913"} Jul 09 15:02:55.928946 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:55.928873 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" event={"ID":"84a10578-4913-456b-9fb7-e6440914b677","Type":"ContainerStarted","Data":"2d6fe0f7448d4d356273e752c371bd07d0443e29369e2a6468ad5ecc20d1c9e4"} Jul 09 15:02:55.928946 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:55.928900 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" event={"ID":"84a10578-4913-456b-9fb7-e6440914b677","Type":"ContainerStarted","Data":"7fd730a8cb55925132f8ad0b3505bc5555d178fc2cf235638417d58fc42c2883"} Jul 09 15:02:55.930071 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:55.930046 2567 generic.go:356] "Generic (PLEG): container finished" podID="6cf97264-90f5-4d6e-9e44-50e1431776a8" containerID="ffc741d94ea2393ee4e79a5cae939e82ce14e4219d355acfec1838e8388ee579" exitCode=0 Jul 09 15:02:55.930188 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:55.930123 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b9zn8" event={"ID":"6cf97264-90f5-4d6e-9e44-50e1431776a8","Type":"ContainerDied","Data":"ffc741d94ea2393ee4e79a5cae939e82ce14e4219d355acfec1838e8388ee579"} Jul 09 15:02:55.931576 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:55.931555 2567 generic.go:356] "Generic (PLEG): container finished" podID="c3942009bd02b28965ce62ce1b271d1a" containerID="a411204fcf012a775c6e80eb9a5db125ce44f4ffb1d4137ef674a8fce1cfe4a7" exitCode=0 Jul 09 15:02:55.931656 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:55.931622 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal" event={"ID":"c3942009bd02b28965ce62ce1b271d1a","Type":"ContainerDied","Data":"a411204fcf012a775c6e80eb9a5db125ce44f4ffb1d4137ef674a8fce1cfe4a7"} Jul 09 15:02:55.932914 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:55.932882 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-gnc2d" event={"ID":"d7e1ab55-7472-458f-9078-fe9ca998cbd9","Type":"ContainerStarted","Data":"301ca848e679f0651b87194355014ad89179c4225b2970f7f2013043e54d09dc"} Jul 09 15:02:55.970775 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:55.970717 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-gnc2d" podStartSLOduration=8.490980406 podStartE2EDuration="22.970701164s" totalImagesPullingTime="14.479720758s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:02:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:02:55.970174449 +0000 UTC m=+22.700680940" watchObservedRunningTime="2026-07-09 15:02:55.970701164 +0000 UTC m=+22.701207655" Jul 09 15:02:56.119261 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:56.119230 2567 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 09 15:02:56.186544 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:56.186507 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:02:56.287968 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:56.287936 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:02:56.462053 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:56.461969 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:02:56.715787 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:56.715613 2567 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T15:02:56.119253979Z","UUID":"bf193bd0-2be1-4a34-a368-f5b71af36f85","Handler":null,"Name":"","Endpoint":""} Jul 09 15:02:56.718077 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:56.718055 2567 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 09 15:02:56.718228 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:56.718098 2567 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 09 15:02:56.823058 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:56.823028 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:02:56.823253 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:56.823071 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:02:56.823253 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:56.823166 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-96fg2" podUID="d469078c-492e-4dbd-843f-7d1699fa1ce9" Jul 09 15:02:56.823352 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:56.823267 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8vsh" podUID="fce18369-7aaf-4358-b5f5-46b0218924fd" Jul 09 15:02:56.936587 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:56.936552 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9hc8" event={"ID":"51611d68-7db8-44af-81b2-87c97ab134cc","Type":"ContainerStarted","Data":"7f2e7bceca4c6cdaa171f2d1d5d4f2c87da75fbd664ed0c542cdbd74dea633e1"} Jul 09 15:02:56.938356 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:56.938334 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal" event={"ID":"c3942009bd02b28965ce62ce1b271d1a","Type":"ContainerStarted","Data":"051d691d64d8bdfa425521505c475923e9442b3afae137d2bf91c98416f8d51a"} Jul 09 15:02:56.956553 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:56.956501 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal" podStartSLOduration=22.956459345 podStartE2EDuration="22.956459345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:02:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:02:56.955500922 +0000 UTC m=+23.686007414" watchObservedRunningTime="2026-07-09 15:02:56.956459345 +0000 UTC m=+23.686965836" Jul 09 15:02:56.975771 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:56.975677 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9647m" Jul 09 15:02:57.823441 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:57.823125 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:02:57.823646 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:57.823509 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jcctg" podUID="37399ba7-ce43-46e3-b78e-73b29c2196be" Jul 09 15:02:57.944185 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:57.944157 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hml7m_84a10578-4913-456b-9fb7-e6440914b677/ovn-acl-logging/0.log" Jul 09 15:02:57.944610 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:57.944553 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" event={"ID":"84a10578-4913-456b-9fb7-e6440914b677","Type":"ContainerStarted","Data":"c8b3ce2d473d60dc93307c95a87e9a8d1af56d52d8c9759e8e532e3c44d35102"} Jul 09 15:02:58.823106 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:58.823057 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:02:58.823276 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:58.823057 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:02:58.823276 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:58.823188 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-96fg2" podUID="d469078c-492e-4dbd-843f-7d1699fa1ce9" Jul 09 15:02:58.823276 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:58.823244 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8vsh" podUID="fce18369-7aaf-4358-b5f5-46b0218924fd" Jul 09 15:02:59.246891 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:59.246782 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9647m" Jul 09 15:02:59.247576 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:59.247555 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9647m" Jul 09 15:02:59.827060 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:59.826983 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:02:59.827215 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:02:59.827105 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jcctg" podUID="37399ba7-ce43-46e3-b78e-73b29c2196be" Jul 09 15:02:59.950034 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:02:59.950000 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9647m" Jul 09 15:03:00.028448 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:00.028414 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:00.823084 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:00.822905 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:03:00.823504 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:00.822956 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:03:00.823504 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:00.823161 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-96fg2" podUID="d469078c-492e-4dbd-843f-7d1699fa1ce9" Jul 09 15:03:00.823504 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:00.823239 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8vsh" podUID="fce18369-7aaf-4358-b5f5-46b0218924fd" Jul 09 15:03:00.955127 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:00.955016 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hml7m_84a10578-4913-456b-9fb7-e6440914b677/ovn-acl-logging/0.log" Jul 09 15:03:00.955476 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:00.955446 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" event={"ID":"84a10578-4913-456b-9fb7-e6440914b677","Type":"ContainerStarted","Data":"7291644e57c0f619656068211e6f6db5dfb8c6f8c7dfdb466d14a2a0ebffaaab"} Jul 09 15:03:00.955782 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:00.955696 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:03:00.956035 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:00.956020 2567 scope.go:117] "RemoveContainer" containerID="9b267100f369523ac917f6ad533ae8c207d8b785d2a9c562611b4e4d14f96ddf" Jul 09 15:03:00.958016 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:00.957989 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9hc8" event={"ID":"51611d68-7db8-44af-81b2-87c97ab134cc","Type":"ContainerStarted","Data":"4a920c8ec0d1cf515f83a4f14e607bfee9a0611de1a27a966c0d6e349b8287cf"} Jul 09 15:03:00.974961 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:00.974935 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:03:01.078054 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:01.077984 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9hc8" podStartSLOduration=3.247774535 podStartE2EDuration="28.077960659s" totalImagesPullingTime="24.830186124s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:02:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:03:01.076929504 +0000 UTC m=+27.807435995" watchObservedRunningTime="2026-07-09 15:03:01.077960659 +0000 UTC m=+27.808467151" Jul 09 15:03:01.216178 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:01.216093 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:01.318829 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:01.318780 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:01.823153 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:01.823061 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:03:01.823527 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:01.823209 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jcctg" podUID="37399ba7-ce43-46e3-b78e-73b29c2196be" Jul 09 15:03:01.964217 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:01.964191 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hml7m_84a10578-4913-456b-9fb7-e6440914b677/ovn-acl-logging/0.log" Jul 09 15:03:01.964577 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:01.964554 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" event={"ID":"84a10578-4913-456b-9fb7-e6440914b677","Type":"ContainerStarted","Data":"18c6327061f2dbbee16e6263cbf3f4352cc291349863081b2415b2557838057b"} Jul 09 15:03:01.964783 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:01.964748 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:03:01.964941 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:01.964859 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:03:01.966524 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:01.966500 2567 generic.go:356] "Generic (PLEG): container finished" podID="6cf97264-90f5-4d6e-9e44-50e1431776a8" containerID="6c38d502143f4b452c5cb66a25009332fa673fef3633dd845bf142219d48cf2d" exitCode=0 Jul 09 15:03:01.966633 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:01.966580 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b9zn8" event={"ID":"6cf97264-90f5-4d6e-9e44-50e1431776a8","Type":"ContainerDied","Data":"6c38d502143f4b452c5cb66a25009332fa673fef3633dd845bf142219d48cf2d"} Jul 09 15:03:01.980396 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:01.980371 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:03:02.043348 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:02.043318 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:02.090388 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:02.090296 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" podStartSLOduration=9.513950284 podStartE2EDuration="29.090277307s" totalImagesPullingTime="19.576327023s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:02:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:03:02.025705058 +0000 UTC m=+28.756211550" watchObservedRunningTime="2026-07-09 15:03:02.090277307 +0000 UTC m=+28.820783798" Jul 09 15:03:02.215328 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:02.215297 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:02.316493 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:02.316456 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:02.419219 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:02.419138 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:02.543771 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:02.543725 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:02.753345 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:02.753248 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:02.822710 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:02.822674 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:03:02.822886 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:02.822786 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-96fg2" podUID="d469078c-492e-4dbd-843f-7d1699fa1ce9" Jul 09 15:03:02.822886 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:02.822840 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:03:02.822985 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:02.822936 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8vsh" podUID="fce18369-7aaf-4358-b5f5-46b0218924fd" Jul 09 15:03:02.986647 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:02.986614 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-96fg2"] Jul 09 15:03:02.987004 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:02.986707 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:03:02.987004 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:02.986789 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-96fg2" podUID="d469078c-492e-4dbd-843f-7d1699fa1ce9" Jul 09 15:03:02.990760 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:02.990724 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-p8vsh"] Jul 09 15:03:02.990902 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:02.990856 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:03:02.990996 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:02.990970 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8vsh" podUID="fce18369-7aaf-4358-b5f5-46b0218924fd" Jul 09 15:03:02.991423 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:02.991401 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jcctg"] Jul 09 15:03:02.991516 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:02.991489 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:03:02.991580 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:02.991564 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jcctg" podUID="37399ba7-ce43-46e3-b78e-73b29c2196be" Jul 09 15:03:03.034346 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:03.034255 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:03.142185 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:03.142137 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:03.344573 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:03.344487 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:03.443780 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:03.443751 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:03.546297 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:03.546261 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:03.852361 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:03.852271 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:03.953512 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:03.953478 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:04.435695 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:04.435670 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-k9ppv_67924f4e-f69d-48b8-ba23-e4c110bb24e5/dns-node-resolver/0.log" Jul 09 15:03:04.703883 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:04.703790 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:04.822478 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:04.822450 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:03:04.822478 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:04.822482 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:03:04.822642 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:04.822484 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:03:04.822642 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:04.822570 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-96fg2" podUID="d469078c-492e-4dbd-843f-7d1699fa1ce9" Jul 09 15:03:04.822697 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:04.822628 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jcctg" podUID="37399ba7-ce43-46e3-b78e-73b29c2196be" Jul 09 15:03:04.822758 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:04.822732 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8vsh" podUID="fce18369-7aaf-4358-b5f5-46b0218924fd" Jul 09 15:03:04.859270 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:04.859239 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:04.975235 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:04.975151 2567 generic.go:356] "Generic (PLEG): container finished" podID="6cf97264-90f5-4d6e-9e44-50e1431776a8" containerID="0f4553e770ee1b88ad1e5d1024351f8172d777c20068b452458884eed61060f6" exitCode=0 Jul 09 15:03:04.975235 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:04.975210 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b9zn8" event={"ID":"6cf97264-90f5-4d6e-9e44-50e1431776a8","Type":"ContainerDied","Data":"0f4553e770ee1b88ad1e5d1024351f8172d777c20068b452458884eed61060f6"} Jul 09 15:03:05.133445 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:05.133413 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:05.211760 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:05.211722 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:05.637354 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:05.637331 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-w9rr7_ffeceed9-e851-4ef7-8ad8-603243ee5157/node-ca/0.log" Jul 09 15:03:05.979716 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:05.979628 2567 generic.go:356] "Generic (PLEG): container finished" podID="6cf97264-90f5-4d6e-9e44-50e1431776a8" containerID="75c10df074eae67cb49cea0fc84511a8ae203c82d2f9caeb080c0f38270e8324" exitCode=0 Jul 09 15:03:05.979716 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:05.979683 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b9zn8" event={"ID":"6cf97264-90f5-4d6e-9e44-50e1431776a8","Type":"ContainerDied","Data":"75c10df074eae67cb49cea0fc84511a8ae203c82d2f9caeb080c0f38270e8324"} Jul 09 15:03:05.997390 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:05.997357 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:06.422121 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:06.421759 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:06.503803 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:06.503773 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fce18369-7aaf-4358-b5f5-46b0218924fd-original-pull-secret\") pod \"global-pull-secret-syncer-p8vsh\" (UID: \"fce18369-7aaf-4358-b5f5-46b0218924fd\") " pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:03:06.503983 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:06.503837 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37399ba7-ce43-46e3-b78e-73b29c2196be-metrics-certs\") pod \"network-metrics-daemon-jcctg\" (UID: \"37399ba7-ce43-46e3-b78e-73b29c2196be\") " pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:03:06.503983 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:06.503965 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 15:03:06.503983 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:06.503974 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 15:03:06.504136 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:06.504036 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37399ba7-ce43-46e3-b78e-73b29c2196be-metrics-certs podName:37399ba7-ce43-46e3-b78e-73b29c2196be nodeName:}" failed. No retries permitted until 2026-07-09 15:03:38.504014904 +0000 UTC m=+65.234521389 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37399ba7-ce43-46e3-b78e-73b29c2196be-metrics-certs") pod "network-metrics-daemon-jcctg" (UID: "37399ba7-ce43-46e3-b78e-73b29c2196be") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 15:03:06.504136 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:06.504056 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fce18369-7aaf-4358-b5f5-46b0218924fd-original-pull-secret podName:fce18369-7aaf-4358-b5f5-46b0218924fd nodeName:}" failed. No retries permitted until 2026-07-09 15:03:38.504046129 +0000 UTC m=+65.234552604 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fce18369-7aaf-4358-b5f5-46b0218924fd-original-pull-secret") pod "global-pull-secret-syncer-p8vsh" (UID: "fce18369-7aaf-4358-b5f5-46b0218924fd") : object "kube-system"/"original-pull-secret" not registered Jul 09 15:03:06.555619 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:06.555540 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:06.605042 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:06.605010 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4c4r\" (UniqueName: \"kubernetes.io/projected/d469078c-492e-4dbd-843f-7d1699fa1ce9-kube-api-access-r4c4r\") pod \"network-check-target-96fg2\" (UID: \"d469078c-492e-4dbd-843f-7d1699fa1ce9\") " pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:03:06.605240 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:06.605199 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 15:03:06.605240 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:06.605220 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 15:03:06.605240 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:06.605231 2567 projected.go:196] Error preparing data for projected volume kube-api-access-r4c4r for pod openshift-network-diagnostics/network-check-target-96fg2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 15:03:06.605386 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:06.605291 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d469078c-492e-4dbd-843f-7d1699fa1ce9-kube-api-access-r4c4r podName:d469078c-492e-4dbd-843f-7d1699fa1ce9 nodeName:}" failed. No retries permitted until 2026-07-09 15:03:38.605272528 +0000 UTC m=+65.335779012 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-r4c4r" (UniqueName: "kubernetes.io/projected/d469078c-492e-4dbd-843f-7d1699fa1ce9-kube-api-access-r4c4r") pod "network-check-target-96fg2" (UID: "d469078c-492e-4dbd-843f-7d1699fa1ce9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 15:03:06.664618 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:06.664532 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:06.775370 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:06.775270 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:06.822190 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:06.822161 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:03:06.822190 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:06.822175 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:03:06.822401 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:06.822280 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:03:06.822401 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:06.822279 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-96fg2" podUID="d469078c-492e-4dbd-843f-7d1699fa1ce9" Jul 09 15:03:06.822401 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:06.822381 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jcctg" podUID="37399ba7-ce43-46e3-b78e-73b29c2196be" Jul 09 15:03:06.822565 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:06.822478 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8vsh" podUID="fce18369-7aaf-4358-b5f5-46b0218924fd" Jul 09 15:03:06.878410 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:06.878378 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:06.977340 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:06.977303 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:07.139097 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:07.138995 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:07.926695 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:07.926660 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:08.822994 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:08.822956 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:03:08.823167 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:08.822999 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:03:08.823167 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:08.823054 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8vsh" podUID="fce18369-7aaf-4358-b5f5-46b0218924fd" Jul 09 15:03:08.823167 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:08.822999 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:03:08.823167 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:08.823088 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-96fg2" podUID="d469078c-492e-4dbd-843f-7d1699fa1ce9" Jul 09 15:03:08.823167 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:08.823160 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jcctg" podUID="37399ba7-ce43-46e3-b78e-73b29c2196be" Jul 09 15:03:09.894060 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:09.894017 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:09.987237 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:09.987194 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:10.127099 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:10.127059 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:10.381253 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:10.381161 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:10.499303 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:10.499256 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:10.578865 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:10.578801 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:10.729147 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:10.729023 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:10.822453 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:10.822419 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:03:10.822662 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:10.822464 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:03:10.822662 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:10.822466 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:03:10.822662 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:10.822556 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jcctg" podUID="37399ba7-ce43-46e3-b78e-73b29c2196be" Jul 09 15:03:10.822844 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:10.822672 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-96fg2" podUID="d469078c-492e-4dbd-843f-7d1699fa1ce9" Jul 09 15:03:10.822844 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:10.822752 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8vsh" podUID="fce18369-7aaf-4358-b5f5-46b0218924fd" Jul 09 15:03:10.924892 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:10.924852 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:11.023390 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:11.023299 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:11.327421 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:11.327385 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:11.720898 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:11.720784 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:11.820903 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:11.820862 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:11.995559 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:11.995479 2567 generic.go:356] "Generic (PLEG): container finished" podID="6cf97264-90f5-4d6e-9e44-50e1431776a8" containerID="bc3942ea400575086b416a8e95d939e274429b8a3eaee0b11ba8878c05785fc9" exitCode=0 Jul 09 15:03:11.995559 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:11.995535 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b9zn8" event={"ID":"6cf97264-90f5-4d6e-9e44-50e1431776a8","Type":"ContainerDied","Data":"bc3942ea400575086b416a8e95d939e274429b8a3eaee0b11ba8878c05785fc9"} Jul 09 15:03:12.823221 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:12.823136 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:03:12.823372 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:12.823234 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:03:12.823372 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:12.823326 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jcctg" podUID="37399ba7-ce43-46e3-b78e-73b29c2196be" Jul 09 15:03:12.823372 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:12.823231 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-96fg2" podUID="d469078c-492e-4dbd-843f-7d1699fa1ce9" Jul 09 15:03:12.823372 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:12.823229 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:03:12.823511 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:12.823412 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8vsh" podUID="fce18369-7aaf-4358-b5f5-46b0218924fd" Jul 09 15:03:13.001375 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:13.001338 2567 generic.go:356] "Generic (PLEG): container finished" podID="6cf97264-90f5-4d6e-9e44-50e1431776a8" containerID="68ccc0c6193139acd36d1c757cad1ee2e1d4eeeae3db7faf0f947721b9eda42c" exitCode=0 Jul 09 15:03:13.001756 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:13.001384 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b9zn8" event={"ID":"6cf97264-90f5-4d6e-9e44-50e1431776a8","Type":"ContainerDied","Data":"68ccc0c6193139acd36d1c757cad1ee2e1d4eeeae3db7faf0f947721b9eda42c"} Jul 09 15:03:13.003683 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:13.003658 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:13.104082 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:13.104039 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:13.251832 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:13.251786 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:13.352833 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:13.352721 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:13.918729 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:13.918692 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:14.006975 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:14.006942 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b9zn8" event={"ID":"6cf97264-90f5-4d6e-9e44-50e1431776a8","Type":"ContainerStarted","Data":"cb9668bc22e696a4824a62d336d3b68012ede5c20b00fe659781a11e14e9860a"} Jul 09 15:03:14.032069 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:14.032014 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-b9zn8" podStartSLOduration=9.029447804 podStartE2EDuration="41.03199797s" totalImagesPullingTime="32.002550166s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:02:33 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:03:14.031386666 +0000 UTC m=+40.761893171" watchObservedRunningTime="2026-07-09 15:03:14.03199797 +0000 UTC m=+40.762504460" Jul 09 15:03:14.360688 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:14.360650 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:14.739490 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:14.739459 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:14.822731 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:14.822695 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:03:14.822731 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:14.822724 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:03:14.822917 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:14.822718 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:03:14.822917 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:14.822838 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-96fg2" podUID="d469078c-492e-4dbd-843f-7d1699fa1ce9" Jul 09 15:03:14.822917 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:14.822900 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jcctg" podUID="37399ba7-ce43-46e3-b78e-73b29c2196be" Jul 09 15:03:14.823027 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:14.822955 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8vsh" podUID="fce18369-7aaf-4358-b5f5-46b0218924fd" Jul 09 15:03:14.956545 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:14.956480 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:15.140502 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:15.140409 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:15.322550 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:15.322507 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:16.118149 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:16.118114 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:16.524281 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:16.524245 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:16.732605 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:16.732575 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:16.822433 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:16.822341 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:03:16.822569 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:16.822449 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:03:16.822569 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:16.822457 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jcctg" podUID="37399ba7-ce43-46e3-b78e-73b29c2196be" Jul 09 15:03:16.822569 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:16.822478 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:03:16.822569 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:16.822548 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-96fg2" podUID="d469078c-492e-4dbd-843f-7d1699fa1ce9" Jul 09 15:03:16.822738 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:16.822618 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8vsh" podUID="fce18369-7aaf-4358-b5f5-46b0218924fd" Jul 09 15:03:18.405733 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:18.405700 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:18.485189 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:18.485158 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:18.823006 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:18.822969 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:03:18.823209 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:18.823027 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:03:18.823209 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:18.823134 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:03:18.823209 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:18.823149 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8vsh" podUID="fce18369-7aaf-4358-b5f5-46b0218924fd" Jul 09 15:03:18.823378 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:18.823247 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jcctg" podUID="37399ba7-ce43-46e3-b78e-73b29c2196be" Jul 09 15:03:18.823378 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:18.823320 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-96fg2" podUID="d469078c-492e-4dbd-843f-7d1699fa1ce9" Jul 09 15:03:20.822873 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:20.822836 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:03:20.823348 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:20.822882 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:03:20.823348 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:20.822837 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:03:20.823348 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:20.822965 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8vsh" podUID="fce18369-7aaf-4358-b5f5-46b0218924fd" Jul 09 15:03:20.823348 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:20.823037 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jcctg" podUID="37399ba7-ce43-46e3-b78e-73b29c2196be" Jul 09 15:03:20.823348 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:20.823093 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-96fg2" podUID="d469078c-492e-4dbd-843f-7d1699fa1ce9" Jul 09 15:03:22.822958 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:22.822921 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:03:22.822958 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:22.822936 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:03:22.823371 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:22.822924 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:03:22.823371 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:22.823022 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-96fg2" podUID="d469078c-492e-4dbd-843f-7d1699fa1ce9" Jul 09 15:03:22.823371 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:22.823113 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jcctg" podUID="37399ba7-ce43-46e3-b78e-73b29c2196be" Jul 09 15:03:22.823371 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:22.823163 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8vsh" podUID="fce18369-7aaf-4358-b5f5-46b0218924fd" Jul 09 15:03:24.602075 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:24.602037 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:24.785106 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:24.785050 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:24.822152 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:24.822115 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:03:24.822357 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:24.822162 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:03:24.822357 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:24.822216 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8vsh" podUID="fce18369-7aaf-4358-b5f5-46b0218924fd" Jul 09 15:03:24.822357 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:24.822162 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:03:24.822357 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:24.822275 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jcctg" podUID="37399ba7-ce43-46e3-b78e-73b29c2196be" Jul 09 15:03:24.822357 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:24.822315 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-96fg2" podUID="d469078c-492e-4dbd-843f-7d1699fa1ce9" Jul 09 15:03:24.895753 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:24.895638 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:25.094666 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.094639 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeReady" Jul 09 15:03:25.094844 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.094755 2567 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 09 15:03:25.189784 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.189704 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-sg7zr"] Jul 09 15:03:25.202320 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.202299 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-j4w6r"] Jul 09 15:03:25.202447 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.202433 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sg7zr" Jul 09 15:03:25.206024 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.205668 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lcjqv\"" Jul 09 15:03:25.206024 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.206008 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 09 15:03:25.209045 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.209014 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 09 15:03:25.212529 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.212507 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-sg7zr"] Jul 09 15:03:25.212634 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.212549 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-j4w6r"] Jul 09 15:03:25.212634 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.212587 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-j4w6r" Jul 09 15:03:25.216948 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.216928 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 09 15:03:25.217040 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.216986 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-lqzhv\"" Jul 09 15:03:25.217299 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.217283 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 09 15:03:25.219335 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.219313 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 09 15:03:25.276397 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.276365 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5g8vz"] Jul 09 15:03:25.302654 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.302624 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5g8vz" Jul 09 15:03:25.304738 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.304708 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5g8vz"] Jul 09 15:03:25.305963 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.305939 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nt8wt\"" Jul 09 15:03:25.306103 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.305993 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 09 15:03:25.306103 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.306066 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 09 15:03:25.306103 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.306096 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 09 15:03:25.306248 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.306234 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 09 15:03:25.339108 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.339075 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bc92e63d-1f89-4cf1-b209-a53b3f7687ff-cert\") pod \"ingress-canary-j4w6r\" (UID: \"bc92e63d-1f89-4cf1-b209-a53b3f7687ff\") " pod="openshift-ingress-canary/ingress-canary-j4w6r" Jul 09 15:03:25.339108 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.339108 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t56t\" (UniqueName: \"kubernetes.io/projected/ae72603b-84db-40aa-8fef-f79bfb5459ae-kube-api-access-9t56t\") pod \"dns-default-sg7zr\" (UID: \"ae72603b-84db-40aa-8fef-f79bfb5459ae\") " pod="openshift-dns/dns-default-sg7zr" Jul 09 15:03:25.339343 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.339143 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae72603b-84db-40aa-8fef-f79bfb5459ae-tmp-dir\") pod \"dns-default-sg7zr\" (UID: \"ae72603b-84db-40aa-8fef-f79bfb5459ae\") " pod="openshift-dns/dns-default-sg7zr" Jul 09 15:03:25.339343 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.339174 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ae72603b-84db-40aa-8fef-f79bfb5459ae-config-volume\") pod \"dns-default-sg7zr\" (UID: \"ae72603b-84db-40aa-8fef-f79bfb5459ae\") " pod="openshift-dns/dns-default-sg7zr" Jul 09 15:03:25.339343 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.339203 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lh8z4\" (UniqueName: \"kubernetes.io/projected/bc92e63d-1f89-4cf1-b209-a53b3f7687ff-kube-api-access-lh8z4\") pod \"ingress-canary-j4w6r\" (UID: \"bc92e63d-1f89-4cf1-b209-a53b3f7687ff\") " pod="openshift-ingress-canary/ingress-canary-j4w6r" Jul 09 15:03:25.339343 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.339229 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae72603b-84db-40aa-8fef-f79bfb5459ae-metrics-tls\") pod \"dns-default-sg7zr\" (UID: \"ae72603b-84db-40aa-8fef-f79bfb5459ae\") " pod="openshift-dns/dns-default-sg7zr" Jul 09 15:03:25.439576 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.439525 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae72603b-84db-40aa-8fef-f79bfb5459ae-metrics-tls\") pod \"dns-default-sg7zr\" (UID: \"ae72603b-84db-40aa-8fef-f79bfb5459ae\") " pod="openshift-dns/dns-default-sg7zr" Jul 09 15:03:25.439576 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.439578 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bc92e63d-1f89-4cf1-b209-a53b3f7687ff-cert\") pod \"ingress-canary-j4w6r\" (UID: \"bc92e63d-1f89-4cf1-b209-a53b3f7687ff\") " pod="openshift-ingress-canary/ingress-canary-j4w6r" Jul 09 15:03:25.439849 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.439603 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9t56t\" (UniqueName: \"kubernetes.io/projected/ae72603b-84db-40aa-8fef-f79bfb5459ae-kube-api-access-9t56t\") pod \"dns-default-sg7zr\" (UID: \"ae72603b-84db-40aa-8fef-f79bfb5459ae\") " pod="openshift-dns/dns-default-sg7zr" Jul 09 15:03:25.439849 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.439626 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b3c37120-9403-437d-bb77-51a322b71fd6-crio-socket\") pod \"insights-runtime-extractor-5g8vz\" (UID: \"b3c37120-9403-437d-bb77-51a322b71fd6\") " pod="openshift-insights/insights-runtime-extractor-5g8vz" Jul 09 15:03:25.439849 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.439645 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b3c37120-9403-437d-bb77-51a322b71fd6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5g8vz\" (UID: \"b3c37120-9403-437d-bb77-51a322b71fd6\") " pod="openshift-insights/insights-runtime-extractor-5g8vz" Jul 09 15:03:25.439849 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.439662 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48xl2\" (UniqueName: \"kubernetes.io/projected/b3c37120-9403-437d-bb77-51a322b71fd6-kube-api-access-48xl2\") pod \"insights-runtime-extractor-5g8vz\" (UID: \"b3c37120-9403-437d-bb77-51a322b71fd6\") " pod="openshift-insights/insights-runtime-extractor-5g8vz" Jul 09 15:03:25.440066 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.439889 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b3c37120-9403-437d-bb77-51a322b71fd6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5g8vz\" (UID: \"b3c37120-9403-437d-bb77-51a322b71fd6\") " pod="openshift-insights/insights-runtime-extractor-5g8vz" Jul 09 15:03:25.440066 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.439933 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae72603b-84db-40aa-8fef-f79bfb5459ae-tmp-dir\") pod \"dns-default-sg7zr\" (UID: \"ae72603b-84db-40aa-8fef-f79bfb5459ae\") " pod="openshift-dns/dns-default-sg7zr" Jul 09 15:03:25.440066 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.439963 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b3c37120-9403-437d-bb77-51a322b71fd6-data-volume\") pod \"insights-runtime-extractor-5g8vz\" (UID: \"b3c37120-9403-437d-bb77-51a322b71fd6\") " pod="openshift-insights/insights-runtime-extractor-5g8vz" Jul 09 15:03:25.440066 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.439999 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ae72603b-84db-40aa-8fef-f79bfb5459ae-config-volume\") pod \"dns-default-sg7zr\" (UID: \"ae72603b-84db-40aa-8fef-f79bfb5459ae\") " pod="openshift-dns/dns-default-sg7zr" Jul 09 15:03:25.440066 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.440038 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lh8z4\" (UniqueName: \"kubernetes.io/projected/bc92e63d-1f89-4cf1-b209-a53b3f7687ff-kube-api-access-lh8z4\") pod \"ingress-canary-j4w6r\" (UID: \"bc92e63d-1f89-4cf1-b209-a53b3f7687ff\") " pod="openshift-ingress-canary/ingress-canary-j4w6r" Jul 09 15:03:25.440441 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.440247 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae72603b-84db-40aa-8fef-f79bfb5459ae-tmp-dir\") pod \"dns-default-sg7zr\" (UID: \"ae72603b-84db-40aa-8fef-f79bfb5459ae\") " pod="openshift-dns/dns-default-sg7zr" Jul 09 15:03:25.440503 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.440486 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ae72603b-84db-40aa-8fef-f79bfb5459ae-config-volume\") pod \"dns-default-sg7zr\" (UID: \"ae72603b-84db-40aa-8fef-f79bfb5459ae\") " pod="openshift-dns/dns-default-sg7zr" Jul 09 15:03:25.443650 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.443630 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae72603b-84db-40aa-8fef-f79bfb5459ae-metrics-tls\") pod \"dns-default-sg7zr\" (UID: \"ae72603b-84db-40aa-8fef-f79bfb5459ae\") " pod="openshift-dns/dns-default-sg7zr" Jul 09 15:03:25.443712 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.443667 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bc92e63d-1f89-4cf1-b209-a53b3f7687ff-cert\") pod \"ingress-canary-j4w6r\" (UID: \"bc92e63d-1f89-4cf1-b209-a53b3f7687ff\") " pod="openshift-ingress-canary/ingress-canary-j4w6r" Jul 09 15:03:25.451779 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.451751 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lh8z4\" (UniqueName: \"kubernetes.io/projected/bc92e63d-1f89-4cf1-b209-a53b3f7687ff-kube-api-access-lh8z4\") pod \"ingress-canary-j4w6r\" (UID: \"bc92e63d-1f89-4cf1-b209-a53b3f7687ff\") " pod="openshift-ingress-canary/ingress-canary-j4w6r" Jul 09 15:03:25.451993 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.451977 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t56t\" (UniqueName: \"kubernetes.io/projected/ae72603b-84db-40aa-8fef-f79bfb5459ae-kube-api-access-9t56t\") pod \"dns-default-sg7zr\" (UID: \"ae72603b-84db-40aa-8fef-f79bfb5459ae\") " pod="openshift-dns/dns-default-sg7zr" Jul 09 15:03:25.515949 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.515910 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sg7zr" Jul 09 15:03:25.522827 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.522794 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-j4w6r" Jul 09 15:03:25.540921 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.540896 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b3c37120-9403-437d-bb77-51a322b71fd6-crio-socket\") pod \"insights-runtime-extractor-5g8vz\" (UID: \"b3c37120-9403-437d-bb77-51a322b71fd6\") " pod="openshift-insights/insights-runtime-extractor-5g8vz" Jul 09 15:03:25.541059 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.540927 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b3c37120-9403-437d-bb77-51a322b71fd6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5g8vz\" (UID: \"b3c37120-9403-437d-bb77-51a322b71fd6\") " pod="openshift-insights/insights-runtime-extractor-5g8vz" Jul 09 15:03:25.541059 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.540947 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48xl2\" (UniqueName: \"kubernetes.io/projected/b3c37120-9403-437d-bb77-51a322b71fd6-kube-api-access-48xl2\") pod \"insights-runtime-extractor-5g8vz\" (UID: \"b3c37120-9403-437d-bb77-51a322b71fd6\") " pod="openshift-insights/insights-runtime-extractor-5g8vz" Jul 09 15:03:25.541059 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.541019 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b3c37120-9403-437d-bb77-51a322b71fd6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5g8vz\" (UID: \"b3c37120-9403-437d-bb77-51a322b71fd6\") " pod="openshift-insights/insights-runtime-extractor-5g8vz" Jul 09 15:03:25.541059 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.541050 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b3c37120-9403-437d-bb77-51a322b71fd6-data-volume\") pod \"insights-runtime-extractor-5g8vz\" (UID: \"b3c37120-9403-437d-bb77-51a322b71fd6\") " pod="openshift-insights/insights-runtime-extractor-5g8vz" Jul 09 15:03:25.541228 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.541128 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b3c37120-9403-437d-bb77-51a322b71fd6-crio-socket\") pod \"insights-runtime-extractor-5g8vz\" (UID: \"b3c37120-9403-437d-bb77-51a322b71fd6\") " pod="openshift-insights/insights-runtime-extractor-5g8vz" Jul 09 15:03:25.541425 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.541403 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b3c37120-9403-437d-bb77-51a322b71fd6-data-volume\") pod \"insights-runtime-extractor-5g8vz\" (UID: \"b3c37120-9403-437d-bb77-51a322b71fd6\") " pod="openshift-insights/insights-runtime-extractor-5g8vz" Jul 09 15:03:25.541651 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.541634 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b3c37120-9403-437d-bb77-51a322b71fd6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5g8vz\" (UID: \"b3c37120-9403-437d-bb77-51a322b71fd6\") " pod="openshift-insights/insights-runtime-extractor-5g8vz" Jul 09 15:03:25.543310 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.543285 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b3c37120-9403-437d-bb77-51a322b71fd6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5g8vz\" (UID: \"b3c37120-9403-437d-bb77-51a322b71fd6\") " pod="openshift-insights/insights-runtime-extractor-5g8vz" Jul 09 15:03:25.549295 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.549267 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-9llhr"] Jul 09 15:03:25.551756 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.551737 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-9llhr" Jul 09 15:03:25.553037 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.553017 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48xl2\" (UniqueName: \"kubernetes.io/projected/b3c37120-9403-437d-bb77-51a322b71fd6-kube-api-access-48xl2\") pod \"insights-runtime-extractor-5g8vz\" (UID: \"b3c37120-9403-437d-bb77-51a322b71fd6\") " pod="openshift-insights/insights-runtime-extractor-5g8vz" Jul 09 15:03:25.554252 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.554230 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-dngtd\"" Jul 09 15:03:25.554399 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.554258 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 09 15:03:25.554399 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.554288 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 09 15:03:25.564681 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.564655 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-9llhr"] Jul 09 15:03:25.612494 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.612422 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5g8vz" Jul 09 15:03:25.677230 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.677200 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-sg7zr"] Jul 09 15:03:25.696978 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.696951 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-j4w6r"] Jul 09 15:03:25.700512 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:03:25.700486 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc92e63d_1f89_4cf1_b209_a53b3f7687ff.slice/crio-989d0db6a4f18fe8b7fec8062b983c8ad150e5c78b0d5bffe42b68166336f393 WatchSource:0}: Error finding container 989d0db6a4f18fe8b7fec8062b983c8ad150e5c78b0d5bffe42b68166336f393: Status 404 returned error can't find the container with id 989d0db6a4f18fe8b7fec8062b983c8ad150e5c78b0d5bffe42b68166336f393 Jul 09 15:03:25.743015 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.742983 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhmtf\" (UniqueName: \"kubernetes.io/projected/0eb4fb70-53ef-4394-9acc-63bf16dba693-kube-api-access-xhmtf\") pod \"downloads-68c9cbb497-9llhr\" (UID: \"0eb4fb70-53ef-4394-9acc-63bf16dba693\") " pod="openshift-console/downloads-68c9cbb497-9llhr" Jul 09 15:03:25.750348 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.750318 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5g8vz"] Jul 09 15:03:25.753163 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:03:25.753138 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3c37120_9403_437d_bb77_51a322b71fd6.slice/crio-79cb95de70e50845754be9884fc14b69110566afac3e37f6409bbc94ac7cc280 WatchSource:0}: Error finding container 79cb95de70e50845754be9884fc14b69110566afac3e37f6409bbc94ac7cc280: Status 404 returned error can't find the container with id 79cb95de70e50845754be9884fc14b69110566afac3e37f6409bbc94ac7cc280 Jul 09 15:03:25.843968 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.843937 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhmtf\" (UniqueName: \"kubernetes.io/projected/0eb4fb70-53ef-4394-9acc-63bf16dba693-kube-api-access-xhmtf\") pod \"downloads-68c9cbb497-9llhr\" (UID: \"0eb4fb70-53ef-4394-9acc-63bf16dba693\") " pod="openshift-console/downloads-68c9cbb497-9llhr" Jul 09 15:03:25.854626 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.854594 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhmtf\" (UniqueName: \"kubernetes.io/projected/0eb4fb70-53ef-4394-9acc-63bf16dba693-kube-api-access-xhmtf\") pod \"downloads-68c9cbb497-9llhr\" (UID: \"0eb4fb70-53ef-4394-9acc-63bf16dba693\") " pod="openshift-console/downloads-68c9cbb497-9llhr" Jul 09 15:03:25.862429 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.862409 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-9llhr" Jul 09 15:03:25.993801 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:25.993769 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-9llhr"] Jul 09 15:03:25.997511 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:03:25.997485 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0eb4fb70_53ef_4394_9acc_63bf16dba693.slice/crio-1ccee61d77260de253843cb9c84af14b0666c5baf16a8da7009f231461d5b0f9 WatchSource:0}: Error finding container 1ccee61d77260de253843cb9c84af14b0666c5baf16a8da7009f231461d5b0f9: Status 404 returned error can't find the container with id 1ccee61d77260de253843cb9c84af14b0666c5baf16a8da7009f231461d5b0f9 Jul 09 15:03:26.035308 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:26.035271 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sg7zr" event={"ID":"ae72603b-84db-40aa-8fef-f79bfb5459ae","Type":"ContainerStarted","Data":"4919ca82b8b8ae8246d8aad4751803725c4d848804ef66cdb47b355429f3571a"} Jul 09 15:03:26.036558 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:26.036527 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-9llhr" event={"ID":"0eb4fb70-53ef-4394-9acc-63bf16dba693","Type":"ContainerStarted","Data":"1ccee61d77260de253843cb9c84af14b0666c5baf16a8da7009f231461d5b0f9"} Jul 09 15:03:26.037708 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:26.037665 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5g8vz" event={"ID":"b3c37120-9403-437d-bb77-51a322b71fd6","Type":"ContainerStarted","Data":"79cb95de70e50845754be9884fc14b69110566afac3e37f6409bbc94ac7cc280"} Jul 09 15:03:26.038869 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:26.038847 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-j4w6r" event={"ID":"bc92e63d-1f89-4cf1-b209-a53b3f7687ff","Type":"ContainerStarted","Data":"989d0db6a4f18fe8b7fec8062b983c8ad150e5c78b0d5bffe42b68166336f393"} Jul 09 15:03:26.578684 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:26.578588 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:26.649767 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:26.649696 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:26.822718 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:26.822682 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:03:26.825307 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:26.823153 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:03:26.825307 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:26.823576 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:03:26.829875 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:26.827858 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 15:03:26.829875 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:26.827930 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 15:03:26.829875 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:26.828073 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7htgt\"" Jul 09 15:03:26.829875 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:26.828129 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z6fx5\"" Jul 09 15:03:26.829875 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:26.828145 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 15:03:26.829875 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:26.828225 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 15:03:27.442969 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:27.442925 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:27.633967 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:27.633860 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:28.289735 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:28.289649 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:28.658176 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:28.658053 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:28.733962 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:28.733908 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:30.578360 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:30.578275 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:30.645557 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:30.645514 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:31.307678 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:31.307643 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:31.399621 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:31.399591 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:31.601036 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:31.601005 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:31.608950 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:31.608922 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-4cpbv"] Jul 09 15:03:31.630459 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:31.630401 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-4cpbv"] Jul 09 15:03:31.630639 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:31.630524 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-4cpbv" Jul 09 15:03:31.633452 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:31.633393 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 09 15:03:31.633452 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:31.633426 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-bn686\"" Jul 09 15:03:31.693198 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:31.693155 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b988013e-d1a6-416c-885c-b200d6686a8d-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-4cpbv\" (UID: \"b988013e-d1a6-416c-885c-b200d6686a8d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-4cpbv" Jul 09 15:03:31.794032 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:31.793999 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b988013e-d1a6-416c-885c-b200d6686a8d-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-4cpbv\" (UID: \"b988013e-d1a6-416c-885c-b200d6686a8d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-4cpbv" Jul 09 15:03:31.797078 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:31.797053 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b988013e-d1a6-416c-885c-b200d6686a8d-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-4cpbv\" (UID: \"b988013e-d1a6-416c-885c-b200d6686a8d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-4cpbv" Jul 09 15:03:31.928289 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:31.928237 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:31.942124 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:31.942102 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-4cpbv" Jul 09 15:03:32.071943 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:32.063732 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5g8vz" event={"ID":"b3c37120-9403-437d-bb77-51a322b71fd6","Type":"ContainerStarted","Data":"d14461e0f677a8e318fb1860be17616ab1914bcd9917ef43e935fae38cba82cf"} Jul 09 15:03:32.071943 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:32.065734 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-j4w6r" event={"ID":"bc92e63d-1f89-4cf1-b209-a53b3f7687ff","Type":"ContainerStarted","Data":"e6a8d2b374d3e552d38b60f4a018ab036250e3214feb33e61deb62773fe98387"} Jul 09 15:03:32.071943 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:32.068833 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sg7zr" event={"ID":"ae72603b-84db-40aa-8fef-f79bfb5459ae","Type":"ContainerStarted","Data":"536cb3124767019313d73e164582f2f8be231b5420f02600729af441463d2450"} Jul 09 15:03:32.071943 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:32.068884 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sg7zr" event={"ID":"ae72603b-84db-40aa-8fef-f79bfb5459ae","Type":"ContainerStarted","Data":"9ecfc8927c7e12c0f1d55747e71cdb36065bd95c5d9f934bada722584b8191d9"} Jul 09 15:03:32.071943 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:32.069425 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-sg7zr" Jul 09 15:03:32.088177 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:32.088118 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-j4w6r" podStartSLOduration=1.348576162 podStartE2EDuration="7.088094627s" totalImagesPullingTime="5.739518465s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:03:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:03:32.087467407 +0000 UTC m=+58.817973909" watchObservedRunningTime="2026-07-09 15:03:32.088094627 +0000 UTC m=+58.818601113" Jul 09 15:03:32.113227 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:32.113175 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-sg7zr" podStartSLOduration=1.364195021 podStartE2EDuration="7.113156759s" totalImagesPullingTime="5.748961738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:03:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:03:32.111999451 +0000 UTC m=+58.842505970" watchObservedRunningTime="2026-07-09 15:03:32.113156759 +0000 UTC m=+58.843663249" Jul 09 15:03:32.125442 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:32.125411 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-4cpbv"] Jul 09 15:03:32.129318 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:03:32.129283 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb988013e_d1a6_416c_885c_b200d6686a8d.slice/crio-ab66a12727eda554aba3975cfbbcd892d7fbdf99f27c1c340e2dcd717e775577 WatchSource:0}: Error finding container ab66a12727eda554aba3975cfbbcd892d7fbdf99f27c1c340e2dcd717e775577: Status 404 returned error can't find the container with id ab66a12727eda554aba3975cfbbcd892d7fbdf99f27c1c340e2dcd717e775577 Jul 09 15:03:32.314510 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:32.314474 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:32.404604 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:32.404520 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:33.075220 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:33.075181 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5g8vz" event={"ID":"b3c37120-9403-437d-bb77-51a322b71fd6","Type":"ContainerStarted","Data":"d09b1a9e9eec4165a2bc5bf8e5e53b84707a581087d9661131d6ede5d10a1390"} Jul 09 15:03:33.076560 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:33.076529 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-4cpbv" event={"ID":"b988013e-d1a6-416c-885c-b200d6686a8d","Type":"ContainerStarted","Data":"ab66a12727eda554aba3975cfbbcd892d7fbdf99f27c1c340e2dcd717e775577"} Jul 09 15:03:33.984554 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:33.984526 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hml7m" Jul 09 15:03:34.082998 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:34.082918 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-4cpbv" event={"ID":"b988013e-d1a6-416c-885c-b200d6686a8d","Type":"ContainerStarted","Data":"2ee50bef8d88605cb6d2b4f84b185f85e3e07a2a73105a47e04cb34db64fb181"} Jul 09 15:03:34.083444 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:34.083176 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-4cpbv" Jul 09 15:03:34.089047 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:34.089018 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-4cpbv" Jul 09 15:03:34.102925 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:34.102856 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-4cpbv" podStartSLOduration=1.891240044 podStartE2EDuration="3.102836028s" totalImagesPullingTime="1.211595984s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:03:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:03:34.102314637 +0000 UTC m=+60.832821131" watchObservedRunningTime="2026-07-09 15:03:34.102836028 +0000 UTC m=+60.833342520" Jul 09 15:03:34.141355 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:34.141319 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:34.243104 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:34.242905 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:34.353077 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:34.352973 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:34.425368 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:34.425328 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:34.609587 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:34.609506 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:34.655844 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:34.655800 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-wc22m"] Jul 09 15:03:34.658858 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:34.658833 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-wc22m" Jul 09 15:03:34.661308 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:34.661283 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 09 15:03:34.661437 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:34.661309 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 09 15:03:34.661437 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:34.661428 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 09 15:03:34.661554 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:34.661428 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-vs8ch\"" Jul 09 15:03:34.661609 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:34.661566 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 09 15:03:34.661740 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:34.661721 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 09 15:03:34.675621 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:34.675544 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-wc22m"] Jul 09 15:03:34.692727 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:34.692687 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:34.718000 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:34.717912 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qg8m\" (UniqueName: \"kubernetes.io/projected/3f1edfb6-5fd6-4635-adf3-e2662f222fb0-kube-api-access-7qg8m\") pod \"prometheus-operator-64785b6675-wc22m\" (UID: \"3f1edfb6-5fd6-4635-adf3-e2662f222fb0\") " pod="openshift-monitoring/prometheus-operator-64785b6675-wc22m" Jul 09 15:03:34.718000 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:34.717981 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3f1edfb6-5fd6-4635-adf3-e2662f222fb0-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-wc22m\" (UID: \"3f1edfb6-5fd6-4635-adf3-e2662f222fb0\") " pod="openshift-monitoring/prometheus-operator-64785b6675-wc22m" Jul 09 15:03:34.718254 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:34.718041 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f1edfb6-5fd6-4635-adf3-e2662f222fb0-metrics-client-ca\") pod \"prometheus-operator-64785b6675-wc22m\" (UID: \"3f1edfb6-5fd6-4635-adf3-e2662f222fb0\") " pod="openshift-monitoring/prometheus-operator-64785b6675-wc22m" Jul 09 15:03:34.718254 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:34.718137 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3f1edfb6-5fd6-4635-adf3-e2662f222fb0-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-wc22m\" (UID: \"3f1edfb6-5fd6-4635-adf3-e2662f222fb0\") " pod="openshift-monitoring/prometheus-operator-64785b6675-wc22m" Jul 09 15:03:34.818685 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:34.818649 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f1edfb6-5fd6-4635-adf3-e2662f222fb0-metrics-client-ca\") pod \"prometheus-operator-64785b6675-wc22m\" (UID: \"3f1edfb6-5fd6-4635-adf3-e2662f222fb0\") " pod="openshift-monitoring/prometheus-operator-64785b6675-wc22m" Jul 09 15:03:34.818880 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:34.818701 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3f1edfb6-5fd6-4635-adf3-e2662f222fb0-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-wc22m\" (UID: \"3f1edfb6-5fd6-4635-adf3-e2662f222fb0\") " pod="openshift-monitoring/prometheus-operator-64785b6675-wc22m" Jul 09 15:03:34.818880 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:34.818729 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qg8m\" (UniqueName: \"kubernetes.io/projected/3f1edfb6-5fd6-4635-adf3-e2662f222fb0-kube-api-access-7qg8m\") pod \"prometheus-operator-64785b6675-wc22m\" (UID: \"3f1edfb6-5fd6-4635-adf3-e2662f222fb0\") " pod="openshift-monitoring/prometheus-operator-64785b6675-wc22m" Jul 09 15:03:34.818880 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:34.818757 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3f1edfb6-5fd6-4635-adf3-e2662f222fb0-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-wc22m\" (UID: \"3f1edfb6-5fd6-4635-adf3-e2662f222fb0\") " pod="openshift-monitoring/prometheus-operator-64785b6675-wc22m" Jul 09 15:03:34.819609 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:34.819579 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f1edfb6-5fd6-4635-adf3-e2662f222fb0-metrics-client-ca\") pod \"prometheus-operator-64785b6675-wc22m\" (UID: \"3f1edfb6-5fd6-4635-adf3-e2662f222fb0\") " pod="openshift-monitoring/prometheus-operator-64785b6675-wc22m" Jul 09 15:03:34.821449 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:34.821422 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3f1edfb6-5fd6-4635-adf3-e2662f222fb0-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-wc22m\" (UID: \"3f1edfb6-5fd6-4635-adf3-e2662f222fb0\") " pod="openshift-monitoring/prometheus-operator-64785b6675-wc22m" Jul 09 15:03:34.821568 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:34.821534 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3f1edfb6-5fd6-4635-adf3-e2662f222fb0-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-wc22m\" (UID: \"3f1edfb6-5fd6-4635-adf3-e2662f222fb0\") " pod="openshift-monitoring/prometheus-operator-64785b6675-wc22m" Jul 09 15:03:34.827933 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:34.827906 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qg8m\" (UniqueName: \"kubernetes.io/projected/3f1edfb6-5fd6-4635-adf3-e2662f222fb0-kube-api-access-7qg8m\") pod \"prometheus-operator-64785b6675-wc22m\" (UID: \"3f1edfb6-5fd6-4635-adf3-e2662f222fb0\") " pod="openshift-monitoring/prometheus-operator-64785b6675-wc22m" Jul 09 15:03:34.838245 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:34.838214 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:34.969787 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:34.969746 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-wc22m" Jul 09 15:03:35.652967 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:35.652931 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:35.814603 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:35.814564 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-wc22m"] Jul 09 15:03:35.818011 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:03:35.817983 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f1edfb6_5fd6_4635_adf3_e2662f222fb0.slice/crio-9f755d5f74a8c62802b5364d62649fb75c495cdf8bd023d777e2019c052db252 WatchSource:0}: Error finding container 9f755d5f74a8c62802b5364d62649fb75c495cdf8bd023d777e2019c052db252: Status 404 returned error can't find the container with id 9f755d5f74a8c62802b5364d62649fb75c495cdf8bd023d777e2019c052db252 Jul 09 15:03:35.870863 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:35.870793 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:35.940733 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:35.940639 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:36.092246 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:36.092204 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5g8vz" event={"ID":"b3c37120-9403-437d-bb77-51a322b71fd6","Type":"ContainerStarted","Data":"e798871efe667a1ffdad5e5804228b9d97cfdc868583f5d0c0883cd1caa0cb76"} Jul 09 15:03:36.093917 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:36.093884 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-wc22m" event={"ID":"3f1edfb6-5fd6-4635-adf3-e2662f222fb0","Type":"ContainerStarted","Data":"9f755d5f74a8c62802b5364d62649fb75c495cdf8bd023d777e2019c052db252"} Jul 09 15:03:36.112360 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:36.112302 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-5g8vz" podStartSLOduration=1.339398874 podStartE2EDuration="11.11228257s" totalImagesPullingTime="9.772883696s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:03:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:03:36.111771802 +0000 UTC m=+62.842278305" watchObservedRunningTime="2026-07-09 15:03:36.11228257 +0000 UTC m=+62.842789060" Jul 09 15:03:37.268950 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:37.268901 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b6cbb4b6d-qfx28"] Jul 09 15:03:37.272087 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:37.272064 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b6cbb4b6d-qfx28" Jul 09 15:03:37.279036 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:37.278812 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 09 15:03:37.280307 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:37.279874 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 09 15:03:37.280307 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:37.279914 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 09 15:03:37.280307 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:37.280195 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 09 15:03:37.280467 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:37.280305 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 09 15:03:37.280467 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:37.280387 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-rl5vz\"" Jul 09 15:03:37.286683 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:37.286654 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b6cbb4b6d-qfx28"] Jul 09 15:03:37.339974 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:37.339934 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7aa645ea-62ea-493b-b863-9ea493f80c7b-console-oauth-config\") pod \"console-5b6cbb4b6d-qfx28\" (UID: \"7aa645ea-62ea-493b-b863-9ea493f80c7b\") " pod="openshift-console/console-5b6cbb4b6d-qfx28" Jul 09 15:03:37.340153 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:37.339981 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7aa645ea-62ea-493b-b863-9ea493f80c7b-oauth-serving-cert\") pod \"console-5b6cbb4b6d-qfx28\" (UID: \"7aa645ea-62ea-493b-b863-9ea493f80c7b\") " pod="openshift-console/console-5b6cbb4b6d-qfx28" Jul 09 15:03:37.340153 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:37.340031 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdmqf\" (UniqueName: \"kubernetes.io/projected/7aa645ea-62ea-493b-b863-9ea493f80c7b-kube-api-access-wdmqf\") pod \"console-5b6cbb4b6d-qfx28\" (UID: \"7aa645ea-62ea-493b-b863-9ea493f80c7b\") " pod="openshift-console/console-5b6cbb4b6d-qfx28" Jul 09 15:03:37.340153 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:37.340069 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7aa645ea-62ea-493b-b863-9ea493f80c7b-console-config\") pod \"console-5b6cbb4b6d-qfx28\" (UID: \"7aa645ea-62ea-493b-b863-9ea493f80c7b\") " pod="openshift-console/console-5b6cbb4b6d-qfx28" Jul 09 15:03:37.340295 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:37.340201 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7aa645ea-62ea-493b-b863-9ea493f80c7b-console-serving-cert\") pod \"console-5b6cbb4b6d-qfx28\" (UID: \"7aa645ea-62ea-493b-b863-9ea493f80c7b\") " pod="openshift-console/console-5b6cbb4b6d-qfx28" Jul 09 15:03:37.340295 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:37.340236 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7aa645ea-62ea-493b-b863-9ea493f80c7b-service-ca\") pod \"console-5b6cbb4b6d-qfx28\" (UID: \"7aa645ea-62ea-493b-b863-9ea493f80c7b\") " pod="openshift-console/console-5b6cbb4b6d-qfx28" Jul 09 15:03:37.441653 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:37.441617 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7aa645ea-62ea-493b-b863-9ea493f80c7b-console-serving-cert\") pod \"console-5b6cbb4b6d-qfx28\" (UID: \"7aa645ea-62ea-493b-b863-9ea493f80c7b\") " pod="openshift-console/console-5b6cbb4b6d-qfx28" Jul 09 15:03:37.441653 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:37.441655 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7aa645ea-62ea-493b-b863-9ea493f80c7b-service-ca\") pod \"console-5b6cbb4b6d-qfx28\" (UID: \"7aa645ea-62ea-493b-b863-9ea493f80c7b\") " pod="openshift-console/console-5b6cbb4b6d-qfx28" Jul 09 15:03:37.441909 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:37.441859 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7aa645ea-62ea-493b-b863-9ea493f80c7b-console-oauth-config\") pod \"console-5b6cbb4b6d-qfx28\" (UID: \"7aa645ea-62ea-493b-b863-9ea493f80c7b\") " pod="openshift-console/console-5b6cbb4b6d-qfx28" Jul 09 15:03:37.441970 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:37.441919 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7aa645ea-62ea-493b-b863-9ea493f80c7b-oauth-serving-cert\") pod \"console-5b6cbb4b6d-qfx28\" (UID: \"7aa645ea-62ea-493b-b863-9ea493f80c7b\") " pod="openshift-console/console-5b6cbb4b6d-qfx28" Jul 09 15:03:37.441970 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:37.441956 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdmqf\" (UniqueName: \"kubernetes.io/projected/7aa645ea-62ea-493b-b863-9ea493f80c7b-kube-api-access-wdmqf\") pod \"console-5b6cbb4b6d-qfx28\" (UID: \"7aa645ea-62ea-493b-b863-9ea493f80c7b\") " pod="openshift-console/console-5b6cbb4b6d-qfx28" Jul 09 15:03:37.442057 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:37.442002 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7aa645ea-62ea-493b-b863-9ea493f80c7b-console-config\") pod \"console-5b6cbb4b6d-qfx28\" (UID: \"7aa645ea-62ea-493b-b863-9ea493f80c7b\") " pod="openshift-console/console-5b6cbb4b6d-qfx28" Jul 09 15:03:37.442469 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:37.442434 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7aa645ea-62ea-493b-b863-9ea493f80c7b-service-ca\") pod \"console-5b6cbb4b6d-qfx28\" (UID: \"7aa645ea-62ea-493b-b863-9ea493f80c7b\") " pod="openshift-console/console-5b6cbb4b6d-qfx28" Jul 09 15:03:37.442641 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:37.442610 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7aa645ea-62ea-493b-b863-9ea493f80c7b-console-config\") pod \"console-5b6cbb4b6d-qfx28\" (UID: \"7aa645ea-62ea-493b-b863-9ea493f80c7b\") " pod="openshift-console/console-5b6cbb4b6d-qfx28" Jul 09 15:03:37.442768 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:37.442694 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7aa645ea-62ea-493b-b863-9ea493f80c7b-oauth-serving-cert\") pod \"console-5b6cbb4b6d-qfx28\" (UID: \"7aa645ea-62ea-493b-b863-9ea493f80c7b\") " pod="openshift-console/console-5b6cbb4b6d-qfx28" Jul 09 15:03:37.444324 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:37.444302 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7aa645ea-62ea-493b-b863-9ea493f80c7b-console-serving-cert\") pod \"console-5b6cbb4b6d-qfx28\" (UID: \"7aa645ea-62ea-493b-b863-9ea493f80c7b\") " pod="openshift-console/console-5b6cbb4b6d-qfx28" Jul 09 15:03:37.444541 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:37.444523 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7aa645ea-62ea-493b-b863-9ea493f80c7b-console-oauth-config\") pod \"console-5b6cbb4b6d-qfx28\" (UID: \"7aa645ea-62ea-493b-b863-9ea493f80c7b\") " pod="openshift-console/console-5b6cbb4b6d-qfx28" Jul 09 15:03:37.453090 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:37.453068 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdmqf\" (UniqueName: \"kubernetes.io/projected/7aa645ea-62ea-493b-b863-9ea493f80c7b-kube-api-access-wdmqf\") pod \"console-5b6cbb4b6d-qfx28\" (UID: \"7aa645ea-62ea-493b-b863-9ea493f80c7b\") " pod="openshift-console/console-5b6cbb4b6d-qfx28" Jul 09 15:03:37.583263 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:37.583149 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b6cbb4b6d-qfx28" Jul 09 15:03:38.550750 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:38.550704 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fce18369-7aaf-4358-b5f5-46b0218924fd-original-pull-secret\") pod \"global-pull-secret-syncer-p8vsh\" (UID: \"fce18369-7aaf-4358-b5f5-46b0218924fd\") " pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:03:38.551218 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:38.550763 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37399ba7-ce43-46e3-b78e-73b29c2196be-metrics-certs\") pod \"network-metrics-daemon-jcctg\" (UID: \"37399ba7-ce43-46e3-b78e-73b29c2196be\") " pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:03:38.553934 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:38.553908 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 15:03:38.554263 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:38.554247 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 15:03:38.564429 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:38.564396 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37399ba7-ce43-46e3-b78e-73b29c2196be-metrics-certs\") pod \"network-metrics-daemon-jcctg\" (UID: \"37399ba7-ce43-46e3-b78e-73b29c2196be\") " pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:03:38.564568 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:38.564458 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fce18369-7aaf-4358-b5f5-46b0218924fd-original-pull-secret\") pod \"global-pull-secret-syncer-p8vsh\" (UID: \"fce18369-7aaf-4358-b5f5-46b0218924fd\") " pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:03:38.651754 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:38.651720 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4c4r\" (UniqueName: \"kubernetes.io/projected/d469078c-492e-4dbd-843f-7d1699fa1ce9-kube-api-access-r4c4r\") pod \"network-check-target-96fg2\" (UID: \"d469078c-492e-4dbd-843f-7d1699fa1ce9\") " pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:03:38.654655 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:38.654626 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 15:03:38.665632 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:38.665608 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 15:03:38.676225 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:38.676195 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4c4r\" (UniqueName: \"kubernetes.io/projected/d469078c-492e-4dbd-843f-7d1699fa1ce9-kube-api-access-r4c4r\") pod \"network-check-target-96fg2\" (UID: \"d469078c-492e-4dbd-843f-7d1699fa1ce9\") " pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:03:38.722356 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:38.722309 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:38.823213 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:38.823123 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:38.842053 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:38.842019 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8vsh" Jul 09 15:03:38.856444 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:38.856415 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z6fx5\"" Jul 09 15:03:38.863886 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:38.863862 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jcctg" Jul 09 15:03:38.864524 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:38.864505 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7htgt\"" Jul 09 15:03:38.871831 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:38.871799 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:03:38.922792 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:38.922736 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:43.085685 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:43.085651 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-sg7zr" Jul 09 15:03:43.450549 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:43.450457 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:43.546946 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:43.546913 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:43.778831 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:43.778774 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jcctg"] Jul 09 15:03:43.787797 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:43.787761 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-p8vsh"] Jul 09 15:03:43.942531 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:03:43.942481 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37399ba7_ce43_46e3_b78e_73b29c2196be.slice/crio-f67ff3c3073f371d4f812ba639d65d89e005a64301eea025335e2d8133e63175 WatchSource:0}: Error finding container f67ff3c3073f371d4f812ba639d65d89e005a64301eea025335e2d8133e63175: Status 404 returned error can't find the container with id f67ff3c3073f371d4f812ba639d65d89e005a64301eea025335e2d8133e63175 Jul 09 15:03:43.943312 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:03:43.943285 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfce18369_7aaf_4358_b5f5_46b0218924fd.slice/crio-e0d1a1d3e992822747cffb1d62ba2f962a44d22b7b42f4a10b54a642b55342d0 WatchSource:0}: Error finding container e0d1a1d3e992822747cffb1d62ba2f962a44d22b7b42f4a10b54a642b55342d0: Status 404 returned error can't find the container with id e0d1a1d3e992822747cffb1d62ba2f962a44d22b7b42f4a10b54a642b55342d0 Jul 09 15:03:44.010491 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:44.010458 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-96fg2"] Jul 09 15:03:44.015041 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:44.015013 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b6cbb4b6d-qfx28"] Jul 09 15:03:44.015335 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:03:44.015312 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd469078c_492e_4dbd_843f_7d1699fa1ce9.slice/crio-315fe90e83416567daf18936af82e55a1e24c9caf18c18b87adf3ffe0ec4fd12 WatchSource:0}: Error finding container 315fe90e83416567daf18936af82e55a1e24c9caf18c18b87adf3ffe0ec4fd12: Status 404 returned error can't find the container with id 315fe90e83416567daf18936af82e55a1e24c9caf18c18b87adf3ffe0ec4fd12 Jul 09 15:03:44.018168 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:03:44.018146 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7aa645ea_62ea_493b_b863_9ea493f80c7b.slice/crio-cfc3b16f977ce37d8c5e1d986adbfc99594578efd6a20d0c12901f9da033d5fb WatchSource:0}: Error finding container cfc3b16f977ce37d8c5e1d986adbfc99594578efd6a20d0c12901f9da033d5fb: Status 404 returned error can't find the container with id cfc3b16f977ce37d8c5e1d986adbfc99594578efd6a20d0c12901f9da033d5fb Jul 09 15:03:44.122211 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:44.122174 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-9llhr" event={"ID":"0eb4fb70-53ef-4394-9acc-63bf16dba693","Type":"ContainerStarted","Data":"afd2cd5a7623b74969c51a6734099ca1d01600907add33ffcc77d740671b5c42"} Jul 09 15:03:44.123772 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:44.123733 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b6cbb4b6d-qfx28" event={"ID":"7aa645ea-62ea-493b-b863-9ea493f80c7b","Type":"ContainerStarted","Data":"cfc3b16f977ce37d8c5e1d986adbfc99594578efd6a20d0c12901f9da033d5fb"} Jul 09 15:03:44.125207 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:44.125174 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-p8vsh" event={"ID":"fce18369-7aaf-4358-b5f5-46b0218924fd","Type":"ContainerStarted","Data":"e0d1a1d3e992822747cffb1d62ba2f962a44d22b7b42f4a10b54a642b55342d0"} Jul 09 15:03:44.126687 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:44.126658 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-96fg2" event={"ID":"d469078c-492e-4dbd-843f-7d1699fa1ce9","Type":"ContainerStarted","Data":"315fe90e83416567daf18936af82e55a1e24c9caf18c18b87adf3ffe0ec4fd12"} Jul 09 15:03:44.128011 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:44.127982 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jcctg" event={"ID":"37399ba7-ce43-46e3-b78e-73b29c2196be","Type":"ContainerStarted","Data":"f67ff3c3073f371d4f812ba639d65d89e005a64301eea025335e2d8133e63175"} Jul 09 15:03:45.135592 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:45.135509 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-9llhr" Jul 09 15:03:45.150522 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:45.150484 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-9llhr" Jul 09 15:03:45.191400 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:45.190650 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-9llhr" podStartSLOduration=2.175792438 podStartE2EDuration="20.19062549s" totalImagesPullingTime="18.014833052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:03:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:03:45.160692741 +0000 UTC m=+71.891199233" watchObservedRunningTime="2026-07-09 15:03:45.19062549 +0000 UTC m=+71.921131982" Jul 09 15:03:46.145777 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:46.145697 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-wc22m" event={"ID":"3f1edfb6-5fd6-4635-adf3-e2662f222fb0","Type":"ContainerStarted","Data":"762a1fd30a9923790cc75f88bad1abfe525a37c5cfda9e4091c500c3eb9e0ade"} Jul 09 15:03:47.025679 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:47.025628 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:47.156013 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:47.155881 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-wc22m" event={"ID":"3f1edfb6-5fd6-4635-adf3-e2662f222fb0","Type":"ContainerStarted","Data":"c3cdb65f09ff519d5eec5a31137f3951dd23b081698221e604ec9f385abc39d6"} Jul 09 15:03:47.794126 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:47.794077 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:47.917242 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:47.917207 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:48.047047 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:48.046963 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:48.263489 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:48.263352 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-wc22m" podStartSLOduration=4.232051595 podStartE2EDuration="14.263326487s" totalImagesPullingTime="10.031274892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:03:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:03:47.197156516 +0000 UTC m=+73.927663008" watchObservedRunningTime="2026-07-09 15:03:48.263326487 +0000 UTC m=+74.993832979" Jul 09 15:03:48.264544 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:48.263626 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8796dfb78-gjmt4"] Jul 09 15:03:48.282413 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:48.282354 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:48.284796 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:48.284768 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8796dfb78-gjmt4"] Jul 09 15:03:48.285025 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:48.284954 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8796dfb78-gjmt4" Jul 09 15:03:48.298886 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:48.298767 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 09 15:03:48.342132 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:48.342078 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bdaffe4c-36aa-4fee-b356-87897e214f9f-console-oauth-config\") pod \"console-8796dfb78-gjmt4\" (UID: \"bdaffe4c-36aa-4fee-b356-87897e214f9f\") " pod="openshift-console/console-8796dfb78-gjmt4" Jul 09 15:03:48.342132 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:48.342126 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bdaffe4c-36aa-4fee-b356-87897e214f9f-trusted-ca-bundle\") pod \"console-8796dfb78-gjmt4\" (UID: \"bdaffe4c-36aa-4fee-b356-87897e214f9f\") " pod="openshift-console/console-8796dfb78-gjmt4" Jul 09 15:03:48.342407 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:48.342151 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bdaffe4c-36aa-4fee-b356-87897e214f9f-console-serving-cert\") pod \"console-8796dfb78-gjmt4\" (UID: \"bdaffe4c-36aa-4fee-b356-87897e214f9f\") " pod="openshift-console/console-8796dfb78-gjmt4" Jul 09 15:03:48.342407 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:48.342181 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bdaffe4c-36aa-4fee-b356-87897e214f9f-service-ca\") pod \"console-8796dfb78-gjmt4\" (UID: \"bdaffe4c-36aa-4fee-b356-87897e214f9f\") " pod="openshift-console/console-8796dfb78-gjmt4" Jul 09 15:03:48.342407 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:48.342206 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bdaffe4c-36aa-4fee-b356-87897e214f9f-oauth-serving-cert\") pod \"console-8796dfb78-gjmt4\" (UID: \"bdaffe4c-36aa-4fee-b356-87897e214f9f\") " pod="openshift-console/console-8796dfb78-gjmt4" Jul 09 15:03:48.342407 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:48.342273 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8n5vd\" (UniqueName: \"kubernetes.io/projected/bdaffe4c-36aa-4fee-b356-87897e214f9f-kube-api-access-8n5vd\") pod \"console-8796dfb78-gjmt4\" (UID: \"bdaffe4c-36aa-4fee-b356-87897e214f9f\") " pod="openshift-console/console-8796dfb78-gjmt4" Jul 09 15:03:48.342407 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:48.342323 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bdaffe4c-36aa-4fee-b356-87897e214f9f-console-config\") pod \"console-8796dfb78-gjmt4\" (UID: \"bdaffe4c-36aa-4fee-b356-87897e214f9f\") " pod="openshift-console/console-8796dfb78-gjmt4" Jul 09 15:03:48.394348 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:48.394244 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:48.443046 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:48.443001 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8n5vd\" (UniqueName: \"kubernetes.io/projected/bdaffe4c-36aa-4fee-b356-87897e214f9f-kube-api-access-8n5vd\") pod \"console-8796dfb78-gjmt4\" (UID: \"bdaffe4c-36aa-4fee-b356-87897e214f9f\") " pod="openshift-console/console-8796dfb78-gjmt4" Jul 09 15:03:48.443046 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:48.443072 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bdaffe4c-36aa-4fee-b356-87897e214f9f-console-config\") pod \"console-8796dfb78-gjmt4\" (UID: \"bdaffe4c-36aa-4fee-b356-87897e214f9f\") " pod="openshift-console/console-8796dfb78-gjmt4" Jul 09 15:03:48.443046 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:48.443107 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bdaffe4c-36aa-4fee-b356-87897e214f9f-console-oauth-config\") pod \"console-8796dfb78-gjmt4\" (UID: \"bdaffe4c-36aa-4fee-b356-87897e214f9f\") " pod="openshift-console/console-8796dfb78-gjmt4" Jul 09 15:03:48.443046 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:48.443133 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bdaffe4c-36aa-4fee-b356-87897e214f9f-trusted-ca-bundle\") pod \"console-8796dfb78-gjmt4\" (UID: \"bdaffe4c-36aa-4fee-b356-87897e214f9f\") " pod="openshift-console/console-8796dfb78-gjmt4" Jul 09 15:03:48.443046 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:48.443164 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bdaffe4c-36aa-4fee-b356-87897e214f9f-console-serving-cert\") pod \"console-8796dfb78-gjmt4\" (UID: \"bdaffe4c-36aa-4fee-b356-87897e214f9f\") " pod="openshift-console/console-8796dfb78-gjmt4" Jul 09 15:03:48.443046 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:48.443190 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bdaffe4c-36aa-4fee-b356-87897e214f9f-service-ca\") pod \"console-8796dfb78-gjmt4\" (UID: \"bdaffe4c-36aa-4fee-b356-87897e214f9f\") " pod="openshift-console/console-8796dfb78-gjmt4" Jul 09 15:03:48.444042 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:48.443211 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bdaffe4c-36aa-4fee-b356-87897e214f9f-oauth-serving-cert\") pod \"console-8796dfb78-gjmt4\" (UID: \"bdaffe4c-36aa-4fee-b356-87897e214f9f\") " pod="openshift-console/console-8796dfb78-gjmt4" Jul 09 15:03:48.444042 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:48.444024 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bdaffe4c-36aa-4fee-b356-87897e214f9f-oauth-serving-cert\") pod \"console-8796dfb78-gjmt4\" (UID: \"bdaffe4c-36aa-4fee-b356-87897e214f9f\") " pod="openshift-console/console-8796dfb78-gjmt4" Jul 09 15:03:48.444957 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:48.444931 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bdaffe4c-36aa-4fee-b356-87897e214f9f-console-config\") pod \"console-8796dfb78-gjmt4\" (UID: \"bdaffe4c-36aa-4fee-b356-87897e214f9f\") " pod="openshift-console/console-8796dfb78-gjmt4" Jul 09 15:03:48.446293 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:48.446271 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bdaffe4c-36aa-4fee-b356-87897e214f9f-service-ca\") pod \"console-8796dfb78-gjmt4\" (UID: \"bdaffe4c-36aa-4fee-b356-87897e214f9f\") " pod="openshift-console/console-8796dfb78-gjmt4" Jul 09 15:03:48.446540 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:48.446475 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bdaffe4c-36aa-4fee-b356-87897e214f9f-trusted-ca-bundle\") pod \"console-8796dfb78-gjmt4\" (UID: \"bdaffe4c-36aa-4fee-b356-87897e214f9f\") " pod="openshift-console/console-8796dfb78-gjmt4" Jul 09 15:03:48.449300 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:48.449270 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bdaffe4c-36aa-4fee-b356-87897e214f9f-console-oauth-config\") pod \"console-8796dfb78-gjmt4\" (UID: \"bdaffe4c-36aa-4fee-b356-87897e214f9f\") " pod="openshift-console/console-8796dfb78-gjmt4" Jul 09 15:03:48.454887 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:48.454792 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bdaffe4c-36aa-4fee-b356-87897e214f9f-console-serving-cert\") pod \"console-8796dfb78-gjmt4\" (UID: \"bdaffe4c-36aa-4fee-b356-87897e214f9f\") " pod="openshift-console/console-8796dfb78-gjmt4" Jul 09 15:03:48.458452 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:48.458425 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8n5vd\" (UniqueName: \"kubernetes.io/projected/bdaffe4c-36aa-4fee-b356-87897e214f9f-kube-api-access-8n5vd\") pod \"console-8796dfb78-gjmt4\" (UID: \"bdaffe4c-36aa-4fee-b356-87897e214f9f\") " pod="openshift-console/console-8796dfb78-gjmt4" Jul 09 15:03:48.489204 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:48.489162 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:48.605450 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:48.605367 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8796dfb78-gjmt4" Jul 09 15:03:49.152634 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.152596 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-kc99w"] Jul 09 15:03:49.195895 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.194676 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kc99w" Jul 09 15:03:49.195895 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.194737 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6wps2"] Jul 09 15:03:49.199879 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.199845 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-4zl49\"" Jul 09 15:03:49.200001 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.199956 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 09 15:03:49.201003 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.200784 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 09 15:03:49.211623 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.211603 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-kc99w"] Jul 09 15:03:49.211776 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.211652 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-nzgcs"] Jul 09 15:03:49.211776 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.211737 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6wps2" Jul 09 15:03:49.215347 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.215320 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 09 15:03:49.215713 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.215560 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 09 15:03:49.215713 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.215609 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-nz6qf\"" Jul 09 15:03:49.216069 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.215988 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 09 15:03:49.235325 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.235284 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-nzgcs"] Jul 09 15:03:49.235325 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.235434 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-nzgcs" Jul 09 15:03:49.239109 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.239082 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 09 15:03:49.239391 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.239372 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 09 15:03:49.239587 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.239562 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 09 15:03:49.239875 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.239786 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-kc5j6\"" Jul 09 15:03:49.252350 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.252303 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af9403ab-61af-488f-b826-9cf5be0bcd3a-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-kc99w\" (UID: \"af9403ab-61af-488f-b826-9cf5be0bcd3a\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kc99w" Jul 09 15:03:49.252488 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.252389 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4m8rn\" (UniqueName: \"kubernetes.io/projected/af9403ab-61af-488f-b826-9cf5be0bcd3a-kube-api-access-4m8rn\") pod \"openshift-state-metrics-84bd68b896-kc99w\" (UID: \"af9403ab-61af-488f-b826-9cf5be0bcd3a\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kc99w" Jul 09 15:03:49.252580 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.252547 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/af9403ab-61af-488f-b826-9cf5be0bcd3a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-kc99w\" (UID: \"af9403ab-61af-488f-b826-9cf5be0bcd3a\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kc99w" Jul 09 15:03:49.252646 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.252604 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af9403ab-61af-488f-b826-9cf5be0bcd3a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-kc99w\" (UID: \"af9403ab-61af-488f-b826-9cf5be0bcd3a\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kc99w" Jul 09 15:03:49.353348 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.353307 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4m8rn\" (UniqueName: \"kubernetes.io/projected/af9403ab-61af-488f-b826-9cf5be0bcd3a-kube-api-access-4m8rn\") pod \"openshift-state-metrics-84bd68b896-kc99w\" (UID: \"af9403ab-61af-488f-b826-9cf5be0bcd3a\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kc99w" Jul 09 15:03:49.353348 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.353358 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8c194403-1e4f-4469-8b40-ead103f9edef-node-exporter-textfile\") pod \"node-exporter-6wps2\" (UID: \"8c194403-1e4f-4469-8b40-ead103f9edef\") " pod="openshift-monitoring/node-exporter-6wps2" Jul 09 15:03:49.353935 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.353409 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/38241def-a91a-478f-b3cc-cf4f2eea029d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-nzgcs\" (UID: \"38241def-a91a-478f-b3cc-cf4f2eea029d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nzgcs" Jul 09 15:03:49.353935 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.353581 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8c194403-1e4f-4469-8b40-ead103f9edef-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6wps2\" (UID: \"8c194403-1e4f-4469-8b40-ead103f9edef\") " pod="openshift-monitoring/node-exporter-6wps2" Jul 09 15:03:49.353935 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.353612 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38241def-a91a-478f-b3cc-cf4f2eea029d-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-nzgcs\" (UID: \"38241def-a91a-478f-b3cc-cf4f2eea029d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nzgcs" Jul 09 15:03:49.353935 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.353632 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/38241def-a91a-478f-b3cc-cf4f2eea029d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-nzgcs\" (UID: \"38241def-a91a-478f-b3cc-cf4f2eea029d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nzgcs" Jul 09 15:03:49.353935 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.353655 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8c194403-1e4f-4469-8b40-ead103f9edef-sys\") pod \"node-exporter-6wps2\" (UID: \"8c194403-1e4f-4469-8b40-ead103f9edef\") " pod="openshift-monitoring/node-exporter-6wps2" Jul 09 15:03:49.353935 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.353681 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/38241def-a91a-478f-b3cc-cf4f2eea029d-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-nzgcs\" (UID: \"38241def-a91a-478f-b3cc-cf4f2eea029d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nzgcs" Jul 09 15:03:49.353935 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.353807 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/af9403ab-61af-488f-b826-9cf5be0bcd3a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-kc99w\" (UID: \"af9403ab-61af-488f-b826-9cf5be0bcd3a\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kc99w" Jul 09 15:03:49.353935 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.353864 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8c194403-1e4f-4469-8b40-ead103f9edef-node-exporter-wtmp\") pod \"node-exporter-6wps2\" (UID: \"8c194403-1e4f-4469-8b40-ead103f9edef\") " pod="openshift-monitoring/node-exporter-6wps2" Jul 09 15:03:49.353935 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.353886 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/38241def-a91a-478f-b3cc-cf4f2eea029d-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-nzgcs\" (UID: \"38241def-a91a-478f-b3cc-cf4f2eea029d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nzgcs" Jul 09 15:03:49.353935 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.353918 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af9403ab-61af-488f-b826-9cf5be0bcd3a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-kc99w\" (UID: \"af9403ab-61af-488f-b826-9cf5be0bcd3a\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kc99w" Jul 09 15:03:49.353935 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.353945 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8c194403-1e4f-4469-8b40-ead103f9edef-node-exporter-tls\") pod \"node-exporter-6wps2\" (UID: \"8c194403-1e4f-4469-8b40-ead103f9edef\") " pod="openshift-monitoring/node-exporter-6wps2" Jul 09 15:03:49.354483 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.353967 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kcvv\" (UniqueName: \"kubernetes.io/projected/38241def-a91a-478f-b3cc-cf4f2eea029d-kube-api-access-2kcvv\") pod \"kube-state-metrics-785778ff74-nzgcs\" (UID: \"38241def-a91a-478f-b3cc-cf4f2eea029d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nzgcs" Jul 09 15:03:49.354483 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:49.353975 2567 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 09 15:03:49.354483 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.353999 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c194403-1e4f-4469-8b40-ead103f9edef-metrics-client-ca\") pod \"node-exporter-6wps2\" (UID: \"8c194403-1e4f-4469-8b40-ead103f9edef\") " pod="openshift-monitoring/node-exporter-6wps2" Jul 09 15:03:49.354483 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.354024 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8c194403-1e4f-4469-8b40-ead103f9edef-node-exporter-accelerators-collector-config\") pod \"node-exporter-6wps2\" (UID: \"8c194403-1e4f-4469-8b40-ead103f9edef\") " pod="openshift-monitoring/node-exporter-6wps2" Jul 09 15:03:49.354483 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:49.354064 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af9403ab-61af-488f-b826-9cf5be0bcd3a-openshift-state-metrics-tls podName:af9403ab-61af-488f-b826-9cf5be0bcd3a nodeName:}" failed. No retries permitted until 2026-07-09 15:03:49.854040267 +0000 UTC m=+76.584546740 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/af9403ab-61af-488f-b826-9cf5be0bcd3a-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-kc99w" (UID: "af9403ab-61af-488f-b826-9cf5be0bcd3a") : secret "openshift-state-metrics-tls" not found Jul 09 15:03:49.354483 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.354111 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48zwm\" (UniqueName: \"kubernetes.io/projected/8c194403-1e4f-4469-8b40-ead103f9edef-kube-api-access-48zwm\") pod \"node-exporter-6wps2\" (UID: \"8c194403-1e4f-4469-8b40-ead103f9edef\") " pod="openshift-monitoring/node-exporter-6wps2" Jul 09 15:03:49.354483 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.354158 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8c194403-1e4f-4469-8b40-ead103f9edef-root\") pod \"node-exporter-6wps2\" (UID: \"8c194403-1e4f-4469-8b40-ead103f9edef\") " pod="openshift-monitoring/node-exporter-6wps2" Jul 09 15:03:49.354483 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.354199 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af9403ab-61af-488f-b826-9cf5be0bcd3a-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-kc99w\" (UID: \"af9403ab-61af-488f-b826-9cf5be0bcd3a\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kc99w" Jul 09 15:03:49.354924 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.354906 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af9403ab-61af-488f-b826-9cf5be0bcd3a-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-kc99w\" (UID: \"af9403ab-61af-488f-b826-9cf5be0bcd3a\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kc99w" Jul 09 15:03:49.356693 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.356659 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af9403ab-61af-488f-b826-9cf5be0bcd3a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-kc99w\" (UID: \"af9403ab-61af-488f-b826-9cf5be0bcd3a\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kc99w" Jul 09 15:03:49.361009 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.360980 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:49.363351 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.363325 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4m8rn\" (UniqueName: \"kubernetes.io/projected/af9403ab-61af-488f-b826-9cf5be0bcd3a-kube-api-access-4m8rn\") pod \"openshift-state-metrics-84bd68b896-kc99w\" (UID: \"af9403ab-61af-488f-b826-9cf5be0bcd3a\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kc99w" Jul 09 15:03:49.455424 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.455387 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c194403-1e4f-4469-8b40-ead103f9edef-metrics-client-ca\") pod \"node-exporter-6wps2\" (UID: \"8c194403-1e4f-4469-8b40-ead103f9edef\") " pod="openshift-monitoring/node-exporter-6wps2" Jul 09 15:03:49.455424 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.455426 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8c194403-1e4f-4469-8b40-ead103f9edef-node-exporter-accelerators-collector-config\") pod \"node-exporter-6wps2\" (UID: \"8c194403-1e4f-4469-8b40-ead103f9edef\") " pod="openshift-monitoring/node-exporter-6wps2" Jul 09 15:03:49.455697 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.455459 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48zwm\" (UniqueName: \"kubernetes.io/projected/8c194403-1e4f-4469-8b40-ead103f9edef-kube-api-access-48zwm\") pod \"node-exporter-6wps2\" (UID: \"8c194403-1e4f-4469-8b40-ead103f9edef\") " pod="openshift-monitoring/node-exporter-6wps2" Jul 09 15:03:49.455697 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.455490 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8c194403-1e4f-4469-8b40-ead103f9edef-root\") pod \"node-exporter-6wps2\" (UID: \"8c194403-1e4f-4469-8b40-ead103f9edef\") " pod="openshift-monitoring/node-exporter-6wps2" Jul 09 15:03:49.455697 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.455532 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8c194403-1e4f-4469-8b40-ead103f9edef-node-exporter-textfile\") pod \"node-exporter-6wps2\" (UID: \"8c194403-1e4f-4469-8b40-ead103f9edef\") " pod="openshift-monitoring/node-exporter-6wps2" Jul 09 15:03:49.455697 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.455560 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/38241def-a91a-478f-b3cc-cf4f2eea029d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-nzgcs\" (UID: \"38241def-a91a-478f-b3cc-cf4f2eea029d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nzgcs" Jul 09 15:03:49.455697 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.455603 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8c194403-1e4f-4469-8b40-ead103f9edef-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6wps2\" (UID: \"8c194403-1e4f-4469-8b40-ead103f9edef\") " pod="openshift-monitoring/node-exporter-6wps2" Jul 09 15:03:49.455697 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.455625 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38241def-a91a-478f-b3cc-cf4f2eea029d-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-nzgcs\" (UID: \"38241def-a91a-478f-b3cc-cf4f2eea029d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nzgcs" Jul 09 15:03:49.455697 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.455653 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/38241def-a91a-478f-b3cc-cf4f2eea029d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-nzgcs\" (UID: \"38241def-a91a-478f-b3cc-cf4f2eea029d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nzgcs" Jul 09 15:03:49.455697 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.455688 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8c194403-1e4f-4469-8b40-ead103f9edef-sys\") pod \"node-exporter-6wps2\" (UID: \"8c194403-1e4f-4469-8b40-ead103f9edef\") " pod="openshift-monitoring/node-exporter-6wps2" Jul 09 15:03:49.456706 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.455713 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/38241def-a91a-478f-b3cc-cf4f2eea029d-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-nzgcs\" (UID: \"38241def-a91a-478f-b3cc-cf4f2eea029d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nzgcs" Jul 09 15:03:49.456706 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.455770 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8c194403-1e4f-4469-8b40-ead103f9edef-node-exporter-wtmp\") pod \"node-exporter-6wps2\" (UID: \"8c194403-1e4f-4469-8b40-ead103f9edef\") " pod="openshift-monitoring/node-exporter-6wps2" Jul 09 15:03:49.456706 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.455794 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/38241def-a91a-478f-b3cc-cf4f2eea029d-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-nzgcs\" (UID: \"38241def-a91a-478f-b3cc-cf4f2eea029d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nzgcs" Jul 09 15:03:49.456706 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.455840 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8c194403-1e4f-4469-8b40-ead103f9edef-node-exporter-tls\") pod \"node-exporter-6wps2\" (UID: \"8c194403-1e4f-4469-8b40-ead103f9edef\") " pod="openshift-monitoring/node-exporter-6wps2" Jul 09 15:03:49.456706 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.455872 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kcvv\" (UniqueName: \"kubernetes.io/projected/38241def-a91a-478f-b3cc-cf4f2eea029d-kube-api-access-2kcvv\") pod \"kube-state-metrics-785778ff74-nzgcs\" (UID: \"38241def-a91a-478f-b3cc-cf4f2eea029d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nzgcs" Jul 09 15:03:49.456706 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.456038 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8c194403-1e4f-4469-8b40-ead103f9edef-root\") pod \"node-exporter-6wps2\" (UID: \"8c194403-1e4f-4469-8b40-ead103f9edef\") " pod="openshift-monitoring/node-exporter-6wps2" Jul 09 15:03:49.456706 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.456179 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8c194403-1e4f-4469-8b40-ead103f9edef-node-exporter-textfile\") pod \"node-exporter-6wps2\" (UID: \"8c194403-1e4f-4469-8b40-ead103f9edef\") " pod="openshift-monitoring/node-exporter-6wps2" Jul 09 15:03:49.456706 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.456198 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8c194403-1e4f-4469-8b40-ead103f9edef-sys\") pod \"node-exporter-6wps2\" (UID: \"8c194403-1e4f-4469-8b40-ead103f9edef\") " pod="openshift-monitoring/node-exporter-6wps2" Jul 09 15:03:49.456706 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.456279 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8c194403-1e4f-4469-8b40-ead103f9edef-node-exporter-wtmp\") pod \"node-exporter-6wps2\" (UID: \"8c194403-1e4f-4469-8b40-ead103f9edef\") " pod="openshift-monitoring/node-exporter-6wps2" Jul 09 15:03:49.456706 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.456605 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/38241def-a91a-478f-b3cc-cf4f2eea029d-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-nzgcs\" (UID: \"38241def-a91a-478f-b3cc-cf4f2eea029d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nzgcs" Jul 09 15:03:49.456706 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.456620 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8c194403-1e4f-4469-8b40-ead103f9edef-node-exporter-accelerators-collector-config\") pod \"node-exporter-6wps2\" (UID: \"8c194403-1e4f-4469-8b40-ead103f9edef\") " pod="openshift-monitoring/node-exporter-6wps2" Jul 09 15:03:49.457267 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.456786 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38241def-a91a-478f-b3cc-cf4f2eea029d-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-nzgcs\" (UID: \"38241def-a91a-478f-b3cc-cf4f2eea029d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nzgcs" Jul 09 15:03:49.457267 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.457258 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/38241def-a91a-478f-b3cc-cf4f2eea029d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-nzgcs\" (UID: \"38241def-a91a-478f-b3cc-cf4f2eea029d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nzgcs" Jul 09 15:03:49.457598 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.457540 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c194403-1e4f-4469-8b40-ead103f9edef-metrics-client-ca\") pod \"node-exporter-6wps2\" (UID: \"8c194403-1e4f-4469-8b40-ead103f9edef\") " pod="openshift-monitoring/node-exporter-6wps2" Jul 09 15:03:49.459801 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.459752 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8c194403-1e4f-4469-8b40-ead103f9edef-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6wps2\" (UID: \"8c194403-1e4f-4469-8b40-ead103f9edef\") " pod="openshift-monitoring/node-exporter-6wps2" Jul 09 15:03:49.459921 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.459902 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/38241def-a91a-478f-b3cc-cf4f2eea029d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-nzgcs\" (UID: \"38241def-a91a-478f-b3cc-cf4f2eea029d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nzgcs" Jul 09 15:03:49.459988 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.459942 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/38241def-a91a-478f-b3cc-cf4f2eea029d-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-nzgcs\" (UID: \"38241def-a91a-478f-b3cc-cf4f2eea029d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nzgcs" Jul 09 15:03:49.461012 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.460989 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8c194403-1e4f-4469-8b40-ead103f9edef-node-exporter-tls\") pod \"node-exporter-6wps2\" (UID: \"8c194403-1e4f-4469-8b40-ead103f9edef\") " pod="openshift-monitoring/node-exporter-6wps2" Jul 09 15:03:49.463214 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.463147 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:49.469153 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.469049 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48zwm\" (UniqueName: \"kubernetes.io/projected/8c194403-1e4f-4469-8b40-ead103f9edef-kube-api-access-48zwm\") pod \"node-exporter-6wps2\" (UID: \"8c194403-1e4f-4469-8b40-ead103f9edef\") " pod="openshift-monitoring/node-exporter-6wps2" Jul 09 15:03:49.474074 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.474022 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kcvv\" (UniqueName: \"kubernetes.io/projected/38241def-a91a-478f-b3cc-cf4f2eea029d-kube-api-access-2kcvv\") pod \"kube-state-metrics-785778ff74-nzgcs\" (UID: \"38241def-a91a-478f-b3cc-cf4f2eea029d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nzgcs" Jul 09 15:03:49.525854 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.525797 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6wps2" Jul 09 15:03:49.551900 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.551655 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-nzgcs" Jul 09 15:03:49.566540 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.566499 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:49.859895 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.859859 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/af9403ab-61af-488f-b826-9cf5be0bcd3a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-kc99w\" (UID: \"af9403ab-61af-488f-b826-9cf5be0bcd3a\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kc99w" Jul 09 15:03:49.861193 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.861043 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:49.863257 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:49.863160 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/af9403ab-61af-488f-b826-9cf5be0bcd3a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-kc99w\" (UID: \"af9403ab-61af-488f-b826-9cf5be0bcd3a\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kc99w" Jul 09 15:03:50.033189 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.033077 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:50.109609 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.109569 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kc99w" Jul 09 15:03:50.130668 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.130102 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 15:03:50.153287 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.153254 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 15:03:50.153515 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.153452 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.157648 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.157584 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 09 15:03:50.161123 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.158672 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 09 15:03:50.161123 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.158631 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 09 15:03:50.163066 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.162965 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 09 15:03:50.163183 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.163104 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 09 15:03:50.163242 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.163226 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 09 15:03:50.166881 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.163396 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 09 15:03:50.166881 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.166670 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-lf8nl\"" Jul 09 15:03:50.167329 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.166958 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 09 15:03:50.167329 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.167187 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 09 15:03:50.264316 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.264168 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.264316 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.264220 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.264316 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.264256 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-config-volume\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.264614 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.264347 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.264614 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.264430 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.264614 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.264459 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.264614 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.264535 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.264614 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.264574 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.264614 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.264604 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hppgg\" (UniqueName: \"kubernetes.io/projected/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-kube-api-access-hppgg\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.264947 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.264633 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.264947 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.264679 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-config-out\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.264947 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.264701 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-web-config\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.264947 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.264740 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.366082 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.365304 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.366082 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.365374 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.366082 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.365400 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.366082 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.365432 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.366082 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.365457 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.366082 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.365481 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hppgg\" (UniqueName: \"kubernetes.io/projected/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-kube-api-access-hppgg\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.366082 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.365507 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.366082 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.365544 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-config-out\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.366082 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.365558 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-web-config\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.366082 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.365585 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.366082 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.365626 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.366082 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.365648 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.366082 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.365675 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-config-volume\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.366082 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.365796 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.367190 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.366885 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.368548 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.368522 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.368677 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:50.368636 2567 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 09 15:03:50.368745 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:03:50.368696 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-secret-alertmanager-main-tls podName:c0cbd21d-6e9f-4fd3-8500-9e907f3075e1 nodeName:}" failed. No retries permitted until 2026-07-09 15:03:50.868676163 +0000 UTC m=+77.599182637 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "c0cbd21d-6e9f-4fd3-8500-9e907f3075e1") : secret "alertmanager-main-tls" not found Jul 09 15:03:50.368831 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.368753 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.370350 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.370300 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-web-config\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.370922 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.370863 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.371664 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.371630 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-config-volume\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.371772 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.371714 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.372258 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.372104 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.373194 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.373050 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-config-out\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.373194 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.373144 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.378154 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.378108 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hppgg\" (UniqueName: \"kubernetes.io/projected/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-kube-api-access-hppgg\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.870096 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.870052 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:50.875517 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:50.874176 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:51.070507 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:51.070217 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:03:51.120853 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:51.120756 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-78898947d8-cj86l"] Jul 09 15:03:51.143395 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:51.143355 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-78898947d8-cj86l" Jul 09 15:03:51.146721 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:51.146691 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 09 15:03:51.146896 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:51.146762 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 09 15:03:51.147100 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:51.147079 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 09 15:03:51.147275 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:51.147258 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 09 15:03:51.147612 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:51.147590 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-851kpq2urbqov\"" Jul 09 15:03:51.149837 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:51.149790 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 09 15:03:51.151473 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:51.151444 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-hmb7j\"" Jul 09 15:03:51.158309 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:51.158278 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-78898947d8-cj86l"] Jul 09 15:03:51.274921 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:51.274869 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e0367d01-739b-4d39-9457-e5702482e786-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-78898947d8-cj86l\" (UID: \"e0367d01-739b-4d39-9457-e5702482e786\") " pod="openshift-monitoring/thanos-querier-78898947d8-cj86l" Jul 09 15:03:51.275110 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:51.274939 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e0367d01-739b-4d39-9457-e5702482e786-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-78898947d8-cj86l\" (UID: \"e0367d01-739b-4d39-9457-e5702482e786\") " pod="openshift-monitoring/thanos-querier-78898947d8-cj86l" Jul 09 15:03:51.275110 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:51.275018 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e0367d01-739b-4d39-9457-e5702482e786-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-78898947d8-cj86l\" (UID: \"e0367d01-739b-4d39-9457-e5702482e786\") " pod="openshift-monitoring/thanos-querier-78898947d8-cj86l" Jul 09 15:03:51.275221 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:51.275099 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e0367d01-739b-4d39-9457-e5702482e786-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-78898947d8-cj86l\" (UID: \"e0367d01-739b-4d39-9457-e5702482e786\") " pod="openshift-monitoring/thanos-querier-78898947d8-cj86l" Jul 09 15:03:51.275221 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:51.275182 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e0367d01-739b-4d39-9457-e5702482e786-secret-thanos-querier-tls\") pod \"thanos-querier-78898947d8-cj86l\" (UID: \"e0367d01-739b-4d39-9457-e5702482e786\") " pod="openshift-monitoring/thanos-querier-78898947d8-cj86l" Jul 09 15:03:51.275221 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:51.275211 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e0367d01-739b-4d39-9457-e5702482e786-secret-grpc-tls\") pod \"thanos-querier-78898947d8-cj86l\" (UID: \"e0367d01-739b-4d39-9457-e5702482e786\") " pod="openshift-monitoring/thanos-querier-78898947d8-cj86l" Jul 09 15:03:51.275362 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:51.275286 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0367d01-739b-4d39-9457-e5702482e786-metrics-client-ca\") pod \"thanos-querier-78898947d8-cj86l\" (UID: \"e0367d01-739b-4d39-9457-e5702482e786\") " pod="openshift-monitoring/thanos-querier-78898947d8-cj86l" Jul 09 15:03:51.275362 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:51.275341 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkzds\" (UniqueName: \"kubernetes.io/projected/e0367d01-739b-4d39-9457-e5702482e786-kube-api-access-kkzds\") pod \"thanos-querier-78898947d8-cj86l\" (UID: \"e0367d01-739b-4d39-9457-e5702482e786\") " pod="openshift-monitoring/thanos-querier-78898947d8-cj86l" Jul 09 15:03:51.376489 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:51.376390 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e0367d01-739b-4d39-9457-e5702482e786-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-78898947d8-cj86l\" (UID: \"e0367d01-739b-4d39-9457-e5702482e786\") " pod="openshift-monitoring/thanos-querier-78898947d8-cj86l" Jul 09 15:03:51.376489 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:51.376446 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e0367d01-739b-4d39-9457-e5702482e786-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-78898947d8-cj86l\" (UID: \"e0367d01-739b-4d39-9457-e5702482e786\") " pod="openshift-monitoring/thanos-querier-78898947d8-cj86l" Jul 09 15:03:51.376489 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:51.376482 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e0367d01-739b-4d39-9457-e5702482e786-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-78898947d8-cj86l\" (UID: \"e0367d01-739b-4d39-9457-e5702482e786\") " pod="openshift-monitoring/thanos-querier-78898947d8-cj86l" Jul 09 15:03:51.377100 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:51.376533 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e0367d01-739b-4d39-9457-e5702482e786-secret-thanos-querier-tls\") pod \"thanos-querier-78898947d8-cj86l\" (UID: \"e0367d01-739b-4d39-9457-e5702482e786\") " pod="openshift-monitoring/thanos-querier-78898947d8-cj86l" Jul 09 15:03:51.377100 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:51.376552 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e0367d01-739b-4d39-9457-e5702482e786-secret-grpc-tls\") pod \"thanos-querier-78898947d8-cj86l\" (UID: \"e0367d01-739b-4d39-9457-e5702482e786\") " pod="openshift-monitoring/thanos-querier-78898947d8-cj86l" Jul 09 15:03:51.377100 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:51.376625 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0367d01-739b-4d39-9457-e5702482e786-metrics-client-ca\") pod \"thanos-querier-78898947d8-cj86l\" (UID: \"e0367d01-739b-4d39-9457-e5702482e786\") " pod="openshift-monitoring/thanos-querier-78898947d8-cj86l" Jul 09 15:03:51.377100 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:51.376662 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkzds\" (UniqueName: \"kubernetes.io/projected/e0367d01-739b-4d39-9457-e5702482e786-kube-api-access-kkzds\") pod \"thanos-querier-78898947d8-cj86l\" (UID: \"e0367d01-739b-4d39-9457-e5702482e786\") " pod="openshift-monitoring/thanos-querier-78898947d8-cj86l" Jul 09 15:03:51.377100 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:51.376688 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e0367d01-739b-4d39-9457-e5702482e786-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-78898947d8-cj86l\" (UID: \"e0367d01-739b-4d39-9457-e5702482e786\") " pod="openshift-monitoring/thanos-querier-78898947d8-cj86l" Jul 09 15:03:51.377436 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:51.377407 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0367d01-739b-4d39-9457-e5702482e786-metrics-client-ca\") pod \"thanos-querier-78898947d8-cj86l\" (UID: \"e0367d01-739b-4d39-9457-e5702482e786\") " pod="openshift-monitoring/thanos-querier-78898947d8-cj86l" Jul 09 15:03:51.379855 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:51.379784 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e0367d01-739b-4d39-9457-e5702482e786-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-78898947d8-cj86l\" (UID: \"e0367d01-739b-4d39-9457-e5702482e786\") " pod="openshift-monitoring/thanos-querier-78898947d8-cj86l" Jul 09 15:03:51.380151 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:51.379977 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e0367d01-739b-4d39-9457-e5702482e786-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-78898947d8-cj86l\" (UID: \"e0367d01-739b-4d39-9457-e5702482e786\") " pod="openshift-monitoring/thanos-querier-78898947d8-cj86l" Jul 09 15:03:51.380151 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:51.380083 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e0367d01-739b-4d39-9457-e5702482e786-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-78898947d8-cj86l\" (UID: \"e0367d01-739b-4d39-9457-e5702482e786\") " pod="openshift-monitoring/thanos-querier-78898947d8-cj86l" Jul 09 15:03:51.380353 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:51.380324 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e0367d01-739b-4d39-9457-e5702482e786-secret-grpc-tls\") pod \"thanos-querier-78898947d8-cj86l\" (UID: \"e0367d01-739b-4d39-9457-e5702482e786\") " pod="openshift-monitoring/thanos-querier-78898947d8-cj86l" Jul 09 15:03:51.381061 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:51.381037 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e0367d01-739b-4d39-9457-e5702482e786-secret-thanos-querier-tls\") pod \"thanos-querier-78898947d8-cj86l\" (UID: \"e0367d01-739b-4d39-9457-e5702482e786\") " pod="openshift-monitoring/thanos-querier-78898947d8-cj86l" Jul 09 15:03:51.381331 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:51.381309 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e0367d01-739b-4d39-9457-e5702482e786-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-78898947d8-cj86l\" (UID: \"e0367d01-739b-4d39-9457-e5702482e786\") " pod="openshift-monitoring/thanos-querier-78898947d8-cj86l" Jul 09 15:03:51.390155 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:51.390130 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkzds\" (UniqueName: \"kubernetes.io/projected/e0367d01-739b-4d39-9457-e5702482e786-kube-api-access-kkzds\") pod \"thanos-querier-78898947d8-cj86l\" (UID: \"e0367d01-739b-4d39-9457-e5702482e786\") " pod="openshift-monitoring/thanos-querier-78898947d8-cj86l" Jul 09 15:03:51.455109 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:51.455070 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-78898947d8-cj86l" Jul 09 15:03:52.367521 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:52.367453 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:52.436433 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:52.436335 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:52.923150 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:03:52.921182 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c194403_1e4f_4469_8b40_ead103f9edef.slice/crio-5c00930ba344bdcf303e2afdb4a896aeda8ad9390fe6b367c41f8a3862b5fb9d WatchSource:0}: Error finding container 5c00930ba344bdcf303e2afdb4a896aeda8ad9390fe6b367c41f8a3862b5fb9d: Status 404 returned error can't find the container with id 5c00930ba344bdcf303e2afdb4a896aeda8ad9390fe6b367c41f8a3862b5fb9d Jul 09 15:03:53.193791 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.193428 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-nzgcs"] Jul 09 15:03:53.196311 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.195502 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b6cbb4b6d-qfx28" event={"ID":"7aa645ea-62ea-493b-b863-9ea493f80c7b","Type":"ContainerStarted","Data":"3f38b50fb88d55968150c901dcab613f320cec9abbd7de45aa0a01455d596ea5"} Jul 09 15:03:53.200895 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.200792 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-96fg2" event={"ID":"d469078c-492e-4dbd-843f-7d1699fa1ce9","Type":"ContainerStarted","Data":"a23189a45e5228917ad597c47cd13dc6b0d0fc8fdeaed0d94f0c29db30a3d807"} Jul 09 15:03:53.222630 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.220651 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5b6cbb4b6d-qfx28" podStartSLOduration=7.35616867 podStartE2EDuration="16.220628391s" totalImagesPullingTime="8.864459721s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:03:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:03:53.220107942 +0000 UTC m=+79.950614426" watchObservedRunningTime="2026-07-09 15:03:53.220628391 +0000 UTC m=+79.951134888" Jul 09 15:03:53.226144 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.224689 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jcctg" event={"ID":"37399ba7-ce43-46e3-b78e-73b29c2196be","Type":"ContainerStarted","Data":"42df151faa15e7edc3a09e68c85e25dcee5263b405bcc2f21c65d8f2dc9ebdb9"} Jul 09 15:03:53.227297 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.227272 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6wps2" event={"ID":"8c194403-1e4f-4469-8b40-ead103f9edef","Type":"ContainerStarted","Data":"5c00930ba344bdcf303e2afdb4a896aeda8ad9390fe6b367c41f8a3862b5fb9d"} Jul 09 15:03:53.236318 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.236291 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8796dfb78-gjmt4"] Jul 09 15:03:53.239796 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:03:53.239641 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdaffe4c_36aa_4fee_b356_87897e214f9f.slice/crio-b52990795b3905d1dc1924d8492a65f0ca158ae90a5c805a4309b0db19599d8e WatchSource:0}: Error finding container b52990795b3905d1dc1924d8492a65f0ca158ae90a5c805a4309b0db19599d8e: Status 404 returned error can't find the container with id b52990795b3905d1dc1924d8492a65f0ca158ae90a5c805a4309b0db19599d8e Jul 09 15:03:53.240961 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.240290 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-kc99w"] Jul 09 15:03:53.243913 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:03:53.243808 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf9403ab_61af_488f_b826_9cf5be0bcd3a.slice/crio-3737fc9812cf38b182cbd2d5a041554a9af9a298e2db3dff6f9a084af131146d WatchSource:0}: Error finding container 3737fc9812cf38b182cbd2d5a041554a9af9a298e2db3dff6f9a084af131146d: Status 404 returned error can't find the container with id 3737fc9812cf38b182cbd2d5a041554a9af9a298e2db3dff6f9a084af131146d Jul 09 15:03:53.257329 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.257275 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 15:03:53.261464 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:03:53.261427 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0cbd21d_6e9f_4fd3_8500_9e907f3075e1.slice/crio-654fb25a4ad0764ca9de707648f2bfceb92a32ce110b564a5dbea6bf4901ad84 WatchSource:0}: Error finding container 654fb25a4ad0764ca9de707648f2bfceb92a32ce110b564a5dbea6bf4901ad84: Status 404 returned error can't find the container with id 654fb25a4ad0764ca9de707648f2bfceb92a32ce110b564a5dbea6bf4901ad84 Jul 09 15:03:53.277675 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.277650 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-78898947d8-cj86l"] Jul 09 15:03:53.279328 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:03:53.279297 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0367d01_739b_4d39_9457_e5702482e786.slice/crio-29b9a77df252a5a0b24c685f4bf40373f3e4533234ec1e6ce820bbfb1a0a2bc0 WatchSource:0}: Error finding container 29b9a77df252a5a0b24c685f4bf40373f3e4533234ec1e6ce820bbfb1a0a2bc0: Status 404 returned error can't find the container with id 29b9a77df252a5a0b24c685f4bf40373f3e4533234ec1e6ce820bbfb1a0a2bc0 Jul 09 15:03:53.575302 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.575271 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-9bbc5d8d4-gxffh"] Jul 09 15:03:53.600146 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.600105 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-9bbc5d8d4-gxffh"] Jul 09 15:03:53.600319 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.600216 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-9bbc5d8d4-gxffh" Jul 09 15:03:53.603644 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.603423 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 09 15:03:53.603644 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.603454 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 09 15:03:53.603644 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.603512 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-qdf7x\"" Jul 09 15:03:53.603906 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.603750 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 09 15:03:53.603906 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.603810 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 09 15:03:53.604131 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.604109 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-3mtptd39sm6s0\"" Jul 09 15:03:53.702580 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.702503 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4nxt\" (UniqueName: \"kubernetes.io/projected/48425dff-9ae5-4922-a795-503e8ea60a7c-kube-api-access-t4nxt\") pod \"metrics-server-9bbc5d8d4-gxffh\" (UID: \"48425dff-9ae5-4922-a795-503e8ea60a7c\") " pod="openshift-monitoring/metrics-server-9bbc5d8d4-gxffh" Jul 09 15:03:53.702580 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.702557 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/48425dff-9ae5-4922-a795-503e8ea60a7c-metrics-server-audit-profiles\") pod \"metrics-server-9bbc5d8d4-gxffh\" (UID: \"48425dff-9ae5-4922-a795-503e8ea60a7c\") " pod="openshift-monitoring/metrics-server-9bbc5d8d4-gxffh" Jul 09 15:03:53.702768 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.702581 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/48425dff-9ae5-4922-a795-503e8ea60a7c-audit-log\") pod \"metrics-server-9bbc5d8d4-gxffh\" (UID: \"48425dff-9ae5-4922-a795-503e8ea60a7c\") " pod="openshift-monitoring/metrics-server-9bbc5d8d4-gxffh" Jul 09 15:03:53.702768 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.702602 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48425dff-9ae5-4922-a795-503e8ea60a7c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-9bbc5d8d4-gxffh\" (UID: \"48425dff-9ae5-4922-a795-503e8ea60a7c\") " pod="openshift-monitoring/metrics-server-9bbc5d8d4-gxffh" Jul 09 15:03:53.702768 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.702654 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/48425dff-9ae5-4922-a795-503e8ea60a7c-secret-metrics-server-tls\") pod \"metrics-server-9bbc5d8d4-gxffh\" (UID: \"48425dff-9ae5-4922-a795-503e8ea60a7c\") " pod="openshift-monitoring/metrics-server-9bbc5d8d4-gxffh" Jul 09 15:03:53.702768 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.702722 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/48425dff-9ae5-4922-a795-503e8ea60a7c-secret-metrics-server-client-certs\") pod \"metrics-server-9bbc5d8d4-gxffh\" (UID: \"48425dff-9ae5-4922-a795-503e8ea60a7c\") " pod="openshift-monitoring/metrics-server-9bbc5d8d4-gxffh" Jul 09 15:03:53.702768 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.702746 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/48425dff-9ae5-4922-a795-503e8ea60a7c-client-ca-bundle\") pod \"metrics-server-9bbc5d8d4-gxffh\" (UID: \"48425dff-9ae5-4922-a795-503e8ea60a7c\") " pod="openshift-monitoring/metrics-server-9bbc5d8d4-gxffh" Jul 09 15:03:53.787763 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.787731 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:53.803796 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.803767 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/48425dff-9ae5-4922-a795-503e8ea60a7c-metrics-server-audit-profiles\") pod \"metrics-server-9bbc5d8d4-gxffh\" (UID: \"48425dff-9ae5-4922-a795-503e8ea60a7c\") " pod="openshift-monitoring/metrics-server-9bbc5d8d4-gxffh" Jul 09 15:03:53.803964 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.803804 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/48425dff-9ae5-4922-a795-503e8ea60a7c-audit-log\") pod \"metrics-server-9bbc5d8d4-gxffh\" (UID: \"48425dff-9ae5-4922-a795-503e8ea60a7c\") " pod="openshift-monitoring/metrics-server-9bbc5d8d4-gxffh" Jul 09 15:03:53.804044 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.804021 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48425dff-9ae5-4922-a795-503e8ea60a7c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-9bbc5d8d4-gxffh\" (UID: \"48425dff-9ae5-4922-a795-503e8ea60a7c\") " pod="openshift-monitoring/metrics-server-9bbc5d8d4-gxffh" Jul 09 15:03:53.804152 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.804097 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/48425dff-9ae5-4922-a795-503e8ea60a7c-secret-metrics-server-tls\") pod \"metrics-server-9bbc5d8d4-gxffh\" (UID: \"48425dff-9ae5-4922-a795-503e8ea60a7c\") " pod="openshift-monitoring/metrics-server-9bbc5d8d4-gxffh" Jul 09 15:03:53.804206 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.804167 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/48425dff-9ae5-4922-a795-503e8ea60a7c-secret-metrics-server-client-certs\") pod \"metrics-server-9bbc5d8d4-gxffh\" (UID: \"48425dff-9ae5-4922-a795-503e8ea60a7c\") " pod="openshift-monitoring/metrics-server-9bbc5d8d4-gxffh" Jul 09 15:03:53.804258 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.804202 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/48425dff-9ae5-4922-a795-503e8ea60a7c-client-ca-bundle\") pod \"metrics-server-9bbc5d8d4-gxffh\" (UID: \"48425dff-9ae5-4922-a795-503e8ea60a7c\") " pod="openshift-monitoring/metrics-server-9bbc5d8d4-gxffh" Jul 09 15:03:53.804307 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.804261 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4nxt\" (UniqueName: \"kubernetes.io/projected/48425dff-9ae5-4922-a795-503e8ea60a7c-kube-api-access-t4nxt\") pod \"metrics-server-9bbc5d8d4-gxffh\" (UID: \"48425dff-9ae5-4922-a795-503e8ea60a7c\") " pod="openshift-monitoring/metrics-server-9bbc5d8d4-gxffh" Jul 09 15:03:53.804307 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.804281 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/48425dff-9ae5-4922-a795-503e8ea60a7c-audit-log\") pod \"metrics-server-9bbc5d8d4-gxffh\" (UID: \"48425dff-9ae5-4922-a795-503e8ea60a7c\") " pod="openshift-monitoring/metrics-server-9bbc5d8d4-gxffh" Jul 09 15:03:53.804782 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.804730 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48425dff-9ae5-4922-a795-503e8ea60a7c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-9bbc5d8d4-gxffh\" (UID: \"48425dff-9ae5-4922-a795-503e8ea60a7c\") " pod="openshift-monitoring/metrics-server-9bbc5d8d4-gxffh" Jul 09 15:03:53.805006 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.804962 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/48425dff-9ae5-4922-a795-503e8ea60a7c-metrics-server-audit-profiles\") pod \"metrics-server-9bbc5d8d4-gxffh\" (UID: \"48425dff-9ae5-4922-a795-503e8ea60a7c\") " pod="openshift-monitoring/metrics-server-9bbc5d8d4-gxffh" Jul 09 15:03:53.807785 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.806931 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-w44lv"] Jul 09 15:03:53.807785 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.807513 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/48425dff-9ae5-4922-a795-503e8ea60a7c-secret-metrics-server-client-certs\") pod \"metrics-server-9bbc5d8d4-gxffh\" (UID: \"48425dff-9ae5-4922-a795-503e8ea60a7c\") " pod="openshift-monitoring/metrics-server-9bbc5d8d4-gxffh" Jul 09 15:03:53.807785 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.807617 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/48425dff-9ae5-4922-a795-503e8ea60a7c-client-ca-bundle\") pod \"metrics-server-9bbc5d8d4-gxffh\" (UID: \"48425dff-9ae5-4922-a795-503e8ea60a7c\") " pod="openshift-monitoring/metrics-server-9bbc5d8d4-gxffh" Jul 09 15:03:53.808807 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.808784 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/48425dff-9ae5-4922-a795-503e8ea60a7c-secret-metrics-server-tls\") pod \"metrics-server-9bbc5d8d4-gxffh\" (UID: \"48425dff-9ae5-4922-a795-503e8ea60a7c\") " pod="openshift-monitoring/metrics-server-9bbc5d8d4-gxffh" Jul 09 15:03:53.814109 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.814089 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4nxt\" (UniqueName: \"kubernetes.io/projected/48425dff-9ae5-4922-a795-503e8ea60a7c-kube-api-access-t4nxt\") pod \"metrics-server-9bbc5d8d4-gxffh\" (UID: \"48425dff-9ae5-4922-a795-503e8ea60a7c\") " pod="openshift-monitoring/metrics-server-9bbc5d8d4-gxffh" Jul 09 15:03:53.830181 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.829498 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-w44lv" Jul 09 15:03:53.832463 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.832434 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 09 15:03:53.832463 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.832451 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-4xtsh\"" Jul 09 15:03:53.837117 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.837098 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-w44lv"] Jul 09 15:03:53.889089 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.889052 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:53.905151 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.905117 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2e6e9898-83c1-42bf-862a-ee35c5629109-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-w44lv\" (UID: \"2e6e9898-83c1-42bf-862a-ee35c5629109\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-w44lv" Jul 09 15:03:53.905331 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.905188 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzfcj\" (UniqueName: \"kubernetes.io/projected/2e6e9898-83c1-42bf-862a-ee35c5629109-kube-api-access-hzfcj\") pod \"monitoring-plugin-5f68df7f58-w44lv\" (UID: \"2e6e9898-83c1-42bf-862a-ee35c5629109\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-w44lv" Jul 09 15:03:53.912128 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:53.912068 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-9bbc5d8d4-gxffh" Jul 09 15:03:54.001429 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:54.001389 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:54.006512 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:54.006483 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2e6e9898-83c1-42bf-862a-ee35c5629109-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-w44lv\" (UID: \"2e6e9898-83c1-42bf-862a-ee35c5629109\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-w44lv" Jul 09 15:03:54.006685 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:54.006549 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzfcj\" (UniqueName: \"kubernetes.io/projected/2e6e9898-83c1-42bf-862a-ee35c5629109-kube-api-access-hzfcj\") pod \"monitoring-plugin-5f68df7f58-w44lv\" (UID: \"2e6e9898-83c1-42bf-862a-ee35c5629109\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-w44lv" Jul 09 15:03:54.009459 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:54.009432 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2e6e9898-83c1-42bf-862a-ee35c5629109-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-w44lv\" (UID: \"2e6e9898-83c1-42bf-862a-ee35c5629109\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-w44lv" Jul 09 15:03:54.016854 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:54.016826 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzfcj\" (UniqueName: \"kubernetes.io/projected/2e6e9898-83c1-42bf-862a-ee35c5629109-kube-api-access-hzfcj\") pod \"monitoring-plugin-5f68df7f58-w44lv\" (UID: \"2e6e9898-83c1-42bf-862a-ee35c5629109\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-w44lv" Jul 09 15:03:54.155518 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:54.155485 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-w44lv" Jul 09 15:03:54.194186 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:54.194161 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-9bbc5d8d4-gxffh"] Jul 09 15:03:54.197491 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:03:54.197456 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48425dff_9ae5_4922_a795_503e8ea60a7c.slice/crio-06390c73dffc6c2b29185db49e008d75ef3d1e3a4ae9d6fc9272282cf3771c23 WatchSource:0}: Error finding container 06390c73dffc6c2b29185db49e008d75ef3d1e3a4ae9d6fc9272282cf3771c23: Status 404 returned error can't find the container with id 06390c73dffc6c2b29185db49e008d75ef3d1e3a4ae9d6fc9272282cf3771c23 Jul 09 15:03:54.235020 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:54.234889 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jcctg" event={"ID":"37399ba7-ce43-46e3-b78e-73b29c2196be","Type":"ContainerStarted","Data":"1925eb6e3357d6b9ad7bdc3a3d8db5604d54b377ac1d3d1d48ab9f1e1a9747f6"} Jul 09 15:03:54.237318 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:54.237263 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-9bbc5d8d4-gxffh" event={"ID":"48425dff-9ae5-4922-a795-503e8ea60a7c","Type":"ContainerStarted","Data":"06390c73dffc6c2b29185db49e008d75ef3d1e3a4ae9d6fc9272282cf3771c23"} Jul 09 15:03:54.239033 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:54.238942 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78898947d8-cj86l" event={"ID":"e0367d01-739b-4d39-9457-e5702482e786","Type":"ContainerStarted","Data":"29b9a77df252a5a0b24c685f4bf40373f3e4533234ec1e6ce820bbfb1a0a2bc0"} Jul 09 15:03:54.240531 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:54.240503 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1","Type":"ContainerStarted","Data":"654fb25a4ad0764ca9de707648f2bfceb92a32ce110b564a5dbea6bf4901ad84"} Jul 09 15:03:54.242021 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:54.241947 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-nzgcs" event={"ID":"38241def-a91a-478f-b3cc-cf4f2eea029d","Type":"ContainerStarted","Data":"8fdec15ab051734aa2a2b88d7fdd68a20d8a28cd63f89380d46900f2b18721c2"} Jul 09 15:03:54.247543 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:54.246931 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8796dfb78-gjmt4" event={"ID":"bdaffe4c-36aa-4fee-b356-87897e214f9f","Type":"ContainerStarted","Data":"ea04a8898fed2bbf78d1d10c4f0a94bfbfaaad0242d4dcf818d11cb8c36e9c84"} Jul 09 15:03:54.247543 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:54.246992 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8796dfb78-gjmt4" event={"ID":"bdaffe4c-36aa-4fee-b356-87897e214f9f","Type":"ContainerStarted","Data":"b52990795b3905d1dc1924d8492a65f0ca158ae90a5c805a4309b0db19599d8e"} Jul 09 15:03:54.249449 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:54.249423 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kc99w" event={"ID":"af9403ab-61af-488f-b826-9cf5be0bcd3a","Type":"ContainerStarted","Data":"bd7d16673d7cc15dd9192fd425c0d78c93b89e0b7af84331c26ce5e554643951"} Jul 09 15:03:54.249552 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:54.249478 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kc99w" event={"ID":"af9403ab-61af-488f-b826-9cf5be0bcd3a","Type":"ContainerStarted","Data":"abb44cb2d60abcc19879258ff49b2974bf64322502e38fcac37b7b2722247f3f"} Jul 09 15:03:54.249552 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:54.249508 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kc99w" event={"ID":"af9403ab-61af-488f-b826-9cf5be0bcd3a","Type":"ContainerStarted","Data":"3737fc9812cf38b182cbd2d5a041554a9af9a298e2db3dff6f9a084af131146d"} Jul 09 15:03:54.251762 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:54.251707 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-p8vsh" event={"ID":"fce18369-7aaf-4358-b5f5-46b0218924fd","Type":"ContainerStarted","Data":"7fbd17d764c970cdeab7b9da175cbfc736ef335279961c86a961e3c84c53b771"} Jul 09 15:03:54.252059 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:54.252043 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:03:54.288876 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:54.288788 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-jcctg" podStartSLOduration=72.38173411 podStartE2EDuration="1m21.288767675s" totalImagesPullingTime="8.907033565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:02:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:03:54.259947799 +0000 UTC m=+80.990454291" watchObservedRunningTime="2026-07-09 15:03:54.288767675 +0000 UTC m=+81.019274167" Jul 09 15:03:54.289881 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:54.289838 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8796dfb78-gjmt4" podStartSLOduration=6.289811801 podStartE2EDuration="6.289811801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:03:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:03:54.288334369 +0000 UTC m=+81.018840861" watchObservedRunningTime="2026-07-09 15:03:54.289811801 +0000 UTC m=+81.020318291" Jul 09 15:03:54.305914 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:54.305864 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-p8vsh" podStartSLOduration=72.387677806 podStartE2EDuration="1m21.305846212s" totalImagesPullingTime="8.918168406s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:02:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:03:54.304046771 +0000 UTC m=+81.034553273" watchObservedRunningTime="2026-07-09 15:03:54.305846212 +0000 UTC m=+81.036352702" Jul 09 15:03:54.320630 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:54.320580 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-96fg2" podStartSLOduration=72.444813599 podStartE2EDuration="1m21.320564746s" totalImagesPullingTime="8.875751147s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:02:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:03:54.32032356 +0000 UTC m=+81.050830051" watchObservedRunningTime="2026-07-09 15:03:54.320564746 +0000 UTC m=+81.051071237" Jul 09 15:03:54.329121 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:54.329059 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-w44lv"] Jul 09 15:03:54.390674 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:54.390603 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:54.457523 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:54.457139 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:54.736519 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:03:54.736478 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e6e9898_83c1_42bf_862a_ee35c5629109.slice/crio-3f5618961e218c8305a9f917391c1f31e9479b4f8ff2333b21f267f6fc257b9a WatchSource:0}: Error finding container 3f5618961e218c8305a9f917391c1f31e9479b4f8ff2333b21f267f6fc257b9a: Status 404 returned error can't find the container with id 3f5618961e218c8305a9f917391c1f31e9479b4f8ff2333b21f267f6fc257b9a Jul 09 15:03:55.257223 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.257184 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-w44lv" event={"ID":"2e6e9898-83c1-42bf-862a-ee35c5629109","Type":"ContainerStarted","Data":"3f5618961e218c8305a9f917391c1f31e9479b4f8ff2333b21f267f6fc257b9a"} Jul 09 15:03:55.259272 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.259238 2567 generic.go:356] "Generic (PLEG): container finished" podID="8c194403-1e4f-4469-8b40-ead103f9edef" containerID="d428176eb0eb8b2fdd5afa91eca67fe4d1b6d2e94a28e9589576af5bbbb8631d" exitCode=0 Jul 09 15:03:55.259425 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.259364 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6wps2" event={"ID":"8c194403-1e4f-4469-8b40-ead103f9edef","Type":"ContainerDied","Data":"d428176eb0eb8b2fdd5afa91eca67fe4d1b6d2e94a28e9589576af5bbbb8631d"} Jul 09 15:03:55.330376 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.330274 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 15:03:55.338608 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.338572 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:55.352673 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.352645 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.352893 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.352786 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 15:03:55.356718 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.356231 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 09 15:03:55.356718 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.356336 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-etcverbk2i7iu\"" Jul 09 15:03:55.356718 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.356388 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 09 15:03:55.356718 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.356403 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-8cm8l\"" Jul 09 15:03:55.356718 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.356240 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 09 15:03:55.356718 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.356548 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 09 15:03:55.356718 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.356569 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 09 15:03:55.358066 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.357515 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 09 15:03:55.358066 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.357642 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 09 15:03:55.358066 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.357781 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 09 15:03:55.358066 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.357914 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 09 15:03:55.358066 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.358027 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 09 15:03:55.358381 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.358303 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 09 15:03:55.358575 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.358553 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 09 15:03:55.359812 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.359788 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 09 15:03:55.362447 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.362423 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 09 15:03:55.419356 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.419313 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d6a54937-6cfe-4b4e-9860-0af0729aea4a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.419356 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.419356 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d6a54937-6cfe-4b4e-9860-0af0729aea4a-config-out\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.419591 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.419444 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.419591 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.419476 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d6a54937-6cfe-4b4e-9860-0af0729aea4a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.419591 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.419515 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.419591 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.419544 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.419591 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.419587 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-web-config\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.419840 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.419614 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.419840 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.419688 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.419840 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.419729 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.419840 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.419758 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.419840 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.419782 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5w62l\" (UniqueName: \"kubernetes.io/projected/d6a54937-6cfe-4b4e-9860-0af0729aea4a-kube-api-access-5w62l\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.420073 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.419931 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.420073 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.419992 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.420073 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.420011 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.420216 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.420141 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.420216 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.420191 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.420314 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.420224 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-config\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.420314 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.420270 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.420393 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.420330 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.521208 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.521173 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.521404 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.521219 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.521404 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.521250 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.521404 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.521278 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5w62l\" (UniqueName: \"kubernetes.io/projected/d6a54937-6cfe-4b4e-9860-0af0729aea4a-kube-api-access-5w62l\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.521404 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.521306 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.521404 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.521341 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.521404 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.521365 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.521404 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.521394 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.521704 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.521424 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.521704 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.521458 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-config\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.521704 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.521473 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.521704 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.521487 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.521704 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.521504 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d6a54937-6cfe-4b4e-9860-0af0729aea4a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.521704 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.521538 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d6a54937-6cfe-4b4e-9860-0af0729aea4a-config-out\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.521704 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.521580 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.521704 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.521609 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d6a54937-6cfe-4b4e-9860-0af0729aea4a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.521704 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.521648 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.521704 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.521678 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.522275 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.521722 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-web-config\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.522275 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.521748 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.522275 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.522112 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.522275 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.522167 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.522275 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.522206 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.522275 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.522220 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.522563 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.522415 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.523186 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.522873 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d6a54937-6cfe-4b4e-9860-0af0729aea4a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.523623 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.523565 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.528503 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.528477 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.529296 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.529273 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-web-config\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.529421 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.529292 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d6a54937-6cfe-4b4e-9860-0af0729aea4a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.529793 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.529759 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.531195 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.530342 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.531195 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.530744 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-config\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.531195 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.531160 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.532044 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.531988 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.532044 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.532018 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.532198 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.532097 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.532371 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.532351 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d6a54937-6cfe-4b4e-9860-0af0729aea4a-config-out\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.532530 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.532507 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.534068 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.533837 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5w62l\" (UniqueName: \"kubernetes.io/projected/d6a54937-6cfe-4b4e-9860-0af0729aea4a-kube-api-access-5w62l\") pod \"prometheus-k8s-0\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:55.667206 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:55.667121 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:03:56.494256 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:56.494132 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b6cbb4b6d-qfx28"] Jul 09 15:03:56.510659 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:56.510167 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 15:03:56.821226 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:56.821192 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:56.903648 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:56.903608 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:57.269450 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:57.269417 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6wps2" event={"ID":"8c194403-1e4f-4469-8b40-ead103f9edef","Type":"ContainerStarted","Data":"98d8f5a5d9b4fdd59350158aedeaa811ee287bb1b333d7cf2d852f9593196890"} Jul 09 15:03:57.271224 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:57.271192 2567 generic.go:356] "Generic (PLEG): container finished" podID="c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" containerID="ba2aa5b3e3407de69a39d4e006a5fab75c4457c7aff253d201dc8fcf268f0020" exitCode=0 Jul 09 15:03:57.271371 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:57.271285 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1","Type":"ContainerDied","Data":"ba2aa5b3e3407de69a39d4e006a5fab75c4457c7aff253d201dc8fcf268f0020"} Jul 09 15:03:57.273864 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:57.273831 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kc99w" event={"ID":"af9403ab-61af-488f-b826-9cf5be0bcd3a","Type":"ContainerStarted","Data":"55ef0ffa2f9daa3a1594197a0550b1f532632eb8febfb803c2e90b2fc9e0a8b4"} Jul 09 15:03:57.275478 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:57.275447 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-nzgcs" event={"ID":"38241def-a91a-478f-b3cc-cf4f2eea029d","Type":"ContainerStarted","Data":"3bddfc95ca18af9a7b5239ac259ec623aa62344dcc4d5f878d5ea095962ba8cc"} Jul 09 15:03:57.335747 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:57.335686 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kc99w" podStartSLOduration=5.535371487 podStartE2EDuration="8.335669253s" totalImagesPullingTime="2.800297766s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:03:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:03:57.333495315 +0000 UTC m=+84.064001823" watchObservedRunningTime="2026-07-09 15:03:57.335669253 +0000 UTC m=+84.066175743" Jul 09 15:03:57.481708 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:03:57.481678 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6a54937_6cfe_4b4e_9860_0af0729aea4a.slice/crio-20c1c86f7abb7fff636f464c6c036276d74813eca592f5b331304a566bd93c44 WatchSource:0}: Error finding container 20c1c86f7abb7fff636f464c6c036276d74813eca592f5b331304a566bd93c44: Status 404 returned error can't find the container with id 20c1c86f7abb7fff636f464c6c036276d74813eca592f5b331304a566bd93c44 Jul 09 15:03:57.584073 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:57.584038 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b6cbb4b6d-qfx28" Jul 09 15:03:58.285678 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:58.285079 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-9bbc5d8d4-gxffh" event={"ID":"48425dff-9ae5-4922-a795-503e8ea60a7c","Type":"ContainerStarted","Data":"d86422ec455367698939a445877a14f83f18ffcce3a35c72f257c1fad3d676a5"} Jul 09 15:03:58.290085 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:58.289453 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-w44lv" event={"ID":"2e6e9898-83c1-42bf-862a-ee35c5629109","Type":"ContainerStarted","Data":"c93886e1876cd3ea07dde433e72a2a247312578c16d24b00912c6d010465a5fb"} Jul 09 15:03:58.290085 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:58.289993 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-w44lv" Jul 09 15:03:58.299565 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:58.296953 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-nzgcs" event={"ID":"38241def-a91a-478f-b3cc-cf4f2eea029d","Type":"ContainerStarted","Data":"0348f85322971e54d0b57be5d95addab82a8ab6bceb7e3042c7ddb5f1663a207"} Jul 09 15:03:58.299565 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:58.297026 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-nzgcs" event={"ID":"38241def-a91a-478f-b3cc-cf4f2eea029d","Type":"ContainerStarted","Data":"3911ee3404be4b0a0b672359a855dae1c98fbe2e3bf40f6a5b193dd22e3f4254"} Jul 09 15:03:58.299565 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:58.297423 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-w44lv" Jul 09 15:03:58.301230 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:58.300567 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6wps2" event={"ID":"8c194403-1e4f-4469-8b40-ead103f9edef","Type":"ContainerStarted","Data":"7f80574bee6d9f48b27bc32b40ba68ec77a3cd34ca33ad090465bfb288056bc2"} Jul 09 15:03:58.303493 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:58.303462 2567 generic.go:356] "Generic (PLEG): container finished" podID="d6a54937-6cfe-4b4e-9860-0af0729aea4a" containerID="67c4861c1f2c85832af6b0e6f9383cd922269592715adee4c925bac7059f4145" exitCode=0 Jul 09 15:03:58.306470 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:58.306300 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d6a54937-6cfe-4b4e-9860-0af0729aea4a","Type":"ContainerDied","Data":"67c4861c1f2c85832af6b0e6f9383cd922269592715adee4c925bac7059f4145"} Jul 09 15:03:58.306470 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:58.306432 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d6a54937-6cfe-4b4e-9860-0af0729aea4a","Type":"ContainerStarted","Data":"20c1c86f7abb7fff636f464c6c036276d74813eca592f5b331304a566bd93c44"} Jul 09 15:03:58.335241 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:58.335181 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6wps2" podStartSLOduration=7.441173609 podStartE2EDuration="9.33515802s" totalImagesPullingTime="1.830650793s" totalInitContainerRuntime="63.333618ms" isStatefulPod=true podCreationTimestamp="2026-07-09 15:03:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:03:58.334917802 +0000 UTC m=+85.065424294" watchObservedRunningTime="2026-07-09 15:03:58.33515802 +0000 UTC m=+85.065664516" Jul 09 15:03:58.336357 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:58.336303 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-9bbc5d8d4-gxffh" podStartSLOduration=1.6476096550000001 podStartE2EDuration="5.336284726s" totalImagesPullingTime="3.688675071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:03:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:03:58.308085256 +0000 UTC m=+85.038591771" watchObservedRunningTime="2026-07-09 15:03:58.336284726 +0000 UTC m=+85.066791219" Jul 09 15:03:58.388742 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:58.387996 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-nzgcs" podStartSLOduration=6.29018705 podStartE2EDuration="9.387970334s" totalImagesPullingTime="3.097783284s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:03:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:03:58.386330415 +0000 UTC m=+85.116836933" watchObservedRunningTime="2026-07-09 15:03:58.387970334 +0000 UTC m=+85.118476825" Jul 09 15:03:58.605577 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:58.605544 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8796dfb78-gjmt4" Jul 09 15:03:58.606074 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:58.605610 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8796dfb78-gjmt4" Jul 09 15:03:58.611773 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:58.611740 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8796dfb78-gjmt4" Jul 09 15:03:58.638658 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:58.638602 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-w44lv" podStartSLOduration=2.5089957050000002 podStartE2EDuration="5.638581408s" totalImagesPullingTime="3.129585703s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:03:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:03:58.405692273 +0000 UTC m=+85.136198764" watchObservedRunningTime="2026-07-09 15:03:58.638581408 +0000 UTC m=+85.369087893" Jul 09 15:03:58.860300 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:58.860267 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:58.951093 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:58.951043 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:03:59.319473 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:03:59.319095 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8796dfb78-gjmt4" Jul 09 15:04:00.320277 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:00.320202 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1","Type":"ContainerStarted","Data":"6145f817573859661bd28e727b431456427c4fb81842582a4178a06a1ed496a6"} Jul 09 15:04:00.320633 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:00.320285 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1","Type":"ContainerStarted","Data":"4070cd8bc849760dbe2a53ad0c966bfd29f6c481dec96892fcc7918df349472d"} Jul 09 15:04:00.323167 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:00.323123 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78898947d8-cj86l" event={"ID":"e0367d01-739b-4d39-9457-e5702482e786","Type":"ContainerStarted","Data":"57c9a533988e51be3b584a80da97db314982e8bb79cf4e884975f59c38aaaf98"} Jul 09 15:04:00.323280 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:00.323200 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78898947d8-cj86l" event={"ID":"e0367d01-739b-4d39-9457-e5702482e786","Type":"ContainerStarted","Data":"21580d7778861016c20cbdefcb5f0b63bdbb9bd70cb6562a261d1da4ce913029"} Jul 09 15:04:00.377873 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:00.377683 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:04:00.468547 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:00.468457 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:04:01.332809 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:01.332772 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78898947d8-cj86l" event={"ID":"e0367d01-739b-4d39-9457-e5702482e786","Type":"ContainerStarted","Data":"0b632c884099a0a4e302120f823c2a7a7102be4ca9dc0aa5ad823615b5f395cd"} Jul 09 15:04:01.336252 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:01.336221 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1","Type":"ContainerStarted","Data":"41f359a20be86f806772da451ea66e1fbba3a3e2910938cde98df2a429d00f9f"} Jul 09 15:04:01.336415 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:01.336289 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1","Type":"ContainerStarted","Data":"54fcd7d8284d138f8f2cce1b41101a6901aaba25ee3fee9e4a83efa405a8af92"} Jul 09 15:04:01.336415 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:01.336316 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1","Type":"ContainerStarted","Data":"8ec7d5f0ca5c7113e272d21ee0d55c69bb1a3ffb545d00a1f10839066d3f603e"} Jul 09 15:04:03.364675 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:03.364501 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78898947d8-cj86l" event={"ID":"e0367d01-739b-4d39-9457-e5702482e786","Type":"ContainerStarted","Data":"4197cb0f45af81459b65124b49a7ddad6f8efecb66cfac5c10771f4a820bf38c"} Jul 09 15:04:03.364675 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:03.364593 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78898947d8-cj86l" event={"ID":"e0367d01-739b-4d39-9457-e5702482e786","Type":"ContainerStarted","Data":"ec91a0f2e0d1b3757afce5be546101fed4fb9b344c45a0853222ef0fd978407b"} Jul 09 15:04:03.370153 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:03.369443 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d6a54937-6cfe-4b4e-9860-0af0729aea4a","Type":"ContainerStarted","Data":"44d6e12a47ea4ebeb587ec6be072bbbeb0119090d98efe643ebb9eb909b131e9"} Jul 09 15:04:03.370153 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:03.369535 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d6a54937-6cfe-4b4e-9860-0af0729aea4a","Type":"ContainerStarted","Data":"d0e271cb64e3315e8c12cd01d778dc04ae85a8fb65e2889d66971c5b00ee3d74"} Jul 09 15:04:03.377041 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:03.377001 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1","Type":"ContainerStarted","Data":"757a4ca3a9b85773dcf11575d67a81aae250b4a143ea42308a058cf77ecc9630"} Jul 09 15:04:03.427390 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:03.425107 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=5.157689017 podStartE2EDuration="13.425083077s" totalImagesPullingTime="8.246608361s" totalInitContainerRuntime="20.785699ms" isStatefulPod=false podCreationTimestamp="2026-07-09 15:03:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:04:03.419087283 +0000 UTC m=+90.149593786" watchObservedRunningTime="2026-07-09 15:04:03.425083077 +0000 UTC m=+90.155589569" Jul 09 15:04:03.874594 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:03.874536 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:04:03.940413 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:03.940327 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:04:04.387059 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:04.387018 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78898947d8-cj86l" event={"ID":"e0367d01-739b-4d39-9457-e5702482e786","Type":"ContainerStarted","Data":"ece3bdc7d35bf322dd2e10762f6eff9b8799f7db021c7e3996fc281ece3ed208"} Jul 09 15:04:04.387614 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:04.387136 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-78898947d8-cj86l" Jul 09 15:04:04.390205 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:04.390177 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d6a54937-6cfe-4b4e-9860-0af0729aea4a","Type":"ContainerStarted","Data":"d0756f81f1540b764bb51da87cea08cc6efe224344f67388d8ba76023fa5ea53"} Jul 09 15:04:04.390329 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:04.390249 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d6a54937-6cfe-4b4e-9860-0af0729aea4a","Type":"ContainerStarted","Data":"44cb2dc4d500b79fcaa558c7b96a7f4d6b1bde902dbb9ccacb4e4afd7981f777"} Jul 09 15:04:04.390329 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:04.390293 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d6a54937-6cfe-4b4e-9860-0af0729aea4a","Type":"ContainerStarted","Data":"3a1408520418153f2025107bfabbd9d57db7916201bd2878b0d02b8ed05bcb94"} Jul 09 15:04:04.390399 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:04.390333 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d6a54937-6cfe-4b4e-9860-0af0729aea4a","Type":"ContainerStarted","Data":"971b2617db0073606194ffc9c8d782ba2226b9e756e10d4319b2b2a24de7ec20"} Jul 09 15:04:04.413656 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:04.413566 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-78898947d8-cj86l" podStartSLOduration=4.015743497 podStartE2EDuration="13.413549999s" totalImagesPullingTime="9.397806502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:03:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:04:04.411447187 +0000 UTC m=+91.141953689" watchObservedRunningTime="2026-07-09 15:04:04.413549999 +0000 UTC m=+91.144056490" Jul 09 15:04:04.447489 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:04.447415 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.688238721 podStartE2EDuration="9.44738957s" totalImagesPullingTime="4.740938626s" totalInitContainerRuntime="18.212223ms" isStatefulPod=false podCreationTimestamp="2026-07-09 15:03:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:04:04.446246276 +0000 UTC m=+91.176752769" watchObservedRunningTime="2026-07-09 15:04:04.44738957 +0000 UTC m=+91.177896062" Jul 09 15:04:05.311020 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:05.310975 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:04:05.378730 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:05.378691 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:04:05.400784 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:05.400759 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-78898947d8-cj86l" Jul 09 15:04:05.667618 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:05.667508 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:04:06.811217 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:06.811177 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:04:07.362273 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:07.362234 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8796dfb78-gjmt4"] Jul 09 15:04:09.307765 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:09.307729 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:04:10.112008 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:10.111965 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:04:13.912154 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:13.912115 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-9bbc5d8d4-gxffh" Jul 09 15:04:13.912695 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:13.912177 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-9bbc5d8d4-gxffh" Jul 09 15:04:19.339852 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:19.339795 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:04:19.437258 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:19.437221 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:04:21.524323 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:21.524265 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5b6cbb4b6d-qfx28" podUID="7aa645ea-62ea-493b-b863-9ea493f80c7b" containerName="console" containerID="cri-o://3f38b50fb88d55968150c901dcab613f320cec9abbd7de45aa0a01455d596ea5" gracePeriod=15 Jul 09 15:04:21.799175 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:21.799096 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b6cbb4b6d-qfx28" Jul 09 15:04:21.809326 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:21.809292 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:04:21.865948 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:21.865911 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7aa645ea-62ea-493b-b863-9ea493f80c7b-service-ca\") pod \"7aa645ea-62ea-493b-b863-9ea493f80c7b\" (UID: \"7aa645ea-62ea-493b-b863-9ea493f80c7b\") " Jul 09 15:04:21.866147 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:21.866014 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7aa645ea-62ea-493b-b863-9ea493f80c7b-oauth-serving-cert\") pod \"7aa645ea-62ea-493b-b863-9ea493f80c7b\" (UID: \"7aa645ea-62ea-493b-b863-9ea493f80c7b\") " Jul 09 15:04:21.866147 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:21.866065 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7aa645ea-62ea-493b-b863-9ea493f80c7b-console-config\") pod \"7aa645ea-62ea-493b-b863-9ea493f80c7b\" (UID: \"7aa645ea-62ea-493b-b863-9ea493f80c7b\") " Jul 09 15:04:21.866147 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:21.866107 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7aa645ea-62ea-493b-b863-9ea493f80c7b-console-serving-cert\") pod \"7aa645ea-62ea-493b-b863-9ea493f80c7b\" (UID: \"7aa645ea-62ea-493b-b863-9ea493f80c7b\") " Jul 09 15:04:21.866147 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:21.866133 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7aa645ea-62ea-493b-b863-9ea493f80c7b-console-oauth-config\") pod \"7aa645ea-62ea-493b-b863-9ea493f80c7b\" (UID: \"7aa645ea-62ea-493b-b863-9ea493f80c7b\") " Jul 09 15:04:21.866348 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:21.866167 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wdmqf\" (UniqueName: \"kubernetes.io/projected/7aa645ea-62ea-493b-b863-9ea493f80c7b-kube-api-access-wdmqf\") pod \"7aa645ea-62ea-493b-b863-9ea493f80c7b\" (UID: \"7aa645ea-62ea-493b-b863-9ea493f80c7b\") " Jul 09 15:04:21.866467 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:21.866437 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7aa645ea-62ea-493b-b863-9ea493f80c7b-service-ca" (OuterVolumeSpecName: "service-ca") pod "7aa645ea-62ea-493b-b863-9ea493f80c7b" (UID: "7aa645ea-62ea-493b-b863-9ea493f80c7b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:04:21.866528 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:21.866445 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7aa645ea-62ea-493b-b863-9ea493f80c7b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7aa645ea-62ea-493b-b863-9ea493f80c7b" (UID: "7aa645ea-62ea-493b-b863-9ea493f80c7b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:04:21.866578 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:21.866526 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7aa645ea-62ea-493b-b863-9ea493f80c7b-console-config" (OuterVolumeSpecName: "console-config") pod "7aa645ea-62ea-493b-b863-9ea493f80c7b" (UID: "7aa645ea-62ea-493b-b863-9ea493f80c7b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:04:21.868603 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:21.868578 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7aa645ea-62ea-493b-b863-9ea493f80c7b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7aa645ea-62ea-493b-b863-9ea493f80c7b" (UID: "7aa645ea-62ea-493b-b863-9ea493f80c7b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 15:04:21.868718 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:21.868656 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7aa645ea-62ea-493b-b863-9ea493f80c7b-kube-api-access-wdmqf" (OuterVolumeSpecName: "kube-api-access-wdmqf") pod "7aa645ea-62ea-493b-b863-9ea493f80c7b" (UID: "7aa645ea-62ea-493b-b863-9ea493f80c7b"). InnerVolumeSpecName "kube-api-access-wdmqf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:04:21.868718 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:21.868665 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7aa645ea-62ea-493b-b863-9ea493f80c7b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7aa645ea-62ea-493b-b863-9ea493f80c7b" (UID: "7aa645ea-62ea-493b-b863-9ea493f80c7b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 15:04:21.910084 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:21.910043 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:04:21.967742 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:21.967711 2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7aa645ea-62ea-493b-b863-9ea493f80c7b-service-ca\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:04:21.967742 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:21.967745 2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7aa645ea-62ea-493b-b863-9ea493f80c7b-oauth-serving-cert\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:04:21.967958 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:21.967760 2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7aa645ea-62ea-493b-b863-9ea493f80c7b-console-config\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:04:21.967958 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:21.967775 2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7aa645ea-62ea-493b-b863-9ea493f80c7b-console-serving-cert\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:04:21.967958 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:21.967787 2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7aa645ea-62ea-493b-b863-9ea493f80c7b-console-oauth-config\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:04:21.967958 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:21.967801 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wdmqf\" (UniqueName: \"kubernetes.io/projected/7aa645ea-62ea-493b-b863-9ea493f80c7b-kube-api-access-wdmqf\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:04:22.467702 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:22.467669 2567 generic.go:356] "Generic (PLEG): container finished" podID="7aa645ea-62ea-493b-b863-9ea493f80c7b" containerID="3f38b50fb88d55968150c901dcab613f320cec9abbd7de45aa0a01455d596ea5" exitCode=0 Jul 09 15:04:22.467976 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:22.467714 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b6cbb4b6d-qfx28" Jul 09 15:04:22.467976 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:22.467712 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b6cbb4b6d-qfx28" event={"ID":"7aa645ea-62ea-493b-b863-9ea493f80c7b","Type":"ContainerDied","Data":"3f38b50fb88d55968150c901dcab613f320cec9abbd7de45aa0a01455d596ea5"} Jul 09 15:04:22.467976 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:22.467873 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b6cbb4b6d-qfx28" event={"ID":"7aa645ea-62ea-493b-b863-9ea493f80c7b","Type":"ContainerDied","Data":"cfc3b16f977ce37d8c5e1d986adbfc99594578efd6a20d0c12901f9da033d5fb"} Jul 09 15:04:22.467976 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:22.467911 2567 scope.go:117] "RemoveContainer" containerID="3f38b50fb88d55968150c901dcab613f320cec9abbd7de45aa0a01455d596ea5" Jul 09 15:04:22.477275 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:22.477258 2567 scope.go:117] "RemoveContainer" containerID="3f38b50fb88d55968150c901dcab613f320cec9abbd7de45aa0a01455d596ea5" Jul 09 15:04:22.477555 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:04:22.477534 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f38b50fb88d55968150c901dcab613f320cec9abbd7de45aa0a01455d596ea5\": container with ID starting with 3f38b50fb88d55968150c901dcab613f320cec9abbd7de45aa0a01455d596ea5 not found: ID does not exist" containerID="3f38b50fb88d55968150c901dcab613f320cec9abbd7de45aa0a01455d596ea5" Jul 09 15:04:22.477596 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:22.477565 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f38b50fb88d55968150c901dcab613f320cec9abbd7de45aa0a01455d596ea5"} err="failed to get container status \"3f38b50fb88d55968150c901dcab613f320cec9abbd7de45aa0a01455d596ea5\": rpc error: code = NotFound desc = could not find container \"3f38b50fb88d55968150c901dcab613f320cec9abbd7de45aa0a01455d596ea5\": container with ID starting with 3f38b50fb88d55968150c901dcab613f320cec9abbd7de45aa0a01455d596ea5 not found: ID does not exist" Jul 09 15:04:22.491221 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:22.491193 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b6cbb4b6d-qfx28"] Jul 09 15:04:22.497968 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:22.497944 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5b6cbb4b6d-qfx28"] Jul 09 15:04:23.827444 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:23.827408 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7aa645ea-62ea-493b-b863-9ea493f80c7b" path="/var/lib/kubelet/pods/7aa645ea-62ea-493b-b863-9ea493f80c7b/volumes" Jul 09 15:04:25.262260 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:25.262223 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-96fg2" Jul 09 15:04:28.141495 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:28.141455 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-j4w6r_bc92e63d-1f89-4cf1-b209-a53b3f7687ff/serve-healthcheck-canary/0.log" Jul 09 15:04:29.415897 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:29.415859 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:04:29.515763 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:29.515718 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:04:29.588052 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:29.588015 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:04:32.382663 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:32.382600 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8796dfb78-gjmt4" podUID="bdaffe4c-36aa-4fee-b356-87897e214f9f" containerName="console" containerID="cri-o://ea04a8898fed2bbf78d1d10c4f0a94bfbfaaad0242d4dcf818d11cb8c36e9c84" gracePeriod=15 Jul 09 15:04:32.513162 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:32.513129 2567 generic.go:356] "Generic (PLEG): container finished" podID="bdaffe4c-36aa-4fee-b356-87897e214f9f" containerID="ea04a8898fed2bbf78d1d10c4f0a94bfbfaaad0242d4dcf818d11cb8c36e9c84" exitCode=0 Jul 09 15:04:32.513324 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:32.513176 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8796dfb78-gjmt4" event={"ID":"bdaffe4c-36aa-4fee-b356-87897e214f9f","Type":"ContainerDied","Data":"ea04a8898fed2bbf78d1d10c4f0a94bfbfaaad0242d4dcf818d11cb8c36e9c84"} Jul 09 15:04:32.681111 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:32.681079 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8796dfb78-gjmt4" Jul 09 15:04:32.761304 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:32.761266 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bdaffe4c-36aa-4fee-b356-87897e214f9f-console-config\") pod \"bdaffe4c-36aa-4fee-b356-87897e214f9f\" (UID: \"bdaffe4c-36aa-4fee-b356-87897e214f9f\") " Jul 09 15:04:32.761504 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:32.761315 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bdaffe4c-36aa-4fee-b356-87897e214f9f-trusted-ca-bundle\") pod \"bdaffe4c-36aa-4fee-b356-87897e214f9f\" (UID: \"bdaffe4c-36aa-4fee-b356-87897e214f9f\") " Jul 09 15:04:32.761504 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:32.761372 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bdaffe4c-36aa-4fee-b356-87897e214f9f-service-ca\") pod \"bdaffe4c-36aa-4fee-b356-87897e214f9f\" (UID: \"bdaffe4c-36aa-4fee-b356-87897e214f9f\") " Jul 09 15:04:32.761504 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:32.761422 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8n5vd\" (UniqueName: \"kubernetes.io/projected/bdaffe4c-36aa-4fee-b356-87897e214f9f-kube-api-access-8n5vd\") pod \"bdaffe4c-36aa-4fee-b356-87897e214f9f\" (UID: \"bdaffe4c-36aa-4fee-b356-87897e214f9f\") " Jul 09 15:04:32.761504 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:32.761469 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bdaffe4c-36aa-4fee-b356-87897e214f9f-console-oauth-config\") pod \"bdaffe4c-36aa-4fee-b356-87897e214f9f\" (UID: \"bdaffe4c-36aa-4fee-b356-87897e214f9f\") " Jul 09 15:04:32.761504 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:32.761493 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bdaffe4c-36aa-4fee-b356-87897e214f9f-oauth-serving-cert\") pod \"bdaffe4c-36aa-4fee-b356-87897e214f9f\" (UID: \"bdaffe4c-36aa-4fee-b356-87897e214f9f\") " Jul 09 15:04:32.761766 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:32.761520 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bdaffe4c-36aa-4fee-b356-87897e214f9f-console-serving-cert\") pod \"bdaffe4c-36aa-4fee-b356-87897e214f9f\" (UID: \"bdaffe4c-36aa-4fee-b356-87897e214f9f\") " Jul 09 15:04:32.761766 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:32.761729 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdaffe4c-36aa-4fee-b356-87897e214f9f-console-config" (OuterVolumeSpecName: "console-config") pod "bdaffe4c-36aa-4fee-b356-87897e214f9f" (UID: "bdaffe4c-36aa-4fee-b356-87897e214f9f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:04:32.761913 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:32.761782 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdaffe4c-36aa-4fee-b356-87897e214f9f-service-ca" (OuterVolumeSpecName: "service-ca") pod "bdaffe4c-36aa-4fee-b356-87897e214f9f" (UID: "bdaffe4c-36aa-4fee-b356-87897e214f9f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:04:32.761913 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:32.761797 2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bdaffe4c-36aa-4fee-b356-87897e214f9f-console-config\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:04:32.761913 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:32.761874 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdaffe4c-36aa-4fee-b356-87897e214f9f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "bdaffe4c-36aa-4fee-b356-87897e214f9f" (UID: "bdaffe4c-36aa-4fee-b356-87897e214f9f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:04:32.762317 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:32.762283 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdaffe4c-36aa-4fee-b356-87897e214f9f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bdaffe4c-36aa-4fee-b356-87897e214f9f" (UID: "bdaffe4c-36aa-4fee-b356-87897e214f9f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:04:32.763986 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:32.763959 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdaffe4c-36aa-4fee-b356-87897e214f9f-kube-api-access-8n5vd" (OuterVolumeSpecName: "kube-api-access-8n5vd") pod "bdaffe4c-36aa-4fee-b356-87897e214f9f" (UID: "bdaffe4c-36aa-4fee-b356-87897e214f9f"). InnerVolumeSpecName "kube-api-access-8n5vd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:04:32.763986 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:32.763966 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bdaffe4c-36aa-4fee-b356-87897e214f9f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bdaffe4c-36aa-4fee-b356-87897e214f9f" (UID: "bdaffe4c-36aa-4fee-b356-87897e214f9f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 15:04:32.764106 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:32.764000 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bdaffe4c-36aa-4fee-b356-87897e214f9f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bdaffe4c-36aa-4fee-b356-87897e214f9f" (UID: "bdaffe4c-36aa-4fee-b356-87897e214f9f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 15:04:32.862686 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:32.862647 2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bdaffe4c-36aa-4fee-b356-87897e214f9f-service-ca\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:04:32.862686 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:32.862682 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8n5vd\" (UniqueName: \"kubernetes.io/projected/bdaffe4c-36aa-4fee-b356-87897e214f9f-kube-api-access-8n5vd\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:04:32.862686 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:32.862693 2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bdaffe4c-36aa-4fee-b356-87897e214f9f-console-oauth-config\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:04:32.862952 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:32.862702 2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bdaffe4c-36aa-4fee-b356-87897e214f9f-oauth-serving-cert\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:04:32.862952 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:32.862711 2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bdaffe4c-36aa-4fee-b356-87897e214f9f-console-serving-cert\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:04:32.862952 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:32.862719 2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bdaffe4c-36aa-4fee-b356-87897e214f9f-trusted-ca-bundle\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:04:33.518913 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:33.518881 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8796dfb78-gjmt4" Jul 09 15:04:33.519373 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:33.518881 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8796dfb78-gjmt4" event={"ID":"bdaffe4c-36aa-4fee-b356-87897e214f9f","Type":"ContainerDied","Data":"b52990795b3905d1dc1924d8492a65f0ca158ae90a5c805a4309b0db19599d8e"} Jul 09 15:04:33.519373 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:33.519016 2567 scope.go:117] "RemoveContainer" containerID="ea04a8898fed2bbf78d1d10c4f0a94bfbfaaad0242d4dcf818d11cb8c36e9c84" Jul 09 15:04:33.543492 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:33.543432 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8796dfb78-gjmt4"] Jul 09 15:04:33.545263 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:33.545240 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8796dfb78-gjmt4"] Jul 09 15:04:33.827506 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:33.827425 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bdaffe4c-36aa-4fee-b356-87897e214f9f" path="/var/lib/kubelet/pods/bdaffe4c-36aa-4fee-b356-87897e214f9f/volumes" Jul 09 15:04:33.917964 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:33.917931 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-9bbc5d8d4-gxffh" Jul 09 15:04:33.921950 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:33.921930 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-9bbc5d8d4-gxffh" Jul 09 15:04:41.730167 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:41.730058 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:04:41.805211 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:41.805175 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:04:55.667296 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:55.667247 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:04:55.687100 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:55.687070 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:04:56.644452 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:04:56.644418 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:08.398983 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:08.398940 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:05:08.704146 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:08.704102 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:05:09.480399 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:09.480363 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 15:05:09.480846 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:09.480641 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" containerName="alertmanager" containerID="cri-o://4070cd8bc849760dbe2a53ad0c966bfd29f6c481dec96892fcc7918df349472d" gracePeriod=120 Jul 09 15:05:09.480846 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:09.480715 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" containerName="kube-rbac-proxy-metric" containerID="cri-o://41f359a20be86f806772da451ea66e1fbba3a3e2910938cde98df2a429d00f9f" gracePeriod=120 Jul 09 15:05:09.480846 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:09.480749 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" containerName="prom-label-proxy" containerID="cri-o://757a4ca3a9b85773dcf11575d67a81aae250b4a143ea42308a058cf77ecc9630" gracePeriod=120 Jul 09 15:05:09.480846 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:09.480787 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" containerName="kube-rbac-proxy-web" containerID="cri-o://8ec7d5f0ca5c7113e272d21ee0d55c69bb1a3ffb545d00a1f10839066d3f603e" gracePeriod=120 Jul 09 15:05:09.480846 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:09.480807 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" containerName="config-reloader" containerID="cri-o://6145f817573859661bd28e727b431456427c4fb81842582a4178a06a1ed496a6" gracePeriod=120 Jul 09 15:05:09.481131 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:09.480788 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" containerName="kube-rbac-proxy" containerID="cri-o://54fcd7d8284d138f8f2cce1b41101a6901aaba25ee3fee9e4a83efa405a8af92" gracePeriod=120 Jul 09 15:05:09.685042 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:09.685008 2567 generic.go:356] "Generic (PLEG): container finished" podID="c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" containerID="757a4ca3a9b85773dcf11575d67a81aae250b4a143ea42308a058cf77ecc9630" exitCode=0 Jul 09 15:05:09.685042 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:09.685036 2567 generic.go:356] "Generic (PLEG): container finished" podID="c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" containerID="41f359a20be86f806772da451ea66e1fbba3a3e2910938cde98df2a429d00f9f" exitCode=0 Jul 09 15:05:09.685042 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:09.685042 2567 generic.go:356] "Generic (PLEG): container finished" podID="c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" containerID="54fcd7d8284d138f8f2cce1b41101a6901aaba25ee3fee9e4a83efa405a8af92" exitCode=0 Jul 09 15:05:09.685042 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:09.685048 2567 generic.go:356] "Generic (PLEG): container finished" podID="c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" containerID="6145f817573859661bd28e727b431456427c4fb81842582a4178a06a1ed496a6" exitCode=0 Jul 09 15:05:09.685310 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:09.685054 2567 generic.go:356] "Generic (PLEG): container finished" podID="c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" containerID="4070cd8bc849760dbe2a53ad0c966bfd29f6c481dec96892fcc7918df349472d" exitCode=0 Jul 09 15:05:09.685310 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:09.685079 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1","Type":"ContainerDied","Data":"757a4ca3a9b85773dcf11575d67a81aae250b4a143ea42308a058cf77ecc9630"} Jul 09 15:05:09.685310 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:09.685189 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1","Type":"ContainerDied","Data":"41f359a20be86f806772da451ea66e1fbba3a3e2910938cde98df2a429d00f9f"} Jul 09 15:05:09.685310 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:09.685231 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1","Type":"ContainerDied","Data":"54fcd7d8284d138f8f2cce1b41101a6901aaba25ee3fee9e4a83efa405a8af92"} Jul 09 15:05:09.685310 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:09.685260 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1","Type":"ContainerDied","Data":"6145f817573859661bd28e727b431456427c4fb81842582a4178a06a1ed496a6"} Jul 09 15:05:09.685310 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:09.685284 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1","Type":"ContainerDied","Data":"4070cd8bc849760dbe2a53ad0c966bfd29f6c481dec96892fcc7918df349472d"} Jul 09 15:05:10.693658 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.693620 2567 generic.go:356] "Generic (PLEG): container finished" podID="c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" containerID="8ec7d5f0ca5c7113e272d21ee0d55c69bb1a3ffb545d00a1f10839066d3f603e" exitCode=0 Jul 09 15:05:10.693658 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.693659 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1","Type":"ContainerDied","Data":"8ec7d5f0ca5c7113e272d21ee0d55c69bb1a3ffb545d00a1f10839066d3f603e"} Jul 09 15:05:10.730617 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.730591 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:10.790558 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.790528 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-alertmanager-main-db\") pod \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " Jul 09 15:05:10.790735 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.790569 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-tls-assets\") pod \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " Jul 09 15:05:10.790735 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.790591 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-alertmanager-trusted-ca-bundle\") pod \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " Jul 09 15:05:10.790735 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.790612 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hppgg\" (UniqueName: \"kubernetes.io/projected/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-kube-api-access-hppgg\") pod \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " Jul 09 15:05:10.790735 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.790641 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-config-out\") pod \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " Jul 09 15:05:10.790735 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.790701 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-cluster-tls-config\") pod \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " Jul 09 15:05:10.790735 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.790731 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-secret-alertmanager-main-tls\") pod \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " Jul 09 15:05:10.791090 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.790772 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-secret-alertmanager-kube-rbac-proxy\") pod \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " Jul 09 15:05:10.791090 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.790807 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-metrics-client-ca\") pod \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " Jul 09 15:05:10.791090 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.790852 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-secret-alertmanager-kube-rbac-proxy-web\") pod \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " Jul 09 15:05:10.791090 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.790879 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-web-config\") pod \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " Jul 09 15:05:10.791090 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.790904 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-config-volume\") pod \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " Jul 09 15:05:10.791090 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.790937 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\" (UID: \"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1\") " Jul 09 15:05:10.791090 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.790931 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" (UID: "c0cbd21d-6e9f-4fd3-8500-9e907f3075e1"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:05:10.791090 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.791036 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" (UID: "c0cbd21d-6e9f-4fd3-8500-9e907f3075e1"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:05:10.791547 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.791305 2567 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-alertmanager-main-db\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:05:10.791547 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.791326 2567 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:05:10.791547 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.791342 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" (UID: "c0cbd21d-6e9f-4fd3-8500-9e907f3075e1"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:05:10.794089 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.794058 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" (UID: "c0cbd21d-6e9f-4fd3-8500-9e907f3075e1"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 15:05:10.794608 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.794344 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-config-out" (OuterVolumeSpecName: "config-out") pod "c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" (UID: "c0cbd21d-6e9f-4fd3-8500-9e907f3075e1"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:05:10.794608 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.794385 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-kube-api-access-hppgg" (OuterVolumeSpecName: "kube-api-access-hppgg") pod "c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" (UID: "c0cbd21d-6e9f-4fd3-8500-9e907f3075e1"). InnerVolumeSpecName "kube-api-access-hppgg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:05:10.794898 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.794879 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-config-volume" (OuterVolumeSpecName: "config-volume") pod "c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" (UID: "c0cbd21d-6e9f-4fd3-8500-9e907f3075e1"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 15:05:10.794991 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.794968 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" (UID: "c0cbd21d-6e9f-4fd3-8500-9e907f3075e1"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:05:10.795083 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.795064 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" (UID: "c0cbd21d-6e9f-4fd3-8500-9e907f3075e1"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 15:05:10.795189 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.795160 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" (UID: "c0cbd21d-6e9f-4fd3-8500-9e907f3075e1"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 15:05:10.796471 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.796441 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" (UID: "c0cbd21d-6e9f-4fd3-8500-9e907f3075e1"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 15:05:10.800455 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.800334 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" (UID: "c0cbd21d-6e9f-4fd3-8500-9e907f3075e1"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 15:05:10.807373 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.807337 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-web-config" (OuterVolumeSpecName: "web-config") pod "c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" (UID: "c0cbd21d-6e9f-4fd3-8500-9e907f3075e1"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 15:05:10.891793 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.891756 2567 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:05:10.891793 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.891785 2567 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-metrics-client-ca\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:05:10.891793 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.891795 2567 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:05:10.891793 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.891804 2567 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-web-config\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:05:10.892061 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.891835 2567 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-config-volume\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:05:10.892061 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.891845 2567 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:05:10.892061 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.891853 2567 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-tls-assets\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:05:10.892061 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.891863 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hppgg\" (UniqueName: \"kubernetes.io/projected/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-kube-api-access-hppgg\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:05:10.892061 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.891872 2567 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-config-out\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:05:10.892061 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.891879 2567 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-cluster-tls-config\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:05:10.892061 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:10.891887 2567 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1-secret-alertmanager-main-tls\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:05:11.701005 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.700968 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0cbd21d-6e9f-4fd3-8500-9e907f3075e1","Type":"ContainerDied","Data":"654fb25a4ad0764ca9de707648f2bfceb92a32ce110b564a5dbea6bf4901ad84"} Jul 09 15:05:11.701005 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.701009 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.701492 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.701058 2567 scope.go:117] "RemoveContainer" containerID="757a4ca3a9b85773dcf11575d67a81aae250b4a143ea42308a058cf77ecc9630" Jul 09 15:05:11.709911 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.709894 2567 scope.go:117] "RemoveContainer" containerID="41f359a20be86f806772da451ea66e1fbba3a3e2910938cde98df2a429d00f9f" Jul 09 15:05:11.717661 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.717641 2567 scope.go:117] "RemoveContainer" containerID="54fcd7d8284d138f8f2cce1b41101a6901aaba25ee3fee9e4a83efa405a8af92" Jul 09 15:05:11.725162 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.725005 2567 scope.go:117] "RemoveContainer" containerID="8ec7d5f0ca5c7113e272d21ee0d55c69bb1a3ffb545d00a1f10839066d3f603e" Jul 09 15:05:11.725993 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.725971 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 15:05:11.731188 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.731163 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 15:05:11.733241 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.733218 2567 scope.go:117] "RemoveContainer" containerID="6145f817573859661bd28e727b431456427c4fb81842582a4178a06a1ed496a6" Jul 09 15:05:11.740498 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.740470 2567 scope.go:117] "RemoveContainer" containerID="4070cd8bc849760dbe2a53ad0c966bfd29f6c481dec96892fcc7918df349472d" Jul 09 15:05:11.747684 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.747668 2567 scope.go:117] "RemoveContainer" containerID="ba2aa5b3e3407de69a39d4e006a5fab75c4457c7aff253d201dc8fcf268f0020" Jul 09 15:05:11.763926 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.763899 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 15:05:11.764243 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.764230 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" containerName="alertmanager" Jul 09 15:05:11.764285 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.764245 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" containerName="alertmanager" Jul 09 15:05:11.764285 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.764253 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" containerName="kube-rbac-proxy-web" Jul 09 15:05:11.764285 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.764259 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" containerName="kube-rbac-proxy-web" Jul 09 15:05:11.764285 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.764268 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" containerName="init-config-reloader" Jul 09 15:05:11.764285 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.764273 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" containerName="init-config-reloader" Jul 09 15:05:11.764285 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.764281 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" containerName="kube-rbac-proxy-metric" Jul 09 15:05:11.764285 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.764286 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" containerName="kube-rbac-proxy-metric" Jul 09 15:05:11.764482 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.764292 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdaffe4c-36aa-4fee-b356-87897e214f9f" containerName="console" Jul 09 15:05:11.764482 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.764299 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdaffe4c-36aa-4fee-b356-87897e214f9f" containerName="console" Jul 09 15:05:11.764482 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.764315 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" containerName="prom-label-proxy" Jul 09 15:05:11.764482 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.764321 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" containerName="prom-label-proxy" Jul 09 15:05:11.764482 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.764328 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7aa645ea-62ea-493b-b863-9ea493f80c7b" containerName="console" Jul 09 15:05:11.764482 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.764333 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7aa645ea-62ea-493b-b863-9ea493f80c7b" containerName="console" Jul 09 15:05:11.764482 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.764341 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" containerName="kube-rbac-proxy" Jul 09 15:05:11.764482 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.764347 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" containerName="kube-rbac-proxy" Jul 09 15:05:11.764482 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.764353 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" containerName="config-reloader" Jul 09 15:05:11.764482 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.764358 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" containerName="config-reloader" Jul 09 15:05:11.764482 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.764414 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7aa645ea-62ea-493b-b863-9ea493f80c7b" containerName="console" Jul 09 15:05:11.764482 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.764421 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" containerName="kube-rbac-proxy" Jul 09 15:05:11.764482 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.764427 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" containerName="kube-rbac-proxy-web" Jul 09 15:05:11.764482 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.764433 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" containerName="kube-rbac-proxy-metric" Jul 09 15:05:11.764482 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.764439 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" containerName="prom-label-proxy" Jul 09 15:05:11.764482 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.764447 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdaffe4c-36aa-4fee-b356-87897e214f9f" containerName="console" Jul 09 15:05:11.764482 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.764452 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" containerName="alertmanager" Jul 09 15:05:11.764482 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.764458 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" containerName="config-reloader" Jul 09 15:05:11.769684 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.769664 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.772557 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.772491 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 09 15:05:11.772557 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.772504 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 09 15:05:11.772557 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.772513 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 09 15:05:11.772557 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.772546 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 09 15:05:11.772811 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.772622 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 09 15:05:11.772911 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.772895 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-lf8nl\"" Jul 09 15:05:11.772969 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.772922 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 09 15:05:11.772969 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.772928 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 09 15:05:11.772969 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.772928 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 09 15:05:11.779901 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.779881 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 09 15:05:11.784647 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.784621 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 15:05:11.799130 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.799055 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.799130 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.799093 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.799130 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.799119 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.799307 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.799191 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.799307 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.799234 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.799307 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.799285 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gb2sw\" (UniqueName: \"kubernetes.io/projected/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-kube-api-access-gb2sw\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.799415 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.799317 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.799415 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.799347 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.799415 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.799377 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.799415 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.799410 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-web-config\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.799555 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.799450 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-config-volume\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.799555 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.799509 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.799555 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.799537 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-config-out\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.828168 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.828134 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0cbd21d-6e9f-4fd3-8500-9e907f3075e1" path="/var/lib/kubelet/pods/c0cbd21d-6e9f-4fd3-8500-9e907f3075e1/volumes" Jul 09 15:05:11.900848 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.900784 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.900992 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.900856 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.900992 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.900896 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.900992 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.900922 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-web-config\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.900992 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.900937 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-config-volume\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.901249 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.901222 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.901317 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.901258 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.901317 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.901275 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-config-out\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.901426 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.901322 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.901426 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.901360 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.901426 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.901390 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.901567 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.901444 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.901567 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.901506 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.901567 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.901551 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gb2sw\" (UniqueName: \"kubernetes.io/projected/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-kube-api-access-gb2sw\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.902018 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.901988 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.902556 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.902399 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.904016 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.903986 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-web-config\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.904209 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.904162 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-config-out\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.904894 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.904450 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.904894 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.904481 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.904894 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.904846 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.905101 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.904962 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.905101 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.905035 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-config-volume\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.905101 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.905068 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.906098 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.905952 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:11.913460 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:11.913437 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gb2sw\" (UniqueName: \"kubernetes.io/projected/f708c85c-e7da-4a8d-ba05-7a4c442fb6f5-kube-api-access-gb2sw\") pod \"alertmanager-main-0\" (UID: \"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:12.079735 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:12.079640 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 15:05:12.219871 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:12.219844 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 15:05:12.222622 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:05:12.222599 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf708c85c_e7da_4a8d_ba05_7a4c442fb6f5.slice/crio-85ae7a62aa1f65515e2a069b84122382b4803231f4fad205babd48eb4dff746d WatchSource:0}: Error finding container 85ae7a62aa1f65515e2a069b84122382b4803231f4fad205babd48eb4dff746d: Status 404 returned error can't find the container with id 85ae7a62aa1f65515e2a069b84122382b4803231f4fad205babd48eb4dff746d Jul 09 15:05:12.707334 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:12.707301 2567 generic.go:356] "Generic (PLEG): container finished" podID="f708c85c-e7da-4a8d-ba05-7a4c442fb6f5" containerID="9d4138bc9e295551ceda79462e12478576a05753d800cb6785976b5cf1a8f070" exitCode=0 Jul 09 15:05:12.707334 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:12.707339 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5","Type":"ContainerDied","Data":"9d4138bc9e295551ceda79462e12478576a05753d800cb6785976b5cf1a8f070"} Jul 09 15:05:12.707738 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:12.707388 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5","Type":"ContainerStarted","Data":"85ae7a62aa1f65515e2a069b84122382b4803231f4fad205babd48eb4dff746d"} Jul 09 15:05:13.502610 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.502571 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-c6854b798-q486f"] Jul 09 15:05:13.505486 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.505463 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-c6854b798-q486f" Jul 09 15:05:13.508094 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.508071 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 09 15:05:13.508419 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.508336 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 09 15:05:13.508645 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.508627 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 09 15:05:13.508982 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.508865 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 09 15:05:13.508982 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.508888 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 09 15:05:13.509454 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.509430 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-bc49j\"" Jul 09 15:05:13.513650 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.513628 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 09 15:05:13.521302 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.521274 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-c6854b798-q486f"] Jul 09 15:05:13.616022 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.615969 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49820873-b1af-45a8-aed3-24c2d227f933-metrics-client-ca\") pod \"telemeter-client-c6854b798-q486f\" (UID: \"49820873-b1af-45a8-aed3-24c2d227f933\") " pod="openshift-monitoring/telemeter-client-c6854b798-q486f" Jul 09 15:05:13.616022 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.616022 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftwfh\" (UniqueName: \"kubernetes.io/projected/49820873-b1af-45a8-aed3-24c2d227f933-kube-api-access-ftwfh\") pod \"telemeter-client-c6854b798-q486f\" (UID: \"49820873-b1af-45a8-aed3-24c2d227f933\") " pod="openshift-monitoring/telemeter-client-c6854b798-q486f" Jul 09 15:05:13.616238 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.616127 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/49820873-b1af-45a8-aed3-24c2d227f933-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-c6854b798-q486f\" (UID: \"49820873-b1af-45a8-aed3-24c2d227f933\") " pod="openshift-monitoring/telemeter-client-c6854b798-q486f" Jul 09 15:05:13.616238 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.616186 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/49820873-b1af-45a8-aed3-24c2d227f933-telemeter-client-tls\") pod \"telemeter-client-c6854b798-q486f\" (UID: \"49820873-b1af-45a8-aed3-24c2d227f933\") " pod="openshift-monitoring/telemeter-client-c6854b798-q486f" Jul 09 15:05:13.616305 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.616221 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49820873-b1af-45a8-aed3-24c2d227f933-telemeter-trusted-ca-bundle\") pod \"telemeter-client-c6854b798-q486f\" (UID: \"49820873-b1af-45a8-aed3-24c2d227f933\") " pod="openshift-monitoring/telemeter-client-c6854b798-q486f" Jul 09 15:05:13.616347 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.616302 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/49820873-b1af-45a8-aed3-24c2d227f933-federate-client-tls\") pod \"telemeter-client-c6854b798-q486f\" (UID: \"49820873-b1af-45a8-aed3-24c2d227f933\") " pod="openshift-monitoring/telemeter-client-c6854b798-q486f" Jul 09 15:05:13.616347 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.616338 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/49820873-b1af-45a8-aed3-24c2d227f933-secret-telemeter-client\") pod \"telemeter-client-c6854b798-q486f\" (UID: \"49820873-b1af-45a8-aed3-24c2d227f933\") " pod="openshift-monitoring/telemeter-client-c6854b798-q486f" Jul 09 15:05:13.616430 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.616377 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49820873-b1af-45a8-aed3-24c2d227f933-serving-certs-ca-bundle\") pod \"telemeter-client-c6854b798-q486f\" (UID: \"49820873-b1af-45a8-aed3-24c2d227f933\") " pod="openshift-monitoring/telemeter-client-c6854b798-q486f" Jul 09 15:05:13.716446 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.716398 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5","Type":"ContainerStarted","Data":"1361b3afa68f7bd82dd33b02442637afdd653d80883094aa798afd4f5826427c"} Jul 09 15:05:13.716929 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.716490 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5","Type":"ContainerStarted","Data":"3deea21dd5d0624e1bafc5d2bfee65f0d31921dd7b727110f5240ddcfc3b55f8"} Jul 09 15:05:13.716929 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.716541 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5","Type":"ContainerStarted","Data":"c80ed61948884309ddbfc8f230f3e39786d7f0e8bf67ea06df29ba3fb4634564"} Jul 09 15:05:13.716929 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.716585 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5","Type":"ContainerStarted","Data":"6aad358eb5614cf64f7ed7e292867c1396159c27c8e45b67b0c8d43d2ba27cdd"} Jul 09 15:05:13.716929 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.716630 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5","Type":"ContainerStarted","Data":"ab6b5980844c66975fdcb7559d64f9b1e7a0aab5d2d7598c339108026cc8c8bf"} Jul 09 15:05:13.716929 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.716674 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f708c85c-e7da-4a8d-ba05-7a4c442fb6f5","Type":"ContainerStarted","Data":"2405297a5d4fd783a01466b1b3d5ef4db1d3ef9e7dc57f2595a764aa0893b19b"} Jul 09 15:05:13.716929 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.716841 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49820873-b1af-45a8-aed3-24c2d227f933-serving-certs-ca-bundle\") pod \"telemeter-client-c6854b798-q486f\" (UID: \"49820873-b1af-45a8-aed3-24c2d227f933\") " pod="openshift-monitoring/telemeter-client-c6854b798-q486f" Jul 09 15:05:13.716929 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.716890 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49820873-b1af-45a8-aed3-24c2d227f933-metrics-client-ca\") pod \"telemeter-client-c6854b798-q486f\" (UID: \"49820873-b1af-45a8-aed3-24c2d227f933\") " pod="openshift-monitoring/telemeter-client-c6854b798-q486f" Jul 09 15:05:13.716929 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.716918 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftwfh\" (UniqueName: \"kubernetes.io/projected/49820873-b1af-45a8-aed3-24c2d227f933-kube-api-access-ftwfh\") pod \"telemeter-client-c6854b798-q486f\" (UID: \"49820873-b1af-45a8-aed3-24c2d227f933\") " pod="openshift-monitoring/telemeter-client-c6854b798-q486f" Jul 09 15:05:13.717301 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.716955 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/49820873-b1af-45a8-aed3-24c2d227f933-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-c6854b798-q486f\" (UID: \"49820873-b1af-45a8-aed3-24c2d227f933\") " pod="openshift-monitoring/telemeter-client-c6854b798-q486f" Jul 09 15:05:13.717301 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.716980 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/49820873-b1af-45a8-aed3-24c2d227f933-telemeter-client-tls\") pod \"telemeter-client-c6854b798-q486f\" (UID: \"49820873-b1af-45a8-aed3-24c2d227f933\") " pod="openshift-monitoring/telemeter-client-c6854b798-q486f" Jul 09 15:05:13.717301 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.717007 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49820873-b1af-45a8-aed3-24c2d227f933-telemeter-trusted-ca-bundle\") pod \"telemeter-client-c6854b798-q486f\" (UID: \"49820873-b1af-45a8-aed3-24c2d227f933\") " pod="openshift-monitoring/telemeter-client-c6854b798-q486f" Jul 09 15:05:13.717301 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.717052 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/49820873-b1af-45a8-aed3-24c2d227f933-federate-client-tls\") pod \"telemeter-client-c6854b798-q486f\" (UID: \"49820873-b1af-45a8-aed3-24c2d227f933\") " pod="openshift-monitoring/telemeter-client-c6854b798-q486f" Jul 09 15:05:13.717301 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.717081 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/49820873-b1af-45a8-aed3-24c2d227f933-secret-telemeter-client\") pod \"telemeter-client-c6854b798-q486f\" (UID: \"49820873-b1af-45a8-aed3-24c2d227f933\") " pod="openshift-monitoring/telemeter-client-c6854b798-q486f" Jul 09 15:05:13.717701 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.717665 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49820873-b1af-45a8-aed3-24c2d227f933-metrics-client-ca\") pod \"telemeter-client-c6854b798-q486f\" (UID: \"49820873-b1af-45a8-aed3-24c2d227f933\") " pod="openshift-monitoring/telemeter-client-c6854b798-q486f" Jul 09 15:05:13.717918 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.717689 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49820873-b1af-45a8-aed3-24c2d227f933-serving-certs-ca-bundle\") pod \"telemeter-client-c6854b798-q486f\" (UID: \"49820873-b1af-45a8-aed3-24c2d227f933\") " pod="openshift-monitoring/telemeter-client-c6854b798-q486f" Jul 09 15:05:13.718189 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.718167 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49820873-b1af-45a8-aed3-24c2d227f933-telemeter-trusted-ca-bundle\") pod \"telemeter-client-c6854b798-q486f\" (UID: \"49820873-b1af-45a8-aed3-24c2d227f933\") " pod="openshift-monitoring/telemeter-client-c6854b798-q486f" Jul 09 15:05:13.721882 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.720021 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/49820873-b1af-45a8-aed3-24c2d227f933-telemeter-client-tls\") pod \"telemeter-client-c6854b798-q486f\" (UID: \"49820873-b1af-45a8-aed3-24c2d227f933\") " pod="openshift-monitoring/telemeter-client-c6854b798-q486f" Jul 09 15:05:13.721882 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.720085 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/49820873-b1af-45a8-aed3-24c2d227f933-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-c6854b798-q486f\" (UID: \"49820873-b1af-45a8-aed3-24c2d227f933\") " pod="openshift-monitoring/telemeter-client-c6854b798-q486f" Jul 09 15:05:13.721882 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.720185 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/49820873-b1af-45a8-aed3-24c2d227f933-secret-telemeter-client\") pod \"telemeter-client-c6854b798-q486f\" (UID: \"49820873-b1af-45a8-aed3-24c2d227f933\") " pod="openshift-monitoring/telemeter-client-c6854b798-q486f" Jul 09 15:05:13.721882 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.720575 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/49820873-b1af-45a8-aed3-24c2d227f933-federate-client-tls\") pod \"telemeter-client-c6854b798-q486f\" (UID: \"49820873-b1af-45a8-aed3-24c2d227f933\") " pod="openshift-monitoring/telemeter-client-c6854b798-q486f" Jul 09 15:05:13.725594 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.725571 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftwfh\" (UniqueName: \"kubernetes.io/projected/49820873-b1af-45a8-aed3-24c2d227f933-kube-api-access-ftwfh\") pod \"telemeter-client-c6854b798-q486f\" (UID: \"49820873-b1af-45a8-aed3-24c2d227f933\") " pod="openshift-monitoring/telemeter-client-c6854b798-q486f" Jul 09 15:05:13.750032 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.749971 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.733213488 podStartE2EDuration="2.749950405s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.736917ms" isStatefulPod=false podCreationTimestamp="2026-07-09 15:05:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:05:13.747537665 +0000 UTC m=+160.478044199" watchObservedRunningTime="2026-07-09 15:05:13.749950405 +0000 UTC m=+160.480456897" Jul 09 15:05:13.816915 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.813153 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 15:05:13.816915 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.813721 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d6a54937-6cfe-4b4e-9860-0af0729aea4a" containerName="prometheus" containerID="cri-o://d0e271cb64e3315e8c12cd01d778dc04ae85a8fb65e2889d66971c5b00ee3d74" gracePeriod=600 Jul 09 15:05:13.816915 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.814211 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d6a54937-6cfe-4b4e-9860-0af0729aea4a" containerName="kube-rbac-proxy-thanos" containerID="cri-o://d0756f81f1540b764bb51da87cea08cc6efe224344f67388d8ba76023fa5ea53" gracePeriod=600 Jul 09 15:05:13.816915 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.814339 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d6a54937-6cfe-4b4e-9860-0af0729aea4a" containerName="kube-rbac-proxy" containerID="cri-o://44cb2dc4d500b79fcaa558c7b96a7f4d6b1bde902dbb9ccacb4e4afd7981f777" gracePeriod=600 Jul 09 15:05:13.816915 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.814439 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d6a54937-6cfe-4b4e-9860-0af0729aea4a" containerName="kube-rbac-proxy-web" containerID="cri-o://3a1408520418153f2025107bfabbd9d57db7916201bd2878b0d02b8ed05bcb94" gracePeriod=600 Jul 09 15:05:13.816915 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.814528 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d6a54937-6cfe-4b4e-9860-0af0729aea4a" containerName="thanos-sidecar" containerID="cri-o://971b2617db0073606194ffc9c8d782ba2226b9e756e10d4319b2b2a24de7ec20" gracePeriod=600 Jul 09 15:05:13.816915 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.814620 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d6a54937-6cfe-4b4e-9860-0af0729aea4a" containerName="config-reloader" containerID="cri-o://44d6e12a47ea4ebeb587ec6be072bbbeb0119090d98efe643ebb9eb909b131e9" gracePeriod=600 Jul 09 15:05:13.818046 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:13.818001 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-c6854b798-q486f" Jul 09 15:05:14.013655 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.013629 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-c6854b798-q486f"] Jul 09 15:05:14.015516 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:05:14.015492 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49820873_b1af_45a8_aed3_24c2d227f933.slice/crio-6d9e1c827d571ee8067ed5a23a4df760fc4c88616bab282a6a67320c309125e1 WatchSource:0}: Error finding container 6d9e1c827d571ee8067ed5a23a4df760fc4c88616bab282a6a67320c309125e1: Status 404 returned error can't find the container with id 6d9e1c827d571ee8067ed5a23a4df760fc4c88616bab282a6a67320c309125e1 Jul 09 15:05:14.093496 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.093464 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.125684 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.125643 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " Jul 09 15:05:14.125915 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.125745 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d6a54937-6cfe-4b4e-9860-0af0729aea4a-config-out\") pod \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " Jul 09 15:05:14.125915 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.125775 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-prometheus-trusted-ca-bundle\") pod \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " Jul 09 15:05:14.125915 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.125891 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-configmap-serving-certs-ca-bundle\") pod \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " Jul 09 15:05:14.126105 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.125950 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-secret-prometheus-k8s-tls\") pod \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " Jul 09 15:05:14.126105 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.125986 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-secret-metrics-client-certs\") pod \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " Jul 09 15:05:14.126105 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.126020 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-configmap-kubelet-serving-ca-bundle\") pod \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " Jul 09 15:05:14.126105 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.126053 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-prometheus-k8s-rulefiles-1\") pod \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " Jul 09 15:05:14.126105 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.126088 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-prometheus-k8s-rulefiles-0\") pod \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " Jul 09 15:05:14.126346 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.126120 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d6a54937-6cfe-4b4e-9860-0af0729aea4a-tls-assets\") pod \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " Jul 09 15:05:14.126346 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.126163 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-prometheus-k8s-rulefiles-2\") pod \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " Jul 09 15:05:14.126346 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.126194 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " Jul 09 15:05:14.126346 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.126204 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "d6a54937-6cfe-4b4e-9860-0af0729aea4a" (UID: "d6a54937-6cfe-4b4e-9860-0af0729aea4a"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:05:14.126550 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.126418 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "d6a54937-6cfe-4b4e-9860-0af0729aea4a" (UID: "d6a54937-6cfe-4b4e-9860-0af0729aea4a"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:05:14.127352 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.126661 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "d6a54937-6cfe-4b4e-9860-0af0729aea4a" (UID: "d6a54937-6cfe-4b4e-9860-0af0729aea4a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:05:14.127352 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.126998 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "d6a54937-6cfe-4b4e-9860-0af0729aea4a" (UID: "d6a54937-6cfe-4b4e-9860-0af0729aea4a"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:05:14.127543 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.127451 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "d6a54937-6cfe-4b4e-9860-0af0729aea4a" (UID: "d6a54937-6cfe-4b4e-9860-0af0729aea4a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:05:14.127609 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.127555 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "d6a54937-6cfe-4b4e-9860-0af0729aea4a" (UID: "d6a54937-6cfe-4b4e-9860-0af0729aea4a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:05:14.127759 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.127738 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5w62l\" (UniqueName: \"kubernetes.io/projected/d6a54937-6cfe-4b4e-9860-0af0729aea4a-kube-api-access-5w62l\") pod \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " Jul 09 15:05:14.127941 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.127922 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-configmap-metrics-client-ca\") pod \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " Jul 09 15:05:14.128069 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.128055 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-secret-kube-rbac-proxy\") pod \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " Jul 09 15:05:14.128240 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.128223 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d6a54937-6cfe-4b4e-9860-0af0729aea4a-prometheus-k8s-db\") pod \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " Jul 09 15:05:14.130012 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.129579 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-web-config\") pod \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " Jul 09 15:05:14.130012 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.129619 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-secret-grpc-tls\") pod \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " Jul 09 15:05:14.130012 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.129666 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-config\") pod \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " Jul 09 15:05:14.130012 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.129705 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-thanos-prometheus-http-client-file\") pod \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\" (UID: \"d6a54937-6cfe-4b4e-9860-0af0729aea4a\") " Jul 09 15:05:14.130274 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.130174 2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-prometheus-trusted-ca-bundle\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:05:14.130274 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.128733 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "d6a54937-6cfe-4b4e-9860-0af0729aea4a" (UID: "d6a54937-6cfe-4b4e-9860-0af0729aea4a"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:05:14.130274 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.129713 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6a54937-6cfe-4b4e-9860-0af0729aea4a-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "d6a54937-6cfe-4b4e-9860-0af0729aea4a" (UID: "d6a54937-6cfe-4b4e-9860-0af0729aea4a"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:05:14.130274 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.130183 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "d6a54937-6cfe-4b4e-9860-0af0729aea4a" (UID: "d6a54937-6cfe-4b4e-9860-0af0729aea4a"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 15:05:14.130274 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.130214 2567 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:05:14.130274 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.130230 2567 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:05:14.130274 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.130246 2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:05:14.130274 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.130260 2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:05:14.130678 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.130288 2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:05:14.130678 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.130439 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6a54937-6cfe-4b4e-9860-0af0729aea4a-config-out" (OuterVolumeSpecName: "config-out") pod "d6a54937-6cfe-4b4e-9860-0af0729aea4a" (UID: "d6a54937-6cfe-4b4e-9860-0af0729aea4a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:05:14.131351 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.131315 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "d6a54937-6cfe-4b4e-9860-0af0729aea4a" (UID: "d6a54937-6cfe-4b4e-9860-0af0729aea4a"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 15:05:14.131616 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.131584 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6a54937-6cfe-4b4e-9860-0af0729aea4a-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "d6a54937-6cfe-4b4e-9860-0af0729aea4a" (UID: "d6a54937-6cfe-4b4e-9860-0af0729aea4a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:05:14.132048 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.132015 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "d6a54937-6cfe-4b4e-9860-0af0729aea4a" (UID: "d6a54937-6cfe-4b4e-9860-0af0729aea4a"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 15:05:14.132248 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.132207 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "d6a54937-6cfe-4b4e-9860-0af0729aea4a" (UID: "d6a54937-6cfe-4b4e-9860-0af0729aea4a"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 15:05:14.132479 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.132455 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "d6a54937-6cfe-4b4e-9860-0af0729aea4a" (UID: "d6a54937-6cfe-4b4e-9860-0af0729aea4a"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 15:05:14.132921 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.132886 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6a54937-6cfe-4b4e-9860-0af0729aea4a-kube-api-access-5w62l" (OuterVolumeSpecName: "kube-api-access-5w62l") pod "d6a54937-6cfe-4b4e-9860-0af0729aea4a" (UID: "d6a54937-6cfe-4b4e-9860-0af0729aea4a"). InnerVolumeSpecName "kube-api-access-5w62l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:05:14.133087 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.133053 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "d6a54937-6cfe-4b4e-9860-0af0729aea4a" (UID: "d6a54937-6cfe-4b4e-9860-0af0729aea4a"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 15:05:14.133575 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.133503 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "d6a54937-6cfe-4b4e-9860-0af0729aea4a" (UID: "d6a54937-6cfe-4b4e-9860-0af0729aea4a"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 15:05:14.134143 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.134112 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-config" (OuterVolumeSpecName: "config") pod "d6a54937-6cfe-4b4e-9860-0af0729aea4a" (UID: "d6a54937-6cfe-4b4e-9860-0af0729aea4a"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 15:05:14.148162 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.148098 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-web-config" (OuterVolumeSpecName: "web-config") pod "d6a54937-6cfe-4b4e-9860-0af0729aea4a" (UID: "d6a54937-6cfe-4b4e-9860-0af0729aea4a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 15:05:14.231359 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.231316 2567 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:05:14.231359 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.231350 2567 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d6a54937-6cfe-4b4e-9860-0af0729aea4a-config-out\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:05:14.231359 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.231365 2567 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-secret-prometheus-k8s-tls\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:05:14.231622 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.231376 2567 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-secret-metrics-client-certs\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:05:14.231622 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.231389 2567 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d6a54937-6cfe-4b4e-9860-0af0729aea4a-tls-assets\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:05:14.231622 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.231401 2567 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:05:14.231622 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.231423 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5w62l\" (UniqueName: \"kubernetes.io/projected/d6a54937-6cfe-4b4e-9860-0af0729aea4a-kube-api-access-5w62l\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:05:14.231622 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.231435 2567 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d6a54937-6cfe-4b4e-9860-0af0729aea4a-configmap-metrics-client-ca\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:05:14.231622 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.231450 2567 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-secret-kube-rbac-proxy\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:05:14.231622 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.231461 2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d6a54937-6cfe-4b4e-9860-0af0729aea4a-prometheus-k8s-db\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:05:14.231622 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.231472 2567 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-web-config\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:05:14.231622 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.231485 2567 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-secret-grpc-tls\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:05:14.231622 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.231497 2567 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-config\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:05:14.231622 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.231508 2567 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d6a54937-6cfe-4b4e-9860-0af0729aea4a-thanos-prometheus-http-client-file\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:05:14.721771 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.721729 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-c6854b798-q486f" event={"ID":"49820873-b1af-45a8-aed3-24c2d227f933","Type":"ContainerStarted","Data":"6d9e1c827d571ee8067ed5a23a4df760fc4c88616bab282a6a67320c309125e1"} Jul 09 15:05:14.724649 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.724622 2567 generic.go:356] "Generic (PLEG): container finished" podID="d6a54937-6cfe-4b4e-9860-0af0729aea4a" containerID="d0756f81f1540b764bb51da87cea08cc6efe224344f67388d8ba76023fa5ea53" exitCode=0 Jul 09 15:05:14.724649 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.724649 2567 generic.go:356] "Generic (PLEG): container finished" podID="d6a54937-6cfe-4b4e-9860-0af0729aea4a" containerID="44cb2dc4d500b79fcaa558c7b96a7f4d6b1bde902dbb9ccacb4e4afd7981f777" exitCode=0 Jul 09 15:05:14.724838 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.724656 2567 generic.go:356] "Generic (PLEG): container finished" podID="d6a54937-6cfe-4b4e-9860-0af0729aea4a" containerID="3a1408520418153f2025107bfabbd9d57db7916201bd2878b0d02b8ed05bcb94" exitCode=0 Jul 09 15:05:14.724838 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.724665 2567 generic.go:356] "Generic (PLEG): container finished" podID="d6a54937-6cfe-4b4e-9860-0af0729aea4a" containerID="971b2617db0073606194ffc9c8d782ba2226b9e756e10d4319b2b2a24de7ec20" exitCode=0 Jul 09 15:05:14.724838 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.724671 2567 generic.go:356] "Generic (PLEG): container finished" podID="d6a54937-6cfe-4b4e-9860-0af0729aea4a" containerID="44d6e12a47ea4ebeb587ec6be072bbbeb0119090d98efe643ebb9eb909b131e9" exitCode=0 Jul 09 15:05:14.724838 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.724675 2567 generic.go:356] "Generic (PLEG): container finished" podID="d6a54937-6cfe-4b4e-9860-0af0729aea4a" containerID="d0e271cb64e3315e8c12cd01d778dc04ae85a8fb65e2889d66971c5b00ee3d74" exitCode=0 Jul 09 15:05:14.724838 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.724704 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.724838 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.724702 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d6a54937-6cfe-4b4e-9860-0af0729aea4a","Type":"ContainerDied","Data":"d0756f81f1540b764bb51da87cea08cc6efe224344f67388d8ba76023fa5ea53"} Jul 09 15:05:14.725134 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.724858 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d6a54937-6cfe-4b4e-9860-0af0729aea4a","Type":"ContainerDied","Data":"44cb2dc4d500b79fcaa558c7b96a7f4d6b1bde902dbb9ccacb4e4afd7981f777"} Jul 09 15:05:14.725134 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.724889 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d6a54937-6cfe-4b4e-9860-0af0729aea4a","Type":"ContainerDied","Data":"3a1408520418153f2025107bfabbd9d57db7916201bd2878b0d02b8ed05bcb94"} Jul 09 15:05:14.725134 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.724914 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d6a54937-6cfe-4b4e-9860-0af0729aea4a","Type":"ContainerDied","Data":"971b2617db0073606194ffc9c8d782ba2226b9e756e10d4319b2b2a24de7ec20"} Jul 09 15:05:14.725134 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.724921 2567 scope.go:117] "RemoveContainer" containerID="d0756f81f1540b764bb51da87cea08cc6efe224344f67388d8ba76023fa5ea53" Jul 09 15:05:14.725134 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.724938 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d6a54937-6cfe-4b4e-9860-0af0729aea4a","Type":"ContainerDied","Data":"44d6e12a47ea4ebeb587ec6be072bbbeb0119090d98efe643ebb9eb909b131e9"} Jul 09 15:05:14.725134 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.724964 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d6a54937-6cfe-4b4e-9860-0af0729aea4a","Type":"ContainerDied","Data":"d0e271cb64e3315e8c12cd01d778dc04ae85a8fb65e2889d66971c5b00ee3d74"} Jul 09 15:05:14.725134 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.724987 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d6a54937-6cfe-4b4e-9860-0af0729aea4a","Type":"ContainerDied","Data":"20c1c86f7abb7fff636f464c6c036276d74813eca592f5b331304a566bd93c44"} Jul 09 15:05:14.733629 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.733613 2567 scope.go:117] "RemoveContainer" containerID="44cb2dc4d500b79fcaa558c7b96a7f4d6b1bde902dbb9ccacb4e4afd7981f777" Jul 09 15:05:14.741304 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.741284 2567 scope.go:117] "RemoveContainer" containerID="3a1408520418153f2025107bfabbd9d57db7916201bd2878b0d02b8ed05bcb94" Jul 09 15:05:14.749527 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.749502 2567 scope.go:117] "RemoveContainer" containerID="971b2617db0073606194ffc9c8d782ba2226b9e756e10d4319b2b2a24de7ec20" Jul 09 15:05:14.753736 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.753710 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 15:05:14.758215 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.758187 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 15:05:14.760009 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.759990 2567 scope.go:117] "RemoveContainer" containerID="44d6e12a47ea4ebeb587ec6be072bbbeb0119090d98efe643ebb9eb909b131e9" Jul 09 15:05:14.768282 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.768256 2567 scope.go:117] "RemoveContainer" containerID="d0e271cb64e3315e8c12cd01d778dc04ae85a8fb65e2889d66971c5b00ee3d74" Jul 09 15:05:14.777633 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.777612 2567 scope.go:117] "RemoveContainer" containerID="67c4861c1f2c85832af6b0e6f9383cd922269592715adee4c925bac7059f4145" Jul 09 15:05:14.785644 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.785615 2567 scope.go:117] "RemoveContainer" containerID="d0756f81f1540b764bb51da87cea08cc6efe224344f67388d8ba76023fa5ea53" Jul 09 15:05:14.785970 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:05:14.785944 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0756f81f1540b764bb51da87cea08cc6efe224344f67388d8ba76023fa5ea53\": container with ID starting with d0756f81f1540b764bb51da87cea08cc6efe224344f67388d8ba76023fa5ea53 not found: ID does not exist" containerID="d0756f81f1540b764bb51da87cea08cc6efe224344f67388d8ba76023fa5ea53" Jul 09 15:05:14.786059 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.785982 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0756f81f1540b764bb51da87cea08cc6efe224344f67388d8ba76023fa5ea53"} err="failed to get container status \"d0756f81f1540b764bb51da87cea08cc6efe224344f67388d8ba76023fa5ea53\": rpc error: code = NotFound desc = could not find container \"d0756f81f1540b764bb51da87cea08cc6efe224344f67388d8ba76023fa5ea53\": container with ID starting with d0756f81f1540b764bb51da87cea08cc6efe224344f67388d8ba76023fa5ea53 not found: ID does not exist" Jul 09 15:05:14.786059 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.786009 2567 scope.go:117] "RemoveContainer" containerID="44cb2dc4d500b79fcaa558c7b96a7f4d6b1bde902dbb9ccacb4e4afd7981f777" Jul 09 15:05:14.786319 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:05:14.786296 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44cb2dc4d500b79fcaa558c7b96a7f4d6b1bde902dbb9ccacb4e4afd7981f777\": container with ID starting with 44cb2dc4d500b79fcaa558c7b96a7f4d6b1bde902dbb9ccacb4e4afd7981f777 not found: ID does not exist" containerID="44cb2dc4d500b79fcaa558c7b96a7f4d6b1bde902dbb9ccacb4e4afd7981f777" Jul 09 15:05:14.786391 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.786326 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44cb2dc4d500b79fcaa558c7b96a7f4d6b1bde902dbb9ccacb4e4afd7981f777"} err="failed to get container status \"44cb2dc4d500b79fcaa558c7b96a7f4d6b1bde902dbb9ccacb4e4afd7981f777\": rpc error: code = NotFound desc = could not find container \"44cb2dc4d500b79fcaa558c7b96a7f4d6b1bde902dbb9ccacb4e4afd7981f777\": container with ID starting with 44cb2dc4d500b79fcaa558c7b96a7f4d6b1bde902dbb9ccacb4e4afd7981f777 not found: ID does not exist" Jul 09 15:05:14.786391 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.786348 2567 scope.go:117] "RemoveContainer" containerID="3a1408520418153f2025107bfabbd9d57db7916201bd2878b0d02b8ed05bcb94" Jul 09 15:05:14.786808 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:05:14.786789 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a1408520418153f2025107bfabbd9d57db7916201bd2878b0d02b8ed05bcb94\": container with ID starting with 3a1408520418153f2025107bfabbd9d57db7916201bd2878b0d02b8ed05bcb94 not found: ID does not exist" containerID="3a1408520418153f2025107bfabbd9d57db7916201bd2878b0d02b8ed05bcb94" Jul 09 15:05:14.786933 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.786912 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a1408520418153f2025107bfabbd9d57db7916201bd2878b0d02b8ed05bcb94"} err="failed to get container status \"3a1408520418153f2025107bfabbd9d57db7916201bd2878b0d02b8ed05bcb94\": rpc error: code = NotFound desc = could not find container \"3a1408520418153f2025107bfabbd9d57db7916201bd2878b0d02b8ed05bcb94\": container with ID starting with 3a1408520418153f2025107bfabbd9d57db7916201bd2878b0d02b8ed05bcb94 not found: ID does not exist" Jul 09 15:05:14.786996 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.786936 2567 scope.go:117] "RemoveContainer" containerID="971b2617db0073606194ffc9c8d782ba2226b9e756e10d4319b2b2a24de7ec20" Jul 09 15:05:14.787251 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:05:14.787227 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"971b2617db0073606194ffc9c8d782ba2226b9e756e10d4319b2b2a24de7ec20\": container with ID starting with 971b2617db0073606194ffc9c8d782ba2226b9e756e10d4319b2b2a24de7ec20 not found: ID does not exist" containerID="971b2617db0073606194ffc9c8d782ba2226b9e756e10d4319b2b2a24de7ec20" Jul 09 15:05:14.787314 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.787260 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"971b2617db0073606194ffc9c8d782ba2226b9e756e10d4319b2b2a24de7ec20"} err="failed to get container status \"971b2617db0073606194ffc9c8d782ba2226b9e756e10d4319b2b2a24de7ec20\": rpc error: code = NotFound desc = could not find container \"971b2617db0073606194ffc9c8d782ba2226b9e756e10d4319b2b2a24de7ec20\": container with ID starting with 971b2617db0073606194ffc9c8d782ba2226b9e756e10d4319b2b2a24de7ec20 not found: ID does not exist" Jul 09 15:05:14.787314 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.787277 2567 scope.go:117] "RemoveContainer" containerID="44d6e12a47ea4ebeb587ec6be072bbbeb0119090d98efe643ebb9eb909b131e9" Jul 09 15:05:14.787566 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:05:14.787547 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44d6e12a47ea4ebeb587ec6be072bbbeb0119090d98efe643ebb9eb909b131e9\": container with ID starting with 44d6e12a47ea4ebeb587ec6be072bbbeb0119090d98efe643ebb9eb909b131e9 not found: ID does not exist" containerID="44d6e12a47ea4ebeb587ec6be072bbbeb0119090d98efe643ebb9eb909b131e9" Jul 09 15:05:14.787638 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.787572 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44d6e12a47ea4ebeb587ec6be072bbbeb0119090d98efe643ebb9eb909b131e9"} err="failed to get container status \"44d6e12a47ea4ebeb587ec6be072bbbeb0119090d98efe643ebb9eb909b131e9\": rpc error: code = NotFound desc = could not find container \"44d6e12a47ea4ebeb587ec6be072bbbeb0119090d98efe643ebb9eb909b131e9\": container with ID starting with 44d6e12a47ea4ebeb587ec6be072bbbeb0119090d98efe643ebb9eb909b131e9 not found: ID does not exist" Jul 09 15:05:14.787638 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.787591 2567 scope.go:117] "RemoveContainer" containerID="d0e271cb64e3315e8c12cd01d778dc04ae85a8fb65e2889d66971c5b00ee3d74" Jul 09 15:05:14.787890 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:05:14.787864 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0e271cb64e3315e8c12cd01d778dc04ae85a8fb65e2889d66971c5b00ee3d74\": container with ID starting with d0e271cb64e3315e8c12cd01d778dc04ae85a8fb65e2889d66971c5b00ee3d74 not found: ID does not exist" containerID="d0e271cb64e3315e8c12cd01d778dc04ae85a8fb65e2889d66971c5b00ee3d74" Jul 09 15:05:14.788006 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.787895 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0e271cb64e3315e8c12cd01d778dc04ae85a8fb65e2889d66971c5b00ee3d74"} err="failed to get container status \"d0e271cb64e3315e8c12cd01d778dc04ae85a8fb65e2889d66971c5b00ee3d74\": rpc error: code = NotFound desc = could not find container \"d0e271cb64e3315e8c12cd01d778dc04ae85a8fb65e2889d66971c5b00ee3d74\": container with ID starting with d0e271cb64e3315e8c12cd01d778dc04ae85a8fb65e2889d66971c5b00ee3d74 not found: ID does not exist" Jul 09 15:05:14.788006 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.787916 2567 scope.go:117] "RemoveContainer" containerID="67c4861c1f2c85832af6b0e6f9383cd922269592715adee4c925bac7059f4145" Jul 09 15:05:14.788228 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:05:14.788197 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67c4861c1f2c85832af6b0e6f9383cd922269592715adee4c925bac7059f4145\": container with ID starting with 67c4861c1f2c85832af6b0e6f9383cd922269592715adee4c925bac7059f4145 not found: ID does not exist" containerID="67c4861c1f2c85832af6b0e6f9383cd922269592715adee4c925bac7059f4145" Jul 09 15:05:14.788289 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.788233 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67c4861c1f2c85832af6b0e6f9383cd922269592715adee4c925bac7059f4145"} err="failed to get container status \"67c4861c1f2c85832af6b0e6f9383cd922269592715adee4c925bac7059f4145\": rpc error: code = NotFound desc = could not find container \"67c4861c1f2c85832af6b0e6f9383cd922269592715adee4c925bac7059f4145\": container with ID starting with 67c4861c1f2c85832af6b0e6f9383cd922269592715adee4c925bac7059f4145 not found: ID does not exist" Jul 09 15:05:14.788289 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.788247 2567 scope.go:117] "RemoveContainer" containerID="d0756f81f1540b764bb51da87cea08cc6efe224344f67388d8ba76023fa5ea53" Jul 09 15:05:14.788556 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.788499 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0756f81f1540b764bb51da87cea08cc6efe224344f67388d8ba76023fa5ea53"} err="failed to get container status \"d0756f81f1540b764bb51da87cea08cc6efe224344f67388d8ba76023fa5ea53\": rpc error: code = NotFound desc = could not find container \"d0756f81f1540b764bb51da87cea08cc6efe224344f67388d8ba76023fa5ea53\": container with ID starting with d0756f81f1540b764bb51da87cea08cc6efe224344f67388d8ba76023fa5ea53 not found: ID does not exist" Jul 09 15:05:14.788556 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.788524 2567 scope.go:117] "RemoveContainer" containerID="44cb2dc4d500b79fcaa558c7b96a7f4d6b1bde902dbb9ccacb4e4afd7981f777" Jul 09 15:05:14.788781 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.788759 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44cb2dc4d500b79fcaa558c7b96a7f4d6b1bde902dbb9ccacb4e4afd7981f777"} err="failed to get container status \"44cb2dc4d500b79fcaa558c7b96a7f4d6b1bde902dbb9ccacb4e4afd7981f777\": rpc error: code = NotFound desc = could not find container \"44cb2dc4d500b79fcaa558c7b96a7f4d6b1bde902dbb9ccacb4e4afd7981f777\": container with ID starting with 44cb2dc4d500b79fcaa558c7b96a7f4d6b1bde902dbb9ccacb4e4afd7981f777 not found: ID does not exist" Jul 09 15:05:14.788856 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.788783 2567 scope.go:117] "RemoveContainer" containerID="3a1408520418153f2025107bfabbd9d57db7916201bd2878b0d02b8ed05bcb94" Jul 09 15:05:14.789060 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.789038 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a1408520418153f2025107bfabbd9d57db7916201bd2878b0d02b8ed05bcb94"} err="failed to get container status \"3a1408520418153f2025107bfabbd9d57db7916201bd2878b0d02b8ed05bcb94\": rpc error: code = NotFound desc = could not find container \"3a1408520418153f2025107bfabbd9d57db7916201bd2878b0d02b8ed05bcb94\": container with ID starting with 3a1408520418153f2025107bfabbd9d57db7916201bd2878b0d02b8ed05bcb94 not found: ID does not exist" Jul 09 15:05:14.789148 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.789061 2567 scope.go:117] "RemoveContainer" containerID="971b2617db0073606194ffc9c8d782ba2226b9e756e10d4319b2b2a24de7ec20" Jul 09 15:05:14.789310 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.789293 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"971b2617db0073606194ffc9c8d782ba2226b9e756e10d4319b2b2a24de7ec20"} err="failed to get container status \"971b2617db0073606194ffc9c8d782ba2226b9e756e10d4319b2b2a24de7ec20\": rpc error: code = NotFound desc = could not find container \"971b2617db0073606194ffc9c8d782ba2226b9e756e10d4319b2b2a24de7ec20\": container with ID starting with 971b2617db0073606194ffc9c8d782ba2226b9e756e10d4319b2b2a24de7ec20 not found: ID does not exist" Jul 09 15:05:14.789360 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.789310 2567 scope.go:117] "RemoveContainer" containerID="44d6e12a47ea4ebeb587ec6be072bbbeb0119090d98efe643ebb9eb909b131e9" Jul 09 15:05:14.789547 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.789511 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44d6e12a47ea4ebeb587ec6be072bbbeb0119090d98efe643ebb9eb909b131e9"} err="failed to get container status \"44d6e12a47ea4ebeb587ec6be072bbbeb0119090d98efe643ebb9eb909b131e9\": rpc error: code = NotFound desc = could not find container \"44d6e12a47ea4ebeb587ec6be072bbbeb0119090d98efe643ebb9eb909b131e9\": container with ID starting with 44d6e12a47ea4ebeb587ec6be072bbbeb0119090d98efe643ebb9eb909b131e9 not found: ID does not exist" Jul 09 15:05:14.789547 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.789547 2567 scope.go:117] "RemoveContainer" containerID="d0e271cb64e3315e8c12cd01d778dc04ae85a8fb65e2889d66971c5b00ee3d74" Jul 09 15:05:14.789747 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.789728 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0e271cb64e3315e8c12cd01d778dc04ae85a8fb65e2889d66971c5b00ee3d74"} err="failed to get container status \"d0e271cb64e3315e8c12cd01d778dc04ae85a8fb65e2889d66971c5b00ee3d74\": rpc error: code = NotFound desc = could not find container \"d0e271cb64e3315e8c12cd01d778dc04ae85a8fb65e2889d66971c5b00ee3d74\": container with ID starting with d0e271cb64e3315e8c12cd01d778dc04ae85a8fb65e2889d66971c5b00ee3d74 not found: ID does not exist" Jul 09 15:05:14.789873 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.789749 2567 scope.go:117] "RemoveContainer" containerID="67c4861c1f2c85832af6b0e6f9383cd922269592715adee4c925bac7059f4145" Jul 09 15:05:14.790019 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.790001 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67c4861c1f2c85832af6b0e6f9383cd922269592715adee4c925bac7059f4145"} err="failed to get container status \"67c4861c1f2c85832af6b0e6f9383cd922269592715adee4c925bac7059f4145\": rpc error: code = NotFound desc = could not find container \"67c4861c1f2c85832af6b0e6f9383cd922269592715adee4c925bac7059f4145\": container with ID starting with 67c4861c1f2c85832af6b0e6f9383cd922269592715adee4c925bac7059f4145 not found: ID does not exist" Jul 09 15:05:14.790084 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.790019 2567 scope.go:117] "RemoveContainer" containerID="d0756f81f1540b764bb51da87cea08cc6efe224344f67388d8ba76023fa5ea53" Jul 09 15:05:14.790245 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.790218 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0756f81f1540b764bb51da87cea08cc6efe224344f67388d8ba76023fa5ea53"} err="failed to get container status \"d0756f81f1540b764bb51da87cea08cc6efe224344f67388d8ba76023fa5ea53\": rpc error: code = NotFound desc = could not find container \"d0756f81f1540b764bb51da87cea08cc6efe224344f67388d8ba76023fa5ea53\": container with ID starting with d0756f81f1540b764bb51da87cea08cc6efe224344f67388d8ba76023fa5ea53 not found: ID does not exist" Jul 09 15:05:14.790329 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.790246 2567 scope.go:117] "RemoveContainer" containerID="44cb2dc4d500b79fcaa558c7b96a7f4d6b1bde902dbb9ccacb4e4afd7981f777" Jul 09 15:05:14.790497 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.790478 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44cb2dc4d500b79fcaa558c7b96a7f4d6b1bde902dbb9ccacb4e4afd7981f777"} err="failed to get container status \"44cb2dc4d500b79fcaa558c7b96a7f4d6b1bde902dbb9ccacb4e4afd7981f777\": rpc error: code = NotFound desc = could not find container \"44cb2dc4d500b79fcaa558c7b96a7f4d6b1bde902dbb9ccacb4e4afd7981f777\": container with ID starting with 44cb2dc4d500b79fcaa558c7b96a7f4d6b1bde902dbb9ccacb4e4afd7981f777 not found: ID does not exist" Jul 09 15:05:14.790574 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.790497 2567 scope.go:117] "RemoveContainer" containerID="3a1408520418153f2025107bfabbd9d57db7916201bd2878b0d02b8ed05bcb94" Jul 09 15:05:14.790804 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.790784 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a1408520418153f2025107bfabbd9d57db7916201bd2878b0d02b8ed05bcb94"} err="failed to get container status \"3a1408520418153f2025107bfabbd9d57db7916201bd2878b0d02b8ed05bcb94\": rpc error: code = NotFound desc = could not find container \"3a1408520418153f2025107bfabbd9d57db7916201bd2878b0d02b8ed05bcb94\": container with ID starting with 3a1408520418153f2025107bfabbd9d57db7916201bd2878b0d02b8ed05bcb94 not found: ID does not exist" Jul 09 15:05:14.790804 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.790804 2567 scope.go:117] "RemoveContainer" containerID="971b2617db0073606194ffc9c8d782ba2226b9e756e10d4319b2b2a24de7ec20" Jul 09 15:05:14.791250 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.791215 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"971b2617db0073606194ffc9c8d782ba2226b9e756e10d4319b2b2a24de7ec20"} err="failed to get container status \"971b2617db0073606194ffc9c8d782ba2226b9e756e10d4319b2b2a24de7ec20\": rpc error: code = NotFound desc = could not find container \"971b2617db0073606194ffc9c8d782ba2226b9e756e10d4319b2b2a24de7ec20\": container with ID starting with 971b2617db0073606194ffc9c8d782ba2226b9e756e10d4319b2b2a24de7ec20 not found: ID does not exist" Jul 09 15:05:14.791250 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.791237 2567 scope.go:117] "RemoveContainer" containerID="44d6e12a47ea4ebeb587ec6be072bbbeb0119090d98efe643ebb9eb909b131e9" Jul 09 15:05:14.791868 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.791839 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44d6e12a47ea4ebeb587ec6be072bbbeb0119090d98efe643ebb9eb909b131e9"} err="failed to get container status \"44d6e12a47ea4ebeb587ec6be072bbbeb0119090d98efe643ebb9eb909b131e9\": rpc error: code = NotFound desc = could not find container \"44d6e12a47ea4ebeb587ec6be072bbbeb0119090d98efe643ebb9eb909b131e9\": container with ID starting with 44d6e12a47ea4ebeb587ec6be072bbbeb0119090d98efe643ebb9eb909b131e9 not found: ID does not exist" Jul 09 15:05:14.791986 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.791869 2567 scope.go:117] "RemoveContainer" containerID="d0e271cb64e3315e8c12cd01d778dc04ae85a8fb65e2889d66971c5b00ee3d74" Jul 09 15:05:14.792154 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.792127 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0e271cb64e3315e8c12cd01d778dc04ae85a8fb65e2889d66971c5b00ee3d74"} err="failed to get container status \"d0e271cb64e3315e8c12cd01d778dc04ae85a8fb65e2889d66971c5b00ee3d74\": rpc error: code = NotFound desc = could not find container \"d0e271cb64e3315e8c12cd01d778dc04ae85a8fb65e2889d66971c5b00ee3d74\": container with ID starting with d0e271cb64e3315e8c12cd01d778dc04ae85a8fb65e2889d66971c5b00ee3d74 not found: ID does not exist" Jul 09 15:05:14.792154 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.792153 2567 scope.go:117] "RemoveContainer" containerID="67c4861c1f2c85832af6b0e6f9383cd922269592715adee4c925bac7059f4145" Jul 09 15:05:14.792529 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.792504 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67c4861c1f2c85832af6b0e6f9383cd922269592715adee4c925bac7059f4145"} err="failed to get container status \"67c4861c1f2c85832af6b0e6f9383cd922269592715adee4c925bac7059f4145\": rpc error: code = NotFound desc = could not find container \"67c4861c1f2c85832af6b0e6f9383cd922269592715adee4c925bac7059f4145\": container with ID starting with 67c4861c1f2c85832af6b0e6f9383cd922269592715adee4c925bac7059f4145 not found: ID does not exist" Jul 09 15:05:14.792622 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.792532 2567 scope.go:117] "RemoveContainer" containerID="d0756f81f1540b764bb51da87cea08cc6efe224344f67388d8ba76023fa5ea53" Jul 09 15:05:14.792901 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.792872 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0756f81f1540b764bb51da87cea08cc6efe224344f67388d8ba76023fa5ea53"} err="failed to get container status \"d0756f81f1540b764bb51da87cea08cc6efe224344f67388d8ba76023fa5ea53\": rpc error: code = NotFound desc = could not find container \"d0756f81f1540b764bb51da87cea08cc6efe224344f67388d8ba76023fa5ea53\": container with ID starting with d0756f81f1540b764bb51da87cea08cc6efe224344f67388d8ba76023fa5ea53 not found: ID does not exist" Jul 09 15:05:14.792901 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.792901 2567 scope.go:117] "RemoveContainer" containerID="44cb2dc4d500b79fcaa558c7b96a7f4d6b1bde902dbb9ccacb4e4afd7981f777" Jul 09 15:05:14.793167 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.793133 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44cb2dc4d500b79fcaa558c7b96a7f4d6b1bde902dbb9ccacb4e4afd7981f777"} err="failed to get container status \"44cb2dc4d500b79fcaa558c7b96a7f4d6b1bde902dbb9ccacb4e4afd7981f777\": rpc error: code = NotFound desc = could not find container \"44cb2dc4d500b79fcaa558c7b96a7f4d6b1bde902dbb9ccacb4e4afd7981f777\": container with ID starting with 44cb2dc4d500b79fcaa558c7b96a7f4d6b1bde902dbb9ccacb4e4afd7981f777 not found: ID does not exist" Jul 09 15:05:14.793167 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.793159 2567 scope.go:117] "RemoveContainer" containerID="3a1408520418153f2025107bfabbd9d57db7916201bd2878b0d02b8ed05bcb94" Jul 09 15:05:14.793509 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.793482 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a1408520418153f2025107bfabbd9d57db7916201bd2878b0d02b8ed05bcb94"} err="failed to get container status \"3a1408520418153f2025107bfabbd9d57db7916201bd2878b0d02b8ed05bcb94\": rpc error: code = NotFound desc = could not find container \"3a1408520418153f2025107bfabbd9d57db7916201bd2878b0d02b8ed05bcb94\": container with ID starting with 3a1408520418153f2025107bfabbd9d57db7916201bd2878b0d02b8ed05bcb94 not found: ID does not exist" Jul 09 15:05:14.793509 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.793509 2567 scope.go:117] "RemoveContainer" containerID="971b2617db0073606194ffc9c8d782ba2226b9e756e10d4319b2b2a24de7ec20" Jul 09 15:05:14.793775 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.793750 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"971b2617db0073606194ffc9c8d782ba2226b9e756e10d4319b2b2a24de7ec20"} err="failed to get container status \"971b2617db0073606194ffc9c8d782ba2226b9e756e10d4319b2b2a24de7ec20\": rpc error: code = NotFound desc = could not find container \"971b2617db0073606194ffc9c8d782ba2226b9e756e10d4319b2b2a24de7ec20\": container with ID starting with 971b2617db0073606194ffc9c8d782ba2226b9e756e10d4319b2b2a24de7ec20 not found: ID does not exist" Jul 09 15:05:14.793881 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.793776 2567 scope.go:117] "RemoveContainer" containerID="44d6e12a47ea4ebeb587ec6be072bbbeb0119090d98efe643ebb9eb909b131e9" Jul 09 15:05:14.794102 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.794084 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 15:05:14.794102 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.794087 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44d6e12a47ea4ebeb587ec6be072bbbeb0119090d98efe643ebb9eb909b131e9"} err="failed to get container status \"44d6e12a47ea4ebeb587ec6be072bbbeb0119090d98efe643ebb9eb909b131e9\": rpc error: code = NotFound desc = could not find container \"44d6e12a47ea4ebeb587ec6be072bbbeb0119090d98efe643ebb9eb909b131e9\": container with ID starting with 44d6e12a47ea4ebeb587ec6be072bbbeb0119090d98efe643ebb9eb909b131e9 not found: ID does not exist" Jul 09 15:05:14.794216 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.794112 2567 scope.go:117] "RemoveContainer" containerID="d0e271cb64e3315e8c12cd01d778dc04ae85a8fb65e2889d66971c5b00ee3d74" Jul 09 15:05:14.794353 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.794335 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0e271cb64e3315e8c12cd01d778dc04ae85a8fb65e2889d66971c5b00ee3d74"} err="failed to get container status \"d0e271cb64e3315e8c12cd01d778dc04ae85a8fb65e2889d66971c5b00ee3d74\": rpc error: code = NotFound desc = could not find container \"d0e271cb64e3315e8c12cd01d778dc04ae85a8fb65e2889d66971c5b00ee3d74\": container with ID starting with d0e271cb64e3315e8c12cd01d778dc04ae85a8fb65e2889d66971c5b00ee3d74 not found: ID does not exist" Jul 09 15:05:14.794390 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.794356 2567 scope.go:117] "RemoveContainer" containerID="67c4861c1f2c85832af6b0e6f9383cd922269592715adee4c925bac7059f4145" Jul 09 15:05:14.794470 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.794453 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6a54937-6cfe-4b4e-9860-0af0729aea4a" containerName="init-config-reloader" Jul 09 15:05:14.794470 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.794471 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6a54937-6cfe-4b4e-9860-0af0729aea4a" containerName="init-config-reloader" Jul 09 15:05:14.794601 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.794491 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6a54937-6cfe-4b4e-9860-0af0729aea4a" containerName="config-reloader" Jul 09 15:05:14.794601 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.794501 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6a54937-6cfe-4b4e-9860-0af0729aea4a" containerName="config-reloader" Jul 09 15:05:14.794601 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.794511 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6a54937-6cfe-4b4e-9860-0af0729aea4a" containerName="kube-rbac-proxy-web" Jul 09 15:05:14.794601 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.794519 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6a54937-6cfe-4b4e-9860-0af0729aea4a" containerName="kube-rbac-proxy-web" Jul 09 15:05:14.794601 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.794532 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6a54937-6cfe-4b4e-9860-0af0729aea4a" containerName="kube-rbac-proxy" Jul 09 15:05:14.794601 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.794540 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6a54937-6cfe-4b4e-9860-0af0729aea4a" containerName="kube-rbac-proxy" Jul 09 15:05:14.794601 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.794556 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6a54937-6cfe-4b4e-9860-0af0729aea4a" containerName="kube-rbac-proxy-thanos" Jul 09 15:05:14.794601 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.794564 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6a54937-6cfe-4b4e-9860-0af0729aea4a" containerName="kube-rbac-proxy-thanos" Jul 09 15:05:14.794601 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.794578 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6a54937-6cfe-4b4e-9860-0af0729aea4a" containerName="prometheus" Jul 09 15:05:14.794601 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.794587 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6a54937-6cfe-4b4e-9860-0af0729aea4a" containerName="prometheus" Jul 09 15:05:14.794601 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.794602 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6a54937-6cfe-4b4e-9860-0af0729aea4a" containerName="thanos-sidecar" Jul 09 15:05:14.795022 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.794611 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6a54937-6cfe-4b4e-9860-0af0729aea4a" containerName="thanos-sidecar" Jul 09 15:05:14.795022 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.794636 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67c4861c1f2c85832af6b0e6f9383cd922269592715adee4c925bac7059f4145"} err="failed to get container status \"67c4861c1f2c85832af6b0e6f9383cd922269592715adee4c925bac7059f4145\": rpc error: code = NotFound desc = could not find container \"67c4861c1f2c85832af6b0e6f9383cd922269592715adee4c925bac7059f4145\": container with ID starting with 67c4861c1f2c85832af6b0e6f9383cd922269592715adee4c925bac7059f4145 not found: ID does not exist" Jul 09 15:05:14.795022 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.794663 2567 scope.go:117] "RemoveContainer" containerID="d0756f81f1540b764bb51da87cea08cc6efe224344f67388d8ba76023fa5ea53" Jul 09 15:05:14.795022 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.794697 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6a54937-6cfe-4b4e-9860-0af0729aea4a" containerName="thanos-sidecar" Jul 09 15:05:14.795022 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.794710 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6a54937-6cfe-4b4e-9860-0af0729aea4a" containerName="kube-rbac-proxy-web" Jul 09 15:05:14.795022 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.794721 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6a54937-6cfe-4b4e-9860-0af0729aea4a" containerName="config-reloader" Jul 09 15:05:14.795022 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.794730 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6a54937-6cfe-4b4e-9860-0af0729aea4a" containerName="kube-rbac-proxy-thanos" Jul 09 15:05:14.795022 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.794739 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6a54937-6cfe-4b4e-9860-0af0729aea4a" containerName="prometheus" Jul 09 15:05:14.795022 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.794748 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6a54937-6cfe-4b4e-9860-0af0729aea4a" containerName="kube-rbac-proxy" Jul 09 15:05:14.795022 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.794978 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0756f81f1540b764bb51da87cea08cc6efe224344f67388d8ba76023fa5ea53"} err="failed to get container status \"d0756f81f1540b764bb51da87cea08cc6efe224344f67388d8ba76023fa5ea53\": rpc error: code = NotFound desc = could not find container \"d0756f81f1540b764bb51da87cea08cc6efe224344f67388d8ba76023fa5ea53\": container with ID starting with d0756f81f1540b764bb51da87cea08cc6efe224344f67388d8ba76023fa5ea53 not found: ID does not exist" Jul 09 15:05:14.795022 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.794999 2567 scope.go:117] "RemoveContainer" containerID="44cb2dc4d500b79fcaa558c7b96a7f4d6b1bde902dbb9ccacb4e4afd7981f777" Jul 09 15:05:14.795468 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.795220 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44cb2dc4d500b79fcaa558c7b96a7f4d6b1bde902dbb9ccacb4e4afd7981f777"} err="failed to get container status \"44cb2dc4d500b79fcaa558c7b96a7f4d6b1bde902dbb9ccacb4e4afd7981f777\": rpc error: code = NotFound desc = could not find container \"44cb2dc4d500b79fcaa558c7b96a7f4d6b1bde902dbb9ccacb4e4afd7981f777\": container with ID starting with 44cb2dc4d500b79fcaa558c7b96a7f4d6b1bde902dbb9ccacb4e4afd7981f777 not found: ID does not exist" Jul 09 15:05:14.795468 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.795238 2567 scope.go:117] "RemoveContainer" containerID="3a1408520418153f2025107bfabbd9d57db7916201bd2878b0d02b8ed05bcb94" Jul 09 15:05:14.795536 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.795472 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a1408520418153f2025107bfabbd9d57db7916201bd2878b0d02b8ed05bcb94"} err="failed to get container status \"3a1408520418153f2025107bfabbd9d57db7916201bd2878b0d02b8ed05bcb94\": rpc error: code = NotFound desc = could not find container \"3a1408520418153f2025107bfabbd9d57db7916201bd2878b0d02b8ed05bcb94\": container with ID starting with 3a1408520418153f2025107bfabbd9d57db7916201bd2878b0d02b8ed05bcb94 not found: ID does not exist" Jul 09 15:05:14.795536 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.795490 2567 scope.go:117] "RemoveContainer" containerID="971b2617db0073606194ffc9c8d782ba2226b9e756e10d4319b2b2a24de7ec20" Jul 09 15:05:14.796145 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.795912 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"971b2617db0073606194ffc9c8d782ba2226b9e756e10d4319b2b2a24de7ec20"} err="failed to get container status \"971b2617db0073606194ffc9c8d782ba2226b9e756e10d4319b2b2a24de7ec20\": rpc error: code = NotFound desc = could not find container \"971b2617db0073606194ffc9c8d782ba2226b9e756e10d4319b2b2a24de7ec20\": container with ID starting with 971b2617db0073606194ffc9c8d782ba2226b9e756e10d4319b2b2a24de7ec20 not found: ID does not exist" Jul 09 15:05:14.796145 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.795934 2567 scope.go:117] "RemoveContainer" containerID="44d6e12a47ea4ebeb587ec6be072bbbeb0119090d98efe643ebb9eb909b131e9" Jul 09 15:05:14.796306 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.796215 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44d6e12a47ea4ebeb587ec6be072bbbeb0119090d98efe643ebb9eb909b131e9"} err="failed to get container status \"44d6e12a47ea4ebeb587ec6be072bbbeb0119090d98efe643ebb9eb909b131e9\": rpc error: code = NotFound desc = could not find container \"44d6e12a47ea4ebeb587ec6be072bbbeb0119090d98efe643ebb9eb909b131e9\": container with ID starting with 44d6e12a47ea4ebeb587ec6be072bbbeb0119090d98efe643ebb9eb909b131e9 not found: ID does not exist" Jul 09 15:05:14.796306 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.796240 2567 scope.go:117] "RemoveContainer" containerID="d0e271cb64e3315e8c12cd01d778dc04ae85a8fb65e2889d66971c5b00ee3d74" Jul 09 15:05:14.796567 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.796514 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0e271cb64e3315e8c12cd01d778dc04ae85a8fb65e2889d66971c5b00ee3d74"} err="failed to get container status \"d0e271cb64e3315e8c12cd01d778dc04ae85a8fb65e2889d66971c5b00ee3d74\": rpc error: code = NotFound desc = could not find container \"d0e271cb64e3315e8c12cd01d778dc04ae85a8fb65e2889d66971c5b00ee3d74\": container with ID starting with d0e271cb64e3315e8c12cd01d778dc04ae85a8fb65e2889d66971c5b00ee3d74 not found: ID does not exist" Jul 09 15:05:14.796567 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.796548 2567 scope.go:117] "RemoveContainer" containerID="67c4861c1f2c85832af6b0e6f9383cd922269592715adee4c925bac7059f4145" Jul 09 15:05:14.796882 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.796859 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67c4861c1f2c85832af6b0e6f9383cd922269592715adee4c925bac7059f4145"} err="failed to get container status \"67c4861c1f2c85832af6b0e6f9383cd922269592715adee4c925bac7059f4145\": rpc error: code = NotFound desc = could not find container \"67c4861c1f2c85832af6b0e6f9383cd922269592715adee4c925bac7059f4145\": container with ID starting with 67c4861c1f2c85832af6b0e6f9383cd922269592715adee4c925bac7059f4145 not found: ID does not exist" Jul 09 15:05:14.797036 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.796884 2567 scope.go:117] "RemoveContainer" containerID="d0756f81f1540b764bb51da87cea08cc6efe224344f67388d8ba76023fa5ea53" Jul 09 15:05:14.797196 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.797170 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0756f81f1540b764bb51da87cea08cc6efe224344f67388d8ba76023fa5ea53"} err="failed to get container status \"d0756f81f1540b764bb51da87cea08cc6efe224344f67388d8ba76023fa5ea53\": rpc error: code = NotFound desc = could not find container \"d0756f81f1540b764bb51da87cea08cc6efe224344f67388d8ba76023fa5ea53\": container with ID starting with d0756f81f1540b764bb51da87cea08cc6efe224344f67388d8ba76023fa5ea53 not found: ID does not exist" Jul 09 15:05:14.797275 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.797199 2567 scope.go:117] "RemoveContainer" containerID="44cb2dc4d500b79fcaa558c7b96a7f4d6b1bde902dbb9ccacb4e4afd7981f777" Jul 09 15:05:14.797543 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.797509 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44cb2dc4d500b79fcaa558c7b96a7f4d6b1bde902dbb9ccacb4e4afd7981f777"} err="failed to get container status \"44cb2dc4d500b79fcaa558c7b96a7f4d6b1bde902dbb9ccacb4e4afd7981f777\": rpc error: code = NotFound desc = could not find container \"44cb2dc4d500b79fcaa558c7b96a7f4d6b1bde902dbb9ccacb4e4afd7981f777\": container with ID starting with 44cb2dc4d500b79fcaa558c7b96a7f4d6b1bde902dbb9ccacb4e4afd7981f777 not found: ID does not exist" Jul 09 15:05:14.797543 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.797536 2567 scope.go:117] "RemoveContainer" containerID="3a1408520418153f2025107bfabbd9d57db7916201bd2878b0d02b8ed05bcb94" Jul 09 15:05:14.797862 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.797779 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a1408520418153f2025107bfabbd9d57db7916201bd2878b0d02b8ed05bcb94"} err="failed to get container status \"3a1408520418153f2025107bfabbd9d57db7916201bd2878b0d02b8ed05bcb94\": rpc error: code = NotFound desc = could not find container \"3a1408520418153f2025107bfabbd9d57db7916201bd2878b0d02b8ed05bcb94\": container with ID starting with 3a1408520418153f2025107bfabbd9d57db7916201bd2878b0d02b8ed05bcb94 not found: ID does not exist" Jul 09 15:05:14.797862 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.797834 2567 scope.go:117] "RemoveContainer" containerID="971b2617db0073606194ffc9c8d782ba2226b9e756e10d4319b2b2a24de7ec20" Jul 09 15:05:14.798152 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.798127 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"971b2617db0073606194ffc9c8d782ba2226b9e756e10d4319b2b2a24de7ec20"} err="failed to get container status \"971b2617db0073606194ffc9c8d782ba2226b9e756e10d4319b2b2a24de7ec20\": rpc error: code = NotFound desc = could not find container \"971b2617db0073606194ffc9c8d782ba2226b9e756e10d4319b2b2a24de7ec20\": container with ID starting with 971b2617db0073606194ffc9c8d782ba2226b9e756e10d4319b2b2a24de7ec20 not found: ID does not exist" Jul 09 15:05:14.798152 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.798153 2567 scope.go:117] "RemoveContainer" containerID="44d6e12a47ea4ebeb587ec6be072bbbeb0119090d98efe643ebb9eb909b131e9" Jul 09 15:05:14.798420 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.798386 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44d6e12a47ea4ebeb587ec6be072bbbeb0119090d98efe643ebb9eb909b131e9"} err="failed to get container status \"44d6e12a47ea4ebeb587ec6be072bbbeb0119090d98efe643ebb9eb909b131e9\": rpc error: code = NotFound desc = could not find container \"44d6e12a47ea4ebeb587ec6be072bbbeb0119090d98efe643ebb9eb909b131e9\": container with ID starting with 44d6e12a47ea4ebeb587ec6be072bbbeb0119090d98efe643ebb9eb909b131e9 not found: ID does not exist" Jul 09 15:05:14.798420 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.798406 2567 scope.go:117] "RemoveContainer" containerID="d0e271cb64e3315e8c12cd01d778dc04ae85a8fb65e2889d66971c5b00ee3d74" Jul 09 15:05:14.798714 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.798693 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0e271cb64e3315e8c12cd01d778dc04ae85a8fb65e2889d66971c5b00ee3d74"} err="failed to get container status \"d0e271cb64e3315e8c12cd01d778dc04ae85a8fb65e2889d66971c5b00ee3d74\": rpc error: code = NotFound desc = could not find container \"d0e271cb64e3315e8c12cd01d778dc04ae85a8fb65e2889d66971c5b00ee3d74\": container with ID starting with d0e271cb64e3315e8c12cd01d778dc04ae85a8fb65e2889d66971c5b00ee3d74 not found: ID does not exist" Jul 09 15:05:14.798770 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.798715 2567 scope.go:117] "RemoveContainer" containerID="67c4861c1f2c85832af6b0e6f9383cd922269592715adee4c925bac7059f4145" Jul 09 15:05:14.799007 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.798980 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67c4861c1f2c85832af6b0e6f9383cd922269592715adee4c925bac7059f4145"} err="failed to get container status \"67c4861c1f2c85832af6b0e6f9383cd922269592715adee4c925bac7059f4145\": rpc error: code = NotFound desc = could not find container \"67c4861c1f2c85832af6b0e6f9383cd922269592715adee4c925bac7059f4145\": container with ID starting with 67c4861c1f2c85832af6b0e6f9383cd922269592715adee4c925bac7059f4145 not found: ID does not exist" Jul 09 15:05:14.799491 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.799474 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.807143 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.805020 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 09 15:05:14.807143 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.805058 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 09 15:05:14.807143 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.805311 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 09 15:05:14.807143 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.805341 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 09 15:05:14.807143 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.805465 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 09 15:05:14.807143 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.805565 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-8cm8l\"" Jul 09 15:05:14.807143 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.805621 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 09 15:05:14.807143 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.805724 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 09 15:05:14.807143 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.805891 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 09 15:05:14.807143 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.805906 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 09 15:05:14.807143 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.805994 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-etcverbk2i7iu\"" Jul 09 15:05:14.808407 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.807347 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 09 15:05:14.808407 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.807510 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 09 15:05:14.808407 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.807515 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 09 15:05:14.811246 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.811220 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 09 15:05:14.814736 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.814713 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 09 15:05:14.825675 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.825648 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 15:05:14.836456 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.836418 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsj4k\" (UniqueName: \"kubernetes.io/projected/f412c893-a405-4dbf-b710-678a4f373ac7-kube-api-access-dsj4k\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.836603 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.836467 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f412c893-a405-4dbf-b710-678a4f373ac7-config-out\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.836603 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.836492 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f412c893-a405-4dbf-b710-678a4f373ac7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.836603 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.836537 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f412c893-a405-4dbf-b710-678a4f373ac7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.836757 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.836638 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f412c893-a405-4dbf-b710-678a4f373ac7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.836757 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.836675 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f412c893-a405-4dbf-b710-678a4f373ac7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.836934 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.836911 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f412c893-a405-4dbf-b710-678a4f373ac7-config\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.837002 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.836959 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f412c893-a405-4dbf-b710-678a4f373ac7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.837002 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.836980 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f412c893-a405-4dbf-b710-678a4f373ac7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.837103 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.837007 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f412c893-a405-4dbf-b710-678a4f373ac7-web-config\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.837226 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.837193 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f412c893-a405-4dbf-b710-678a4f373ac7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.837329 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.837291 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f412c893-a405-4dbf-b710-678a4f373ac7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.837422 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.837401 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f412c893-a405-4dbf-b710-678a4f373ac7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.837483 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.837453 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f412c893-a405-4dbf-b710-678a4f373ac7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.837537 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.837516 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f412c893-a405-4dbf-b710-678a4f373ac7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.837626 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.837591 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f412c893-a405-4dbf-b710-678a4f373ac7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.837681 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.837623 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f412c893-a405-4dbf-b710-678a4f373ac7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.837738 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.837695 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f412c893-a405-4dbf-b710-678a4f373ac7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.837738 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.837719 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f412c893-a405-4dbf-b710-678a4f373ac7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.837879 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.837774 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f412c893-a405-4dbf-b710-678a4f373ac7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.939225 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.939186 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f412c893-a405-4dbf-b710-678a4f373ac7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.939225 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.939224 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f412c893-a405-4dbf-b710-678a4f373ac7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.939423 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.939246 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f412c893-a405-4dbf-b710-678a4f373ac7-config\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.939423 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.939266 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f412c893-a405-4dbf-b710-678a4f373ac7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.939423 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.939290 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f412c893-a405-4dbf-b710-678a4f373ac7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.939423 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.939331 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f412c893-a405-4dbf-b710-678a4f373ac7-web-config\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.939423 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.939363 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f412c893-a405-4dbf-b710-678a4f373ac7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.939423 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.939393 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f412c893-a405-4dbf-b710-678a4f373ac7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.939423 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.939416 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f412c893-a405-4dbf-b710-678a4f373ac7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.939764 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.939445 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f412c893-a405-4dbf-b710-678a4f373ac7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.939764 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.939481 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f412c893-a405-4dbf-b710-678a4f373ac7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.939764 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.939506 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f412c893-a405-4dbf-b710-678a4f373ac7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.939764 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.939537 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f412c893-a405-4dbf-b710-678a4f373ac7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.939764 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.939577 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f412c893-a405-4dbf-b710-678a4f373ac7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.939764 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.939602 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f412c893-a405-4dbf-b710-678a4f373ac7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.939764 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.939641 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f412c893-a405-4dbf-b710-678a4f373ac7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.939764 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.939668 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsj4k\" (UniqueName: \"kubernetes.io/projected/f412c893-a405-4dbf-b710-678a4f373ac7-kube-api-access-dsj4k\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.939764 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.939697 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f412c893-a405-4dbf-b710-678a4f373ac7-config-out\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.939764 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.939727 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f412c893-a405-4dbf-b710-678a4f373ac7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.939764 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.939756 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f412c893-a405-4dbf-b710-678a4f373ac7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.940326 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.939937 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f412c893-a405-4dbf-b710-678a4f373ac7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.940326 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.940103 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f412c893-a405-4dbf-b710-678a4f373ac7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.940428 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.940385 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f412c893-a405-4dbf-b710-678a4f373ac7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.941057 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.940754 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f412c893-a405-4dbf-b710-678a4f373ac7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.941720 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.941358 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f412c893-a405-4dbf-b710-678a4f373ac7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.941720 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.941511 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f412c893-a405-4dbf-b710-678a4f373ac7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.942142 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.942117 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f412c893-a405-4dbf-b710-678a4f373ac7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.942958 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.942932 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f412c893-a405-4dbf-b710-678a4f373ac7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.943512 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.943460 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f412c893-a405-4dbf-b710-678a4f373ac7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.943779 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.943759 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f412c893-a405-4dbf-b710-678a4f373ac7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.943987 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.943944 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f412c893-a405-4dbf-b710-678a4f373ac7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.944556 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.944189 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f412c893-a405-4dbf-b710-678a4f373ac7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.944556 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.944502 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f412c893-a405-4dbf-b710-678a4f373ac7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.944556 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.944548 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f412c893-a405-4dbf-b710-678a4f373ac7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.944737 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.944673 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f412c893-a405-4dbf-b710-678a4f373ac7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.945503 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.945486 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f412c893-a405-4dbf-b710-678a4f373ac7-config-out\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.945747 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.945728 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f412c893-a405-4dbf-b710-678a4f373ac7-web-config\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.945883 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.945866 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f412c893-a405-4dbf-b710-678a4f373ac7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.946034 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.946018 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/f412c893-a405-4dbf-b710-678a4f373ac7-config\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:14.951099 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:14.951080 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsj4k\" (UniqueName: \"kubernetes.io/projected/f412c893-a405-4dbf-b710-678a4f373ac7-kube-api-access-dsj4k\") pod \"prometheus-k8s-0\" (UID: \"f412c893-a405-4dbf-b710-678a4f373ac7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:15.116285 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:15.116192 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:15.275574 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:15.275540 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 15:05:15.664664 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:05:15.664628 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf412c893_a405_4dbf_b710_678a4f373ac7.slice/crio-b03aee4bbdd56357bb267e00c8847e437bb81414e2c2c37ec369bf558b7087bd WatchSource:0}: Error finding container b03aee4bbdd56357bb267e00c8847e437bb81414e2c2c37ec369bf558b7087bd: Status 404 returned error can't find the container with id b03aee4bbdd56357bb267e00c8847e437bb81414e2c2c37ec369bf558b7087bd Jul 09 15:05:15.731783 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:15.731599 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f412c893-a405-4dbf-b710-678a4f373ac7","Type":"ContainerStarted","Data":"b03aee4bbdd56357bb267e00c8847e437bb81414e2c2c37ec369bf558b7087bd"} Jul 09 15:05:15.828062 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:15.828030 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6a54937-6cfe-4b4e-9860-0af0729aea4a" path="/var/lib/kubelet/pods/d6a54937-6cfe-4b4e-9860-0af0729aea4a/volumes" Jul 09 15:05:16.737142 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:16.737106 2567 generic.go:356] "Generic (PLEG): container finished" podID="f412c893-a405-4dbf-b710-678a4f373ac7" containerID="0d3930a097611394eb1abb59b2a952cbb0ad5fda3d50f0491bedb3ad44e63dcb" exitCode=0 Jul 09 15:05:16.737529 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:16.737198 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f412c893-a405-4dbf-b710-678a4f373ac7","Type":"ContainerDied","Data":"0d3930a097611394eb1abb59b2a952cbb0ad5fda3d50f0491bedb3ad44e63dcb"} Jul 09 15:05:16.739326 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:16.739302 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-c6854b798-q486f" event={"ID":"49820873-b1af-45a8-aed3-24c2d227f933","Type":"ContainerStarted","Data":"8a5d175a20cad8a7736c58c3bfaa091793daa4a09ae19add257b8ac3e4c9115d"} Jul 09 15:05:16.739451 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:16.739359 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-c6854b798-q486f" event={"ID":"49820873-b1af-45a8-aed3-24c2d227f933","Type":"ContainerStarted","Data":"58dc8a0d1a0e61679f4a16038379a487952a037a907f2f5205ce7acf87f8ef06"} Jul 09 15:05:16.739451 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:16.739385 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-c6854b798-q486f" event={"ID":"49820873-b1af-45a8-aed3-24c2d227f933","Type":"ContainerStarted","Data":"79b4a72b2297c77a90fb66174a6e9a049f96b0dbc3c6c2f38092faa04ba2b351"} Jul 09 15:05:16.805484 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:16.805434 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-c6854b798-q486f" podStartSLOduration=2.127865374 podStartE2EDuration="3.805418455s" totalImagesPullingTime="1.677553081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:05:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:05:16.802546985 +0000 UTC m=+163.533053481" watchObservedRunningTime="2026-07-09 15:05:16.805418455 +0000 UTC m=+163.535924946" Jul 09 15:05:17.701755 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:17.701711 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:05:17.749299 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:17.749256 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f412c893-a405-4dbf-b710-678a4f373ac7","Type":"ContainerStarted","Data":"43b454cdd3b2480c068b921e7e12118a2886f4ddc20b1d0d7e5dfc382877874f"} Jul 09 15:05:17.749763 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:17.749384 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f412c893-a405-4dbf-b710-678a4f373ac7","Type":"ContainerStarted","Data":"404aa7109e1958c2898136629fe48a70a2539b006df2c68cc6f3b5e4cb0e1fb3"} Jul 09 15:05:17.749763 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:17.749432 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f412c893-a405-4dbf-b710-678a4f373ac7","Type":"ContainerStarted","Data":"e1f7c156d00a61972c45f170409846b8a292204ada2c44b590e8760625850a36"} Jul 09 15:05:17.749763 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:17.749476 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f412c893-a405-4dbf-b710-678a4f373ac7","Type":"ContainerStarted","Data":"8d80da4274cb9b791cc9eb0a0466e60f6764c2a21fc53d8ddd1b8bdb9fbc339d"} Jul 09 15:05:17.749763 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:17.749513 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f412c893-a405-4dbf-b710-678a4f373ac7","Type":"ContainerStarted","Data":"294a2bdc9ffecc44667d42a53c81eaff7afd53dc06c46de44cd8a1464a0a6983"} Jul 09 15:05:17.749763 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:17.749553 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f412c893-a405-4dbf-b710-678a4f373ac7","Type":"ContainerStarted","Data":"71e68c3a6f5f0c0663a99c8ca2146687272057add157143c82055aa078a1a409"} Jul 09 15:05:17.787077 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:17.787010 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.76106157 podStartE2EDuration="3.786985675s" totalImagesPullingTime="0s" totalInitContainerRuntime="25.924105ms" isStatefulPod=false podCreationTimestamp="2026-07-09 15:05:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:05:17.78550459 +0000 UTC m=+164.516011086" watchObservedRunningTime="2026-07-09 15:05:17.786985675 +0000 UTC m=+164.517492178" Jul 09 15:05:17.791537 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:17.791497 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:05:20.117282 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:20.117239 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:05:28.830059 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:28.829977 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:05:31.103042 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:31.103009 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:05:31.189088 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:31.189058 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:05:58.326895 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:58.326852 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:05:58.401799 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:05:58.401764 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:06:15.117231 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:06:15.117181 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:06:15.132969 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:06:15.132941 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:06:16.006665 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:06:16.006638 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 15:06:23.679361 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:06:23.679328 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:06:23.746664 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:06:23.746593 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:06:25.814712 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:06:25.814671 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:06:25.913724 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:06:25.913680 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:06:35.656290 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:06:35.656251 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:06:35.733864 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:06:35.733831 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:06:37.071898 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:06:37.071858 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:06:37.172279 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:06:37.172245 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:06:37.235274 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:06:37.235237 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:06:44.708389 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:06:44.708356 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:06:51.823197 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:06:51.823158 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:06:51.923367 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:06:51.923328 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:06:52.000277 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:06:52.000242 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:07:33.668910 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:07:33.668868 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hml7m_84a10578-4913-456b-9fb7-e6440914b677/ovn-acl-logging/0.log" Jul 09 15:07:33.669413 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:07:33.669389 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hml7m_84a10578-4913-456b-9fb7-e6440914b677/ovn-acl-logging/0.log" Jul 09 15:07:33.673882 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:07:33.673861 2567 kubelet.go:1648] "Image garbage collection succeeded" Jul 09 15:08:11.600737 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:08:11.600693 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:08:15.147075 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:08:15.147036 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:08:18.026856 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:08:18.026810 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:08:18.093263 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:08:18.093224 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:08:42.597363 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:08:42.597271 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:08:43.398248 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:08:43.398216 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:08:46.224203 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:08:46.224160 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:08:46.324913 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:08:46.324872 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:08:49.871965 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:08:49.871933 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:08:49.992683 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:08:49.992644 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:08:51.585567 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:08:51.585529 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:08:51.691617 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:08:51.691583 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:08:51.998830 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:08:51.998775 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:08:52.452746 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:08:52.452645 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:08:56.160270 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:08:56.160233 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-894hm"] Jul 09 15:08:56.162628 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:08:56.162608 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-894hm" Jul 09 15:08:56.165292 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:08:56.165269 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 09 15:08:56.165522 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:08:56.165493 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 09 15:08:56.166482 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:08:56.166459 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-mklzn\"" Jul 09 15:08:56.174557 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:08:56.174533 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-894hm"] Jul 09 15:08:56.228934 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:08:56.228887 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nk28\" (UniqueName: \"kubernetes.io/projected/25ba827d-a61d-4c55-8f38-145aa46c88ca-kube-api-access-6nk28\") pod \"cert-manager-cainjector-559cdd8c87-894hm\" (UID: \"25ba827d-a61d-4c55-8f38-145aa46c88ca\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-894hm" Jul 09 15:08:56.229115 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:08:56.228966 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/25ba827d-a61d-4c55-8f38-145aa46c88ca-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-894hm\" (UID: \"25ba827d-a61d-4c55-8f38-145aa46c88ca\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-894hm" Jul 09 15:08:56.330235 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:08:56.330184 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nk28\" (UniqueName: \"kubernetes.io/projected/25ba827d-a61d-4c55-8f38-145aa46c88ca-kube-api-access-6nk28\") pod \"cert-manager-cainjector-559cdd8c87-894hm\" (UID: \"25ba827d-a61d-4c55-8f38-145aa46c88ca\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-894hm" Jul 09 15:08:56.330427 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:08:56.330246 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/25ba827d-a61d-4c55-8f38-145aa46c88ca-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-894hm\" (UID: \"25ba827d-a61d-4c55-8f38-145aa46c88ca\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-894hm" Jul 09 15:08:56.339523 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:08:56.339486 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/25ba827d-a61d-4c55-8f38-145aa46c88ca-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-894hm\" (UID: \"25ba827d-a61d-4c55-8f38-145aa46c88ca\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-894hm" Jul 09 15:08:56.339652 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:08:56.339586 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nk28\" (UniqueName: \"kubernetes.io/projected/25ba827d-a61d-4c55-8f38-145aa46c88ca-kube-api-access-6nk28\") pod \"cert-manager-cainjector-559cdd8c87-894hm\" (UID: \"25ba827d-a61d-4c55-8f38-145aa46c88ca\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-894hm" Jul 09 15:08:56.480922 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:08:56.480893 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-894hm" Jul 09 15:08:56.613729 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:08:56.613705 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-894hm"] Jul 09 15:08:56.616394 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:08:56.616365 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod25ba827d_a61d_4c55_8f38_145aa46c88ca.slice/crio-381df5e16eb9519b013445bc0786c88ab55a7699ed5f9214e553bc307d6ed8c5 WatchSource:0}: Error finding container 381df5e16eb9519b013445bc0786c88ab55a7699ed5f9214e553bc307d6ed8c5: Status 404 returned error can't find the container with id 381df5e16eb9519b013445bc0786c88ab55a7699ed5f9214e553bc307d6ed8c5 Jul 09 15:08:56.618558 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:08:56.618542 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 15:08:56.635119 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:08:56.635084 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-894hm" event={"ID":"25ba827d-a61d-4c55-8f38-145aa46c88ca","Type":"ContainerStarted","Data":"381df5e16eb9519b013445bc0786c88ab55a7699ed5f9214e553bc307d6ed8c5"} Jul 09 15:08:59.441207 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:08:59.441156 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:08:59.977518 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:08:59.977484 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-p6zj5"] Jul 09 15:08:59.981603 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:08:59.981579 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-p6zj5" Jul 09 15:08:59.984371 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:08:59.984325 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-xkwqq\"" Jul 09 15:08:59.994434 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:08:59.994379 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-p6zj5"] Jul 09 15:09:00.066344 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:00.066289 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrtws\" (UniqueName: \"kubernetes.io/projected/920a7bb4-9d94-44d5-9084-5c2dfee13ebe-kube-api-access-qrtws\") pod \"cert-manager-55b4d954d5-p6zj5\" (UID: \"920a7bb4-9d94-44d5-9084-5c2dfee13ebe\") " pod="cert-manager/cert-manager-55b4d954d5-p6zj5" Jul 09 15:09:00.066518 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:00.066355 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/920a7bb4-9d94-44d5-9084-5c2dfee13ebe-bound-sa-token\") pod \"cert-manager-55b4d954d5-p6zj5\" (UID: \"920a7bb4-9d94-44d5-9084-5c2dfee13ebe\") " pod="cert-manager/cert-manager-55b4d954d5-p6zj5" Jul 09 15:09:00.167161 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:00.167122 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrtws\" (UniqueName: \"kubernetes.io/projected/920a7bb4-9d94-44d5-9084-5c2dfee13ebe-kube-api-access-qrtws\") pod \"cert-manager-55b4d954d5-p6zj5\" (UID: \"920a7bb4-9d94-44d5-9084-5c2dfee13ebe\") " pod="cert-manager/cert-manager-55b4d954d5-p6zj5" Jul 09 15:09:00.167358 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:00.167186 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/920a7bb4-9d94-44d5-9084-5c2dfee13ebe-bound-sa-token\") pod \"cert-manager-55b4d954d5-p6zj5\" (UID: \"920a7bb4-9d94-44d5-9084-5c2dfee13ebe\") " pod="cert-manager/cert-manager-55b4d954d5-p6zj5" Jul 09 15:09:00.178866 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:00.178810 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/920a7bb4-9d94-44d5-9084-5c2dfee13ebe-bound-sa-token\") pod \"cert-manager-55b4d954d5-p6zj5\" (UID: \"920a7bb4-9d94-44d5-9084-5c2dfee13ebe\") " pod="cert-manager/cert-manager-55b4d954d5-p6zj5" Jul 09 15:09:00.179039 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:00.178911 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrtws\" (UniqueName: \"kubernetes.io/projected/920a7bb4-9d94-44d5-9084-5c2dfee13ebe-kube-api-access-qrtws\") pod \"cert-manager-55b4d954d5-p6zj5\" (UID: \"920a7bb4-9d94-44d5-9084-5c2dfee13ebe\") " pod="cert-manager/cert-manager-55b4d954d5-p6zj5" Jul 09 15:09:00.295415 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:00.295324 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-p6zj5" Jul 09 15:09:01.690457 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:01.690422 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-p6zj5"] Jul 09 15:09:01.693677 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:09:01.693649 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod920a7bb4_9d94_44d5_9084_5c2dfee13ebe.slice/crio-feb15e2ac621aa70f6992b4187985624436ed229b4f40e24fe2d731e8c264062 WatchSource:0}: Error finding container feb15e2ac621aa70f6992b4187985624436ed229b4f40e24fe2d731e8c264062: Status 404 returned error can't find the container with id feb15e2ac621aa70f6992b4187985624436ed229b4f40e24fe2d731e8c264062 Jul 09 15:09:02.166180 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:02.166141 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:02.264853 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:02.264783 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:02.403898 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:02.403856 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:02.505149 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:02.505109 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:02.670035 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:02.669995 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-p6zj5" event={"ID":"920a7bb4-9d94-44d5-9084-5c2dfee13ebe","Type":"ContainerStarted","Data":"edf6a3fdc1947442dc1455a2d0d9775e212fc7b924384195b0ca5b919294f545"} Jul 09 15:09:02.670224 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:02.670040 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-p6zj5" event={"ID":"920a7bb4-9d94-44d5-9084-5c2dfee13ebe","Type":"ContainerStarted","Data":"feb15e2ac621aa70f6992b4187985624436ed229b4f40e24fe2d731e8c264062"} Jul 09 15:09:02.671675 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:02.671648 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-894hm" event={"ID":"25ba827d-a61d-4c55-8f38-145aa46c88ca","Type":"ContainerStarted","Data":"ed3eb40861dcf8e4eff4dc704a30aa0c1a6d4f34e5500fbaa2ffed59054e397c"} Jul 09 15:09:02.693571 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:02.693517 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-p6zj5" podStartSLOduration=3.693499426 podStartE2EDuration="3.693499426s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:08:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:09:02.691552075 +0000 UTC m=+389.422058578" watchObservedRunningTime="2026-07-09 15:09:02.693499426 +0000 UTC m=+389.424005916" Jul 09 15:09:02.716849 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:02.716763 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-894hm" podStartSLOduration=1.744060271 podStartE2EDuration="6.716737328s" totalImagesPullingTime="4.972677057s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:08:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:09:02.715958354 +0000 UTC m=+389.446464869" watchObservedRunningTime="2026-07-09 15:09:02.716737328 +0000 UTC m=+389.447243822" Jul 09 15:09:08.262241 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:08.262204 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:08.357926 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:08.357889 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:08.460388 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:08.460352 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:10.317267 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:10.317230 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:22.406061 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:22.406017 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:22.566009 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:22.565975 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:22.718311 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:22.718276 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:23.110370 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:23.110248 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:23.273482 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:23.273451 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66b8fbd475-29fz2"] Jul 09 15:09:23.276065 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:23.276050 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-29fz2" Jul 09 15:09:23.279271 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:23.279243 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 09 15:09:23.279426 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:23.279410 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 09 15:09:23.279645 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:23.279628 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 09 15:09:23.279876 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:23.279861 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 09 15:09:23.282349 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:23.282333 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-5hbzl\"" Jul 09 15:09:23.319945 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:23.319916 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66b8fbd475-29fz2"] Jul 09 15:09:23.376281 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:23.376180 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0461189c-aa86-4d9e-b865-ab27bf0830ef-tmp\") pod \"opendatahub-operator-controller-manager-66b8fbd475-29fz2\" (UID: \"0461189c-aa86-4d9e-b865-ab27bf0830ef\") " pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-29fz2" Jul 09 15:09:23.376281 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:23.376228 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0461189c-aa86-4d9e-b865-ab27bf0830ef-webhook-cert\") pod \"opendatahub-operator-controller-manager-66b8fbd475-29fz2\" (UID: \"0461189c-aa86-4d9e-b865-ab27bf0830ef\") " pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-29fz2" Jul 09 15:09:23.376281 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:23.376249 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lvl9\" (UniqueName: \"kubernetes.io/projected/0461189c-aa86-4d9e-b865-ab27bf0830ef-kube-api-access-6lvl9\") pod \"opendatahub-operator-controller-manager-66b8fbd475-29fz2\" (UID: \"0461189c-aa86-4d9e-b865-ab27bf0830ef\") " pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-29fz2" Jul 09 15:09:23.376535 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:23.376360 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0461189c-aa86-4d9e-b865-ab27bf0830ef-manifests\") pod \"opendatahub-operator-controller-manager-66b8fbd475-29fz2\" (UID: \"0461189c-aa86-4d9e-b865-ab27bf0830ef\") " pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-29fz2" Jul 09 15:09:23.376535 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:23.376474 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0461189c-aa86-4d9e-b865-ab27bf0830ef-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66b8fbd475-29fz2\" (UID: \"0461189c-aa86-4d9e-b865-ab27bf0830ef\") " pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-29fz2" Jul 09 15:09:23.409737 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:23.409703 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:23.477895 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:23.477850 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0461189c-aa86-4d9e-b865-ab27bf0830ef-manifests\") pod \"opendatahub-operator-controller-manager-66b8fbd475-29fz2\" (UID: \"0461189c-aa86-4d9e-b865-ab27bf0830ef\") " pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-29fz2" Jul 09 15:09:23.478072 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:23.477945 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0461189c-aa86-4d9e-b865-ab27bf0830ef-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66b8fbd475-29fz2\" (UID: \"0461189c-aa86-4d9e-b865-ab27bf0830ef\") " pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-29fz2" Jul 09 15:09:23.478072 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:23.477973 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0461189c-aa86-4d9e-b865-ab27bf0830ef-tmp\") pod \"opendatahub-operator-controller-manager-66b8fbd475-29fz2\" (UID: \"0461189c-aa86-4d9e-b865-ab27bf0830ef\") " pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-29fz2" Jul 09 15:09:23.478072 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:23.477998 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0461189c-aa86-4d9e-b865-ab27bf0830ef-webhook-cert\") pod \"opendatahub-operator-controller-manager-66b8fbd475-29fz2\" (UID: \"0461189c-aa86-4d9e-b865-ab27bf0830ef\") " pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-29fz2" Jul 09 15:09:23.478072 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:23.478024 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lvl9\" (UniqueName: \"kubernetes.io/projected/0461189c-aa86-4d9e-b865-ab27bf0830ef-kube-api-access-6lvl9\") pod \"opendatahub-operator-controller-manager-66b8fbd475-29fz2\" (UID: \"0461189c-aa86-4d9e-b865-ab27bf0830ef\") " pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-29fz2" Jul 09 15:09:23.478316 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:23.478292 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0461189c-aa86-4d9e-b865-ab27bf0830ef-manifests\") pod \"opendatahub-operator-controller-manager-66b8fbd475-29fz2\" (UID: \"0461189c-aa86-4d9e-b865-ab27bf0830ef\") " pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-29fz2" Jul 09 15:09:23.480364 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:23.480330 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0461189c-aa86-4d9e-b865-ab27bf0830ef-tmp\") pod \"opendatahub-operator-controller-manager-66b8fbd475-29fz2\" (UID: \"0461189c-aa86-4d9e-b865-ab27bf0830ef\") " pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-29fz2" Jul 09 15:09:23.480497 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:23.480439 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0461189c-aa86-4d9e-b865-ab27bf0830ef-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66b8fbd475-29fz2\" (UID: \"0461189c-aa86-4d9e-b865-ab27bf0830ef\") " pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-29fz2" Jul 09 15:09:23.480497 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:23.480463 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0461189c-aa86-4d9e-b865-ab27bf0830ef-webhook-cert\") pod \"opendatahub-operator-controller-manager-66b8fbd475-29fz2\" (UID: \"0461189c-aa86-4d9e-b865-ab27bf0830ef\") " pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-29fz2" Jul 09 15:09:23.487897 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:23.487869 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lvl9\" (UniqueName: \"kubernetes.io/projected/0461189c-aa86-4d9e-b865-ab27bf0830ef-kube-api-access-6lvl9\") pod \"opendatahub-operator-controller-manager-66b8fbd475-29fz2\" (UID: \"0461189c-aa86-4d9e-b865-ab27bf0830ef\") " pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-29fz2" Jul 09 15:09:23.509084 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:23.509048 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:23.587309 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:23.587274 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-29fz2" Jul 09 15:09:23.619870 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:23.619833 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:23.749412 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:23.749383 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66b8fbd475-29fz2"] Jul 09 15:09:23.752882 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:09:23.752850 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0461189c_aa86_4d9e_b865_ab27bf0830ef.slice/crio-64c01276c854ff7440950f2b2d95bd3ec563f4bb80914c2cb6941e42b0108372 WatchSource:0}: Error finding container 64c01276c854ff7440950f2b2d95bd3ec563f4bb80914c2cb6941e42b0108372: Status 404 returned error can't find the container with id 64c01276c854ff7440950f2b2d95bd3ec563f4bb80914c2cb6941e42b0108372 Jul 09 15:09:24.533945 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:24.533901 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:24.652288 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:24.652249 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:24.766636 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:24.766599 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-29fz2" event={"ID":"0461189c-aa86-4d9e-b865-ab27bf0830ef","Type":"ContainerStarted","Data":"64c01276c854ff7440950f2b2d95bd3ec563f4bb80914c2cb6941e42b0108372"} Jul 09 15:09:27.790137 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:27.790098 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-29fz2" event={"ID":"0461189c-aa86-4d9e-b865-ab27bf0830ef","Type":"ContainerStarted","Data":"a247829bf5dd16aa7ed0822b3004c263e8b3ac68a92cbd63602905ab10f427c9"} Jul 09 15:09:28.796214 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:28.796182 2567 generic.go:356] "Generic (PLEG): container finished" podID="0461189c-aa86-4d9e-b865-ab27bf0830ef" containerID="a247829bf5dd16aa7ed0822b3004c263e8b3ac68a92cbd63602905ab10f427c9" exitCode=0 Jul 09 15:09:28.796619 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:28.796269 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-29fz2" event={"ID":"0461189c-aa86-4d9e-b865-ab27bf0830ef","Type":"ContainerDied","Data":"a247829bf5dd16aa7ed0822b3004c263e8b3ac68a92cbd63602905ab10f427c9"} Jul 09 15:09:29.803363 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:29.803331 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-29fz2" event={"ID":"0461189c-aa86-4d9e-b865-ab27bf0830ef","Type":"ContainerStarted","Data":"879b1f0df18995676a04920a779525b8c20843f5f7c89d840ff550d4c8427cb4"} Jul 09 15:09:29.803745 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:29.803478 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-29fz2" Jul 09 15:09:29.841186 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:29.841135 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-29fz2" podStartSLOduration=2.166638446 podStartE2EDuration="6.841117953s" totalImagesPullingTime="3.622647304s" totalInitContainerRuntime="1.051832203s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:09:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:09:29.840238905 +0000 UTC m=+416.570745395" watchObservedRunningTime="2026-07-09 15:09:29.841117953 +0000 UTC m=+416.571624443" Jul 09 15:09:33.562234 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:33.562196 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:33.674660 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:33.674615 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:40.810645 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:40.810608 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-29fz2" Jul 09 15:09:42.611981 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:42.611947 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:42.731698 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:42.731659 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:43.600409 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:43.600360 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:43.724426 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:43.724397 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:43.893114 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:43.893021 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:44.440873 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:44.440833 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:44.815611 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:44.815571 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:44.906979 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:44.906948 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:44.999264 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:44.999231 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:46.130872 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:46.130833 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:50.859686 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:50.859649 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:50.973714 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:50.973684 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:51.710080 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:51.710051 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb"] Jul 09 15:09:51.717910 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:51.717883 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb" Jul 09 15:09:51.721343 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:51.721319 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 09 15:09:51.721468 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:51.721391 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-xr7kh\"" Jul 09 15:09:51.721738 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:51.721723 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 09 15:09:51.721803 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:51.721787 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 09 15:09:51.731418 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:51.731396 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb"] Jul 09 15:09:51.778078 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:51.778043 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:51.847832 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:51.847790 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e7db3e0-a93f-456c-b4a3-395682060224-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j6xdb\" (UID: \"8e7db3e0-a93f-456c-b4a3-395682060224\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb" Jul 09 15:09:51.848031 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:51.847844 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e7db3e0-a93f-456c-b4a3-395682060224-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j6xdb\" (UID: \"8e7db3e0-a93f-456c-b4a3-395682060224\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb" Jul 09 15:09:51.848031 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:51.847920 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e7db3e0-a93f-456c-b4a3-395682060224-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j6xdb\" (UID: \"8e7db3e0-a93f-456c-b4a3-395682060224\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb" Jul 09 15:09:51.848031 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:51.847959 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e7db3e0-a93f-456c-b4a3-395682060224-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j6xdb\" (UID: \"8e7db3e0-a93f-456c-b4a3-395682060224\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb" Jul 09 15:09:51.848031 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:51.847975 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e7db3e0-a93f-456c-b4a3-395682060224-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j6xdb\" (UID: \"8e7db3e0-a93f-456c-b4a3-395682060224\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb" Jul 09 15:09:51.848031 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:51.848028 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e7db3e0-a93f-456c-b4a3-395682060224-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j6xdb\" (UID: \"8e7db3e0-a93f-456c-b4a3-395682060224\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb" Jul 09 15:09:51.848233 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:51.848050 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e7db3e0-a93f-456c-b4a3-395682060224-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j6xdb\" (UID: \"8e7db3e0-a93f-456c-b4a3-395682060224\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb" Jul 09 15:09:51.848233 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:51.848067 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e7db3e0-a93f-456c-b4a3-395682060224-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j6xdb\" (UID: \"8e7db3e0-a93f-456c-b4a3-395682060224\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb" Jul 09 15:09:51.848233 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:51.848083 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzv8l\" (UniqueName: \"kubernetes.io/projected/8e7db3e0-a93f-456c-b4a3-395682060224-kube-api-access-wzv8l\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j6xdb\" (UID: \"8e7db3e0-a93f-456c-b4a3-395682060224\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb" Jul 09 15:09:51.948604 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:51.948568 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e7db3e0-a93f-456c-b4a3-395682060224-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j6xdb\" (UID: \"8e7db3e0-a93f-456c-b4a3-395682060224\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb" Jul 09 15:09:51.948604 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:51.948609 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e7db3e0-a93f-456c-b4a3-395682060224-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j6xdb\" (UID: \"8e7db3e0-a93f-456c-b4a3-395682060224\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb" Jul 09 15:09:51.949193 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:51.948625 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e7db3e0-a93f-456c-b4a3-395682060224-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j6xdb\" (UID: \"8e7db3e0-a93f-456c-b4a3-395682060224\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb" Jul 09 15:09:51.949193 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:51.948642 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzv8l\" (UniqueName: \"kubernetes.io/projected/8e7db3e0-a93f-456c-b4a3-395682060224-kube-api-access-wzv8l\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j6xdb\" (UID: \"8e7db3e0-a93f-456c-b4a3-395682060224\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb" Jul 09 15:09:51.949193 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:51.948693 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e7db3e0-a93f-456c-b4a3-395682060224-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j6xdb\" (UID: \"8e7db3e0-a93f-456c-b4a3-395682060224\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb" Jul 09 15:09:51.949193 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:51.948718 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e7db3e0-a93f-456c-b4a3-395682060224-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j6xdb\" (UID: \"8e7db3e0-a93f-456c-b4a3-395682060224\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb" Jul 09 15:09:51.949193 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:51.948786 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e7db3e0-a93f-456c-b4a3-395682060224-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j6xdb\" (UID: \"8e7db3e0-a93f-456c-b4a3-395682060224\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb" Jul 09 15:09:51.949193 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:51.948849 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e7db3e0-a93f-456c-b4a3-395682060224-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j6xdb\" (UID: \"8e7db3e0-a93f-456c-b4a3-395682060224\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb" Jul 09 15:09:51.949193 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:51.948872 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e7db3e0-a93f-456c-b4a3-395682060224-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j6xdb\" (UID: \"8e7db3e0-a93f-456c-b4a3-395682060224\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb" Jul 09 15:09:51.949193 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:51.949117 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e7db3e0-a93f-456c-b4a3-395682060224-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j6xdb\" (UID: \"8e7db3e0-a93f-456c-b4a3-395682060224\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb" Jul 09 15:09:51.949606 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:51.949232 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e7db3e0-a93f-456c-b4a3-395682060224-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j6xdb\" (UID: \"8e7db3e0-a93f-456c-b4a3-395682060224\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb" Jul 09 15:09:51.949606 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:51.949320 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e7db3e0-a93f-456c-b4a3-395682060224-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j6xdb\" (UID: \"8e7db3e0-a93f-456c-b4a3-395682060224\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb" Jul 09 15:09:51.949606 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:51.949333 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e7db3e0-a93f-456c-b4a3-395682060224-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j6xdb\" (UID: \"8e7db3e0-a93f-456c-b4a3-395682060224\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb" Jul 09 15:09:51.949606 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:51.949416 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e7db3e0-a93f-456c-b4a3-395682060224-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j6xdb\" (UID: \"8e7db3e0-a93f-456c-b4a3-395682060224\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb" Jul 09 15:09:51.950946 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:51.950924 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e7db3e0-a93f-456c-b4a3-395682060224-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j6xdb\" (UID: \"8e7db3e0-a93f-456c-b4a3-395682060224\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb" Jul 09 15:09:51.951215 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:51.951198 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e7db3e0-a93f-456c-b4a3-395682060224-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j6xdb\" (UID: \"8e7db3e0-a93f-456c-b4a3-395682060224\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb" Jul 09 15:09:51.955567 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:51.955540 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:51.963158 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:51.963099 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzv8l\" (UniqueName: \"kubernetes.io/projected/8e7db3e0-a93f-456c-b4a3-395682060224-kube-api-access-wzv8l\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j6xdb\" (UID: \"8e7db3e0-a93f-456c-b4a3-395682060224\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb" Jul 09 15:09:51.967410 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:51.967390 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e7db3e0-a93f-456c-b4a3-395682060224-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j6xdb\" (UID: \"8e7db3e0-a93f-456c-b4a3-395682060224\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb" Jul 09 15:09:52.028725 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:52.028692 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb" Jul 09 15:09:52.172761 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:52.172727 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:52.181496 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:52.181464 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb"] Jul 09 15:09:52.396534 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:52.396499 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7dcd8df84f-6dwp5"] Jul 09 15:09:52.399549 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:52.399532 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7dcd8df84f-6dwp5" Jul 09 15:09:52.403625 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:52.403600 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 09 15:09:52.403750 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:52.403661 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-zgndh\"" Jul 09 15:09:52.403893 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:52.403879 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 09 15:09:52.404201 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:52.404173 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 09 15:09:52.405744 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:52.405725 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 09 15:09:52.406328 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:52.406307 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 09 15:09:52.411368 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:52.411350 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7dcd8df84f-6dwp5"] Jul 09 15:09:52.554026 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:52.553933 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/706796e9-a59f-4675-88d4-37545a6d864b-manager-config\") pod \"lws-controller-manager-7dcd8df84f-6dwp5\" (UID: \"706796e9-a59f-4675-88d4-37545a6d864b\") " pod="openshift-lws-operator/lws-controller-manager-7dcd8df84f-6dwp5" Jul 09 15:09:52.554176 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:52.554044 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/706796e9-a59f-4675-88d4-37545a6d864b-cert\") pod \"lws-controller-manager-7dcd8df84f-6dwp5\" (UID: \"706796e9-a59f-4675-88d4-37545a6d864b\") " pod="openshift-lws-operator/lws-controller-manager-7dcd8df84f-6dwp5" Jul 09 15:09:52.554176 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:52.554093 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/706796e9-a59f-4675-88d4-37545a6d864b-metrics-cert\") pod \"lws-controller-manager-7dcd8df84f-6dwp5\" (UID: \"706796e9-a59f-4675-88d4-37545a6d864b\") " pod="openshift-lws-operator/lws-controller-manager-7dcd8df84f-6dwp5" Jul 09 15:09:52.554176 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:52.554116 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfjf6\" (UniqueName: \"kubernetes.io/projected/706796e9-a59f-4675-88d4-37545a6d864b-kube-api-access-gfjf6\") pod \"lws-controller-manager-7dcd8df84f-6dwp5\" (UID: \"706796e9-a59f-4675-88d4-37545a6d864b\") " pod="openshift-lws-operator/lws-controller-manager-7dcd8df84f-6dwp5" Jul 09 15:09:52.654803 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:52.654762 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/706796e9-a59f-4675-88d4-37545a6d864b-cert\") pod \"lws-controller-manager-7dcd8df84f-6dwp5\" (UID: \"706796e9-a59f-4675-88d4-37545a6d864b\") " pod="openshift-lws-operator/lws-controller-manager-7dcd8df84f-6dwp5" Jul 09 15:09:52.655023 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:52.654838 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/706796e9-a59f-4675-88d4-37545a6d864b-metrics-cert\") pod \"lws-controller-manager-7dcd8df84f-6dwp5\" (UID: \"706796e9-a59f-4675-88d4-37545a6d864b\") " pod="openshift-lws-operator/lws-controller-manager-7dcd8df84f-6dwp5" Jul 09 15:09:52.655097 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:52.655066 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfjf6\" (UniqueName: \"kubernetes.io/projected/706796e9-a59f-4675-88d4-37545a6d864b-kube-api-access-gfjf6\") pod \"lws-controller-manager-7dcd8df84f-6dwp5\" (UID: \"706796e9-a59f-4675-88d4-37545a6d864b\") " pod="openshift-lws-operator/lws-controller-manager-7dcd8df84f-6dwp5" Jul 09 15:09:52.655162 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:52.655150 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/706796e9-a59f-4675-88d4-37545a6d864b-manager-config\") pod \"lws-controller-manager-7dcd8df84f-6dwp5\" (UID: \"706796e9-a59f-4675-88d4-37545a6d864b\") " pod="openshift-lws-operator/lws-controller-manager-7dcd8df84f-6dwp5" Jul 09 15:09:52.655789 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:52.655769 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/706796e9-a59f-4675-88d4-37545a6d864b-manager-config\") pod \"lws-controller-manager-7dcd8df84f-6dwp5\" (UID: \"706796e9-a59f-4675-88d4-37545a6d864b\") " pod="openshift-lws-operator/lws-controller-manager-7dcd8df84f-6dwp5" Jul 09 15:09:52.657290 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:52.657266 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/706796e9-a59f-4675-88d4-37545a6d864b-cert\") pod \"lws-controller-manager-7dcd8df84f-6dwp5\" (UID: \"706796e9-a59f-4675-88d4-37545a6d864b\") " pod="openshift-lws-operator/lws-controller-manager-7dcd8df84f-6dwp5" Jul 09 15:09:52.657372 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:52.657299 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/706796e9-a59f-4675-88d4-37545a6d864b-metrics-cert\") pod \"lws-controller-manager-7dcd8df84f-6dwp5\" (UID: \"706796e9-a59f-4675-88d4-37545a6d864b\") " pod="openshift-lws-operator/lws-controller-manager-7dcd8df84f-6dwp5" Jul 09 15:09:52.671504 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:52.671479 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfjf6\" (UniqueName: \"kubernetes.io/projected/706796e9-a59f-4675-88d4-37545a6d864b-kube-api-access-gfjf6\") pod \"lws-controller-manager-7dcd8df84f-6dwp5\" (UID: \"706796e9-a59f-4675-88d4-37545a6d864b\") " pod="openshift-lws-operator/lws-controller-manager-7dcd8df84f-6dwp5" Jul 09 15:09:52.710927 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:52.710893 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7dcd8df84f-6dwp5" Jul 09 15:09:52.842062 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:52.841977 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7dcd8df84f-6dwp5"] Jul 09 15:09:52.845016 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:09:52.844974 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod706796e9_a59f_4675_88d4_37545a6d864b.slice/crio-760680610ac533d1c6b679f56da7eb3b377f44528a81ba0e2c6d637f8c165677 WatchSource:0}: Error finding container 760680610ac533d1c6b679f56da7eb3b377f44528a81ba0e2c6d637f8c165677: Status 404 returned error can't find the container with id 760680610ac533d1c6b679f56da7eb3b377f44528a81ba0e2c6d637f8c165677 Jul 09 15:09:52.906065 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:52.906024 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7dcd8df84f-6dwp5" event={"ID":"706796e9-a59f-4675-88d4-37545a6d864b","Type":"ContainerStarted","Data":"760680610ac533d1c6b679f56da7eb3b377f44528a81ba0e2c6d637f8c165677"} Jul 09 15:09:52.907182 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:52.907158 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb" event={"ID":"8e7db3e0-a93f-456c-b4a3-395682060224","Type":"ContainerStarted","Data":"9e7ff13e614eb70ac4fc7bae6b4f328ac5738c46b00f0c7f026cfc1ddca67a09"} Jul 09 15:09:54.391454 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:54.391417 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:54.504771 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:54.504565 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-p44qb"] Jul 09 15:09:54.507945 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:54.507915 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-p44qb" Jul 09 15:09:54.512369 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:54.512343 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 09 15:09:54.512597 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:54.512582 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-4lgpn\"" Jul 09 15:09:54.524247 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:54.524216 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-p44qb"] Jul 09 15:09:54.573551 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:54.572799 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xtcg\" (UniqueName: \"kubernetes.io/projected/026a46f5-a5f0-4cfc-95d6-a4e920ac37ff-kube-api-access-8xtcg\") pod \"kserve-controller-manager-d6f4b65b8-p44qb\" (UID: \"026a46f5-a5f0-4cfc-95d6-a4e920ac37ff\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-p44qb" Jul 09 15:09:54.573551 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:54.572866 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/026a46f5-a5f0-4cfc-95d6-a4e920ac37ff-cert\") pod \"kserve-controller-manager-d6f4b65b8-p44qb\" (UID: \"026a46f5-a5f0-4cfc-95d6-a4e920ac37ff\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-p44qb" Jul 09 15:09:54.674047 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:54.673960 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xtcg\" (UniqueName: \"kubernetes.io/projected/026a46f5-a5f0-4cfc-95d6-a4e920ac37ff-kube-api-access-8xtcg\") pod \"kserve-controller-manager-d6f4b65b8-p44qb\" (UID: \"026a46f5-a5f0-4cfc-95d6-a4e920ac37ff\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-p44qb" Jul 09 15:09:54.674047 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:54.674017 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/026a46f5-a5f0-4cfc-95d6-a4e920ac37ff-cert\") pod \"kserve-controller-manager-d6f4b65b8-p44qb\" (UID: \"026a46f5-a5f0-4cfc-95d6-a4e920ac37ff\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-p44qb" Jul 09 15:09:54.677184 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:54.677156 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/026a46f5-a5f0-4cfc-95d6-a4e920ac37ff-cert\") pod \"kserve-controller-manager-d6f4b65b8-p44qb\" (UID: \"026a46f5-a5f0-4cfc-95d6-a4e920ac37ff\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-p44qb" Jul 09 15:09:54.692884 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:54.692853 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xtcg\" (UniqueName: \"kubernetes.io/projected/026a46f5-a5f0-4cfc-95d6-a4e920ac37ff-kube-api-access-8xtcg\") pod \"kserve-controller-manager-d6f4b65b8-p44qb\" (UID: \"026a46f5-a5f0-4cfc-95d6-a4e920ac37ff\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-p44qb" Jul 09 15:09:54.829289 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:54.829248 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-p44qb" Jul 09 15:09:55.097735 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:55.097691 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:55.238881 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:55.238843 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:55.800702 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:55.800669 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:55.941896 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:55.941847 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:56.600097 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:56.597424 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:09:56.600097 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:56.597521 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:09:56.600097 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:56.597576 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:09:56.776694 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:56.776444 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-p44qb"] Jul 09 15:09:56.780215 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:09:56.780181 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod026a46f5_a5f0_4cfc_95d6_a4e920ac37ff.slice/crio-d2431776cee457bdb3af87dfacb6ae3ec4481b34db85181b3bd07394fa93bcd7 WatchSource:0}: Error finding container d2431776cee457bdb3af87dfacb6ae3ec4481b34db85181b3bd07394fa93bcd7: Status 404 returned error can't find the container with id d2431776cee457bdb3af87dfacb6ae3ec4481b34db85181b3bd07394fa93bcd7 Jul 09 15:09:56.935338 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:56.935234 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb" event={"ID":"8e7db3e0-a93f-456c-b4a3-395682060224","Type":"ContainerStarted","Data":"5209a6ef9b3267accdde85aa9c2451ce9e2e0f767761a5aef3101a55078046ac"} Jul 09 15:09:56.937017 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:56.936991 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7dcd8df84f-6dwp5" event={"ID":"706796e9-a59f-4675-88d4-37545a6d864b","Type":"ContainerStarted","Data":"745d543bbea89f77ded72d60ec7b683498ff74c8b81a4026d379af4d8924550d"} Jul 09 15:09:56.937162 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:56.937082 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7dcd8df84f-6dwp5" Jul 09 15:09:56.938293 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:56.938264 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-p44qb" event={"ID":"026a46f5-a5f0-4cfc-95d6-a4e920ac37ff","Type":"ContainerStarted","Data":"d2431776cee457bdb3af87dfacb6ae3ec4481b34db85181b3bd07394fa93bcd7"} Jul 09 15:09:57.023481 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:57.023423 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7dcd8df84f-6dwp5" podStartSLOduration=1.216294503 podStartE2EDuration="5.023402912s" totalImagesPullingTime="3.807108409s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:09:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:09:57.020243895 +0000 UTC m=+443.750750386" watchObservedRunningTime="2026-07-09 15:09:57.023402912 +0000 UTC m=+443.753909402" Jul 09 15:09:57.023804 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:57.023775 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb" podStartSLOduration=1.61896064 podStartE2EDuration="6.023762257s" totalImagesPullingTime="4.404801617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:09:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:09:56.982521638 +0000 UTC m=+443.713028140" watchObservedRunningTime="2026-07-09 15:09:57.023762257 +0000 UTC m=+443.754268796" Jul 09 15:09:57.029723 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:57.029696 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb" Jul 09 15:09:57.031037 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:57.031009 2567 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 09 15:09:57.031137 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:57.031057 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb" podUID="8e7db3e0-a93f-456c-b4a3-395682060224" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 09 15:09:58.034344 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:58.034308 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb" Jul 09 15:09:58.792310 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:58.792276 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:09:58.950085 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:58.949991 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb" Jul 09 15:09:58.951178 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:58.951151 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j6xdb" Jul 09 15:09:59.435421 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:09:59.435383 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:10:00.962438 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:00.962387 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-p44qb" event={"ID":"026a46f5-a5f0-4cfc-95d6-a4e920ac37ff","Type":"ContainerStarted","Data":"52e94b82ca306350ee953f4ca58c5d03a2789511b2cd710f9edcd01b31e7fbd9"} Jul 09 15:10:00.962913 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:00.962493 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-p44qb" Jul 09 15:10:00.983106 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:00.983053 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-p44qb" podStartSLOduration=2.94889913 podStartE2EDuration="6.983036449s" totalImagesPullingTime="4.034137319s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:09:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:10:00.98145801 +0000 UTC m=+447.711964500" watchObservedRunningTime="2026-07-09 15:10:00.983036449 +0000 UTC m=+447.713542939" Jul 09 15:10:05.527456 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:05.527418 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:10:05.595795 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:05.595753 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:10:07.945528 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:07.945495 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7dcd8df84f-6dwp5" Jul 09 15:10:26.482318 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:26.482273 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56b7f84594-t25x8"] Jul 09 15:10:26.485873 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:26.485848 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56b7f84594-t25x8" Jul 09 15:10:26.488488 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:26.488464 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 09 15:10:26.488677 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:26.488661 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 09 15:10:26.489418 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:26.489402 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 09 15:10:26.490636 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:26.490607 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-rl5vz\"" Jul 09 15:10:26.490764 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:26.490743 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 09 15:10:26.491154 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:26.491121 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 09 15:10:26.498569 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:26.498550 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 09 15:10:26.502504 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:26.502475 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56b7f84594-t25x8"] Jul 09 15:10:26.547352 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:26.547317 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:10:26.567455 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:26.567420 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9445f493-727e-40b8-9fc6-d8ece0562e11-trusted-ca-bundle\") pod \"console-56b7f84594-t25x8\" (UID: \"9445f493-727e-40b8-9fc6-d8ece0562e11\") " pod="openshift-console/console-56b7f84594-t25x8" Jul 09 15:10:26.567637 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:26.567491 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9445f493-727e-40b8-9fc6-d8ece0562e11-console-config\") pod \"console-56b7f84594-t25x8\" (UID: \"9445f493-727e-40b8-9fc6-d8ece0562e11\") " pod="openshift-console/console-56b7f84594-t25x8" Jul 09 15:10:26.567637 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:26.567572 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9445f493-727e-40b8-9fc6-d8ece0562e11-oauth-serving-cert\") pod \"console-56b7f84594-t25x8\" (UID: \"9445f493-727e-40b8-9fc6-d8ece0562e11\") " pod="openshift-console/console-56b7f84594-t25x8" Jul 09 15:10:26.567637 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:26.567612 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhhc2\" (UniqueName: \"kubernetes.io/projected/9445f493-727e-40b8-9fc6-d8ece0562e11-kube-api-access-nhhc2\") pod \"console-56b7f84594-t25x8\" (UID: \"9445f493-727e-40b8-9fc6-d8ece0562e11\") " pod="openshift-console/console-56b7f84594-t25x8" Jul 09 15:10:26.567810 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:26.567666 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9445f493-727e-40b8-9fc6-d8ece0562e11-console-oauth-config\") pod \"console-56b7f84594-t25x8\" (UID: \"9445f493-727e-40b8-9fc6-d8ece0562e11\") " pod="openshift-console/console-56b7f84594-t25x8" Jul 09 15:10:26.567810 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:26.567709 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9445f493-727e-40b8-9fc6-d8ece0562e11-console-serving-cert\") pod \"console-56b7f84594-t25x8\" (UID: \"9445f493-727e-40b8-9fc6-d8ece0562e11\") " pod="openshift-console/console-56b7f84594-t25x8" Jul 09 15:10:26.567810 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:26.567737 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9445f493-727e-40b8-9fc6-d8ece0562e11-service-ca\") pod \"console-56b7f84594-t25x8\" (UID: \"9445f493-727e-40b8-9fc6-d8ece0562e11\") " pod="openshift-console/console-56b7f84594-t25x8" Jul 09 15:10:26.647788 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:26.647749 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:10:26.668937 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:26.668907 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9445f493-727e-40b8-9fc6-d8ece0562e11-console-config\") pod \"console-56b7f84594-t25x8\" (UID: \"9445f493-727e-40b8-9fc6-d8ece0562e11\") " pod="openshift-console/console-56b7f84594-t25x8" Jul 09 15:10:26.669077 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:26.668948 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9445f493-727e-40b8-9fc6-d8ece0562e11-oauth-serving-cert\") pod \"console-56b7f84594-t25x8\" (UID: \"9445f493-727e-40b8-9fc6-d8ece0562e11\") " pod="openshift-console/console-56b7f84594-t25x8" Jul 09 15:10:26.669077 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:26.668983 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhhc2\" (UniqueName: \"kubernetes.io/projected/9445f493-727e-40b8-9fc6-d8ece0562e11-kube-api-access-nhhc2\") pod \"console-56b7f84594-t25x8\" (UID: \"9445f493-727e-40b8-9fc6-d8ece0562e11\") " pod="openshift-console/console-56b7f84594-t25x8" Jul 09 15:10:26.669077 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:26.669016 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9445f493-727e-40b8-9fc6-d8ece0562e11-console-oauth-config\") pod \"console-56b7f84594-t25x8\" (UID: \"9445f493-727e-40b8-9fc6-d8ece0562e11\") " pod="openshift-console/console-56b7f84594-t25x8" Jul 09 15:10:26.669077 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:26.669041 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9445f493-727e-40b8-9fc6-d8ece0562e11-console-serving-cert\") pod \"console-56b7f84594-t25x8\" (UID: \"9445f493-727e-40b8-9fc6-d8ece0562e11\") " pod="openshift-console/console-56b7f84594-t25x8" Jul 09 15:10:26.669077 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:26.669065 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9445f493-727e-40b8-9fc6-d8ece0562e11-service-ca\") pod \"console-56b7f84594-t25x8\" (UID: \"9445f493-727e-40b8-9fc6-d8ece0562e11\") " pod="openshift-console/console-56b7f84594-t25x8" Jul 09 15:10:26.669356 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:26.669146 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9445f493-727e-40b8-9fc6-d8ece0562e11-trusted-ca-bundle\") pod \"console-56b7f84594-t25x8\" (UID: \"9445f493-727e-40b8-9fc6-d8ece0562e11\") " pod="openshift-console/console-56b7f84594-t25x8" Jul 09 15:10:26.669834 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:26.669793 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9445f493-727e-40b8-9fc6-d8ece0562e11-oauth-serving-cert\") pod \"console-56b7f84594-t25x8\" (UID: \"9445f493-727e-40b8-9fc6-d8ece0562e11\") " pod="openshift-console/console-56b7f84594-t25x8" Jul 09 15:10:26.669947 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:26.669801 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9445f493-727e-40b8-9fc6-d8ece0562e11-console-config\") pod \"console-56b7f84594-t25x8\" (UID: \"9445f493-727e-40b8-9fc6-d8ece0562e11\") " pod="openshift-console/console-56b7f84594-t25x8" Jul 09 15:10:26.669947 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:26.669861 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9445f493-727e-40b8-9fc6-d8ece0562e11-service-ca\") pod \"console-56b7f84594-t25x8\" (UID: \"9445f493-727e-40b8-9fc6-d8ece0562e11\") " pod="openshift-console/console-56b7f84594-t25x8" Jul 09 15:10:26.670585 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:26.670562 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9445f493-727e-40b8-9fc6-d8ece0562e11-trusted-ca-bundle\") pod \"console-56b7f84594-t25x8\" (UID: \"9445f493-727e-40b8-9fc6-d8ece0562e11\") " pod="openshift-console/console-56b7f84594-t25x8" Jul 09 15:10:26.671567 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:26.671549 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9445f493-727e-40b8-9fc6-d8ece0562e11-console-oauth-config\") pod \"console-56b7f84594-t25x8\" (UID: \"9445f493-727e-40b8-9fc6-d8ece0562e11\") " pod="openshift-console/console-56b7f84594-t25x8" Jul 09 15:10:26.671663 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:26.671644 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9445f493-727e-40b8-9fc6-d8ece0562e11-console-serving-cert\") pod \"console-56b7f84594-t25x8\" (UID: \"9445f493-727e-40b8-9fc6-d8ece0562e11\") " pod="openshift-console/console-56b7f84594-t25x8" Jul 09 15:10:26.678699 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:26.678677 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhhc2\" (UniqueName: \"kubernetes.io/projected/9445f493-727e-40b8-9fc6-d8ece0562e11-kube-api-access-nhhc2\") pod \"console-56b7f84594-t25x8\" (UID: \"9445f493-727e-40b8-9fc6-d8ece0562e11\") " pod="openshift-console/console-56b7f84594-t25x8" Jul 09 15:10:26.795779 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:26.795688 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56b7f84594-t25x8" Jul 09 15:10:26.936919 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:26.936892 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56b7f84594-t25x8"] Jul 09 15:10:26.939390 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:10:26.939361 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9445f493_727e_40b8_9fc6_d8ece0562e11.slice/crio-b200cb69a68f3249773659568c69fab019d71578381e3eb7bf1bf5a57ec6a822 WatchSource:0}: Error finding container b200cb69a68f3249773659568c69fab019d71578381e3eb7bf1bf5a57ec6a822: Status 404 returned error can't find the container with id b200cb69a68f3249773659568c69fab019d71578381e3eb7bf1bf5a57ec6a822 Jul 09 15:10:27.087148 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:27.087049 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56b7f84594-t25x8" event={"ID":"9445f493-727e-40b8-9fc6-d8ece0562e11","Type":"ContainerStarted","Data":"daf9b3e5d1a92f7c5ec4fab3cc96d8212abc12a6609c1e6b2c95540160b9d51c"} Jul 09 15:10:27.087148 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:27.087108 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56b7f84594-t25x8" event={"ID":"9445f493-727e-40b8-9fc6-d8ece0562e11","Type":"ContainerStarted","Data":"b200cb69a68f3249773659568c69fab019d71578381e3eb7bf1bf5a57ec6a822"} Jul 09 15:10:27.110764 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:27.110707 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56b7f84594-t25x8" podStartSLOduration=1.110688811 podStartE2EDuration="1.110688811s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:10:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:10:27.108533892 +0000 UTC m=+473.839040394" watchObservedRunningTime="2026-07-09 15:10:27.110688811 +0000 UTC m=+473.841195300" Jul 09 15:10:31.972209 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:31.972165 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-p44qb" Jul 09 15:10:36.796463 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:36.796414 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56b7f84594-t25x8" Jul 09 15:10:36.797115 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:36.796509 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-56b7f84594-t25x8" Jul 09 15:10:36.801227 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:36.801199 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-56b7f84594-t25x8" Jul 09 15:10:37.140213 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:37.140135 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-56b7f84594-t25x8" Jul 09 15:10:37.315072 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:37.315036 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:10:37.415958 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:37.415858 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:10:37.517540 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:37.517500 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:10:45.032619 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:45.032576 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:10:45.133046 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:45.133009 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:10:45.200165 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:45.200122 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:10:46.321742 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:10:46.321704 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:11:04.860076 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:04.860041 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:11:04.934325 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:04.934287 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:11:30.268706 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:30.268604 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:11:33.744191 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:33.744140 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:11:34.458143 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:34.458093 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:11:38.084276 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:38.084232 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:11:38.356192 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:38.356105 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:11:38.498955 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:38.498918 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:11:40.333940 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:40.333906 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:11:40.839582 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:40.839536 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:11:41.336782 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:41.336746 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:11:43.326341 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:43.326301 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:11:45.093286 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:45.093241 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zp4v"] Jul 09 15:11:45.095968 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:45.095944 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zp4v" Jul 09 15:11:45.098970 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:45.098949 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-xrlb8\"" Jul 09 15:11:45.099213 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:45.099200 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 09 15:11:45.099665 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:45.099652 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 09 15:11:45.118493 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:45.118458 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zp4v"] Jul 09 15:11:45.189173 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:45.189139 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5wh8\" (UniqueName: \"kubernetes.io/projected/76b43023-a8d9-4c2f-91a5-9247c8080324-kube-api-access-f5wh8\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7zp4v\" (UID: \"76b43023-a8d9-4c2f-91a5-9247c8080324\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zp4v" Jul 09 15:11:45.189367 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:45.189182 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/76b43023-a8d9-4c2f-91a5-9247c8080324-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7zp4v\" (UID: \"76b43023-a8d9-4c2f-91a5-9247c8080324\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zp4v" Jul 09 15:11:45.289901 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:45.289863 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5wh8\" (UniqueName: \"kubernetes.io/projected/76b43023-a8d9-4c2f-91a5-9247c8080324-kube-api-access-f5wh8\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7zp4v\" (UID: \"76b43023-a8d9-4c2f-91a5-9247c8080324\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zp4v" Jul 09 15:11:45.290108 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:45.289913 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/76b43023-a8d9-4c2f-91a5-9247c8080324-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7zp4v\" (UID: \"76b43023-a8d9-4c2f-91a5-9247c8080324\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zp4v" Jul 09 15:11:45.290280 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:45.290262 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/76b43023-a8d9-4c2f-91a5-9247c8080324-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7zp4v\" (UID: \"76b43023-a8d9-4c2f-91a5-9247c8080324\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zp4v" Jul 09 15:11:45.300729 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:45.300700 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5wh8\" (UniqueName: \"kubernetes.io/projected/76b43023-a8d9-4c2f-91a5-9247c8080324-kube-api-access-f5wh8\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7zp4v\" (UID: \"76b43023-a8d9-4c2f-91a5-9247c8080324\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zp4v" Jul 09 15:11:45.406131 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:45.406029 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zp4v" Jul 09 15:11:45.565828 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:45.565783 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zp4v"] Jul 09 15:11:45.569785 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:11:45.569743 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76b43023_a8d9_4c2f_91a5_9247c8080324.slice/crio-afe57c9eb1a62790e49f2197c7ee802d3246a92c154fbb33f4a087331994f978 WatchSource:0}: Error finding container afe57c9eb1a62790e49f2197c7ee802d3246a92c154fbb33f4a087331994f978: Status 404 returned error can't find the container with id afe57c9eb1a62790e49f2197c7ee802d3246a92c154fbb33f4a087331994f978 Jul 09 15:11:46.310082 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:46.310041 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:11:46.460200 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:46.460171 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zp4v" event={"ID":"76b43023-a8d9-4c2f-91a5-9247c8080324","Type":"ContainerStarted","Data":"afe57c9eb1a62790e49f2197c7ee802d3246a92c154fbb33f4a087331994f978"} Jul 09 15:11:47.333287 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:47.333193 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:11:51.707490 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:51.707451 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:11:51.835579 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:51.835542 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:11:52.109173 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:52.109010 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:11:52.221883 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:52.221847 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:11:52.713480 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:52.713444 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:11:53.012852 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:53.012799 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:11:53.501411 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:53.501378 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zp4v" event={"ID":"76b43023-a8d9-4c2f-91a5-9247c8080324","Type":"ContainerStarted","Data":"af14f4ebe48f81041c177272c8f28db150a3203bfcd885990de1c54cea6f66cf"} Jul 09 15:11:53.501633 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:53.501487 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zp4v" Jul 09 15:11:53.532992 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:53.532935 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zp4v" podStartSLOduration=1.599842014 podStartE2EDuration="8.532917611s" totalImagesPullingTime="6.933075597s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:11:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:11:53.531112278 +0000 UTC m=+560.261618769" watchObservedRunningTime="2026-07-09 15:11:53.532917611 +0000 UTC m=+560.263424101" Jul 09 15:11:55.481204 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:55.481150 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:11:55.625265 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:55.625208 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:11:55.856995 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:55.856908 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:11:58.180199 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:58.180157 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:11:58.345533 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:11:58.345489 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:12:04.508163 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:04.508129 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zp4v" Jul 09 15:12:04.509025 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:04.508997 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:12:04.636263 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:04.636221 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:12:04.925466 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:04.925376 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:12:05.086650 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:05.086616 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:12:06.623399 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:06.623365 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zp4v"] Jul 09 15:12:06.623873 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:06.623579 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zp4v" podUID="76b43023-a8d9-4c2f-91a5-9247c8080324" containerName="manager" containerID="cri-o://af14f4ebe48f81041c177272c8f28db150a3203bfcd885990de1c54cea6f66cf" gracePeriod=2 Jul 09 15:12:06.625611 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:06.625590 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zp4v"] Jul 09 15:12:06.631909 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:12:06.631879 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7zp4v\" is forbidden: User \"system:node:ip-10-0-130-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-114.ec2.internal' and this object" podUID="76b43023-a8d9-4c2f-91a5-9247c8080324" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zp4v" Jul 09 15:12:06.690850 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:06.690800 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgq47"] Jul 09 15:12:06.691266 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:06.691250 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76b43023-a8d9-4c2f-91a5-9247c8080324" containerName="manager" Jul 09 15:12:06.691328 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:06.691268 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="76b43023-a8d9-4c2f-91a5-9247c8080324" containerName="manager" Jul 09 15:12:06.691381 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:06.691366 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="76b43023-a8d9-4c2f-91a5-9247c8080324" containerName="manager" Jul 09 15:12:06.693743 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:06.693717 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgq47" Jul 09 15:12:06.723616 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:06.723578 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgq47"] Jul 09 15:12:06.775563 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:12:06.775517 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7zp4v\" is forbidden: User \"system:node:ip-10-0-130-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-114.ec2.internal' and this object" podUID="76b43023-a8d9-4c2f-91a5-9247c8080324" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zp4v" Jul 09 15:12:06.794126 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:06.794091 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrkqw\" (UniqueName: \"kubernetes.io/projected/af627aa6-399d-4829-b536-9b2672872846-kube-api-access-zrkqw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fgq47\" (UID: \"af627aa6-399d-4829-b536-9b2672872846\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgq47" Jul 09 15:12:06.794319 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:06.794297 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af627aa6-399d-4829-b536-9b2672872846-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fgq47\" (UID: \"af627aa6-399d-4829-b536-9b2672872846\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgq47" Jul 09 15:12:06.878730 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:06.878672 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zp4v" Jul 09 15:12:06.881328 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:12:06.881306 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7zp4v\" is forbidden: User \"system:node:ip-10-0-130-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-114.ec2.internal' and this object" podUID="76b43023-a8d9-4c2f-91a5-9247c8080324" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zp4v" Jul 09 15:12:06.895796 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:06.895761 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrkqw\" (UniqueName: \"kubernetes.io/projected/af627aa6-399d-4829-b536-9b2672872846-kube-api-access-zrkqw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fgq47\" (UID: \"af627aa6-399d-4829-b536-9b2672872846\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgq47" Jul 09 15:12:06.896076 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:06.896039 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af627aa6-399d-4829-b536-9b2672872846-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fgq47\" (UID: \"af627aa6-399d-4829-b536-9b2672872846\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgq47" Jul 09 15:12:06.896440 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:06.896424 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af627aa6-399d-4829-b536-9b2672872846-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fgq47\" (UID: \"af627aa6-399d-4829-b536-9b2672872846\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgq47" Jul 09 15:12:06.908122 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:06.908099 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrkqw\" (UniqueName: \"kubernetes.io/projected/af627aa6-399d-4829-b536-9b2672872846-kube-api-access-zrkqw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fgq47\" (UID: \"af627aa6-399d-4829-b536-9b2672872846\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgq47" Jul 09 15:12:06.997150 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:06.997115 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f5wh8\" (UniqueName: \"kubernetes.io/projected/76b43023-a8d9-4c2f-91a5-9247c8080324-kube-api-access-f5wh8\") pod \"76b43023-a8d9-4c2f-91a5-9247c8080324\" (UID: \"76b43023-a8d9-4c2f-91a5-9247c8080324\") " Jul 09 15:12:06.997325 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:06.997182 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/76b43023-a8d9-4c2f-91a5-9247c8080324-extensions-socket-volume\") pod \"76b43023-a8d9-4c2f-91a5-9247c8080324\" (UID: \"76b43023-a8d9-4c2f-91a5-9247c8080324\") " Jul 09 15:12:06.997662 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:06.997635 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76b43023-a8d9-4c2f-91a5-9247c8080324-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "76b43023-a8d9-4c2f-91a5-9247c8080324" (UID: "76b43023-a8d9-4c2f-91a5-9247c8080324"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:12:06.999396 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:06.999372 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76b43023-a8d9-4c2f-91a5-9247c8080324-kube-api-access-f5wh8" (OuterVolumeSpecName: "kube-api-access-f5wh8") pod "76b43023-a8d9-4c2f-91a5-9247c8080324" (UID: "76b43023-a8d9-4c2f-91a5-9247c8080324"). InnerVolumeSpecName "kube-api-access-f5wh8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:12:07.014130 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:07.014107 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgq47" Jul 09 15:12:07.098409 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:07.098381 2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/76b43023-a8d9-4c2f-91a5-9247c8080324-extensions-socket-volume\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:12:07.098409 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:07.098408 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f5wh8\" (UniqueName: \"kubernetes.io/projected/76b43023-a8d9-4c2f-91a5-9247c8080324-kube-api-access-f5wh8\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:12:07.171664 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:07.171640 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgq47"] Jul 09 15:12:07.171802 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:12:07.171778 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf627aa6_399d_4829_b536_9b2672872846.slice/crio-a5d160f5b98572efe976939e559c2dad38a4e2e40ae944f1173990ed8758670a WatchSource:0}: Error finding container a5d160f5b98572efe976939e559c2dad38a4e2e40ae944f1173990ed8758670a: Status 404 returned error can't find the container with id a5d160f5b98572efe976939e559c2dad38a4e2e40ae944f1173990ed8758670a Jul 09 15:12:07.582264 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:07.582223 2567 generic.go:356] "Generic (PLEG): container finished" podID="76b43023-a8d9-4c2f-91a5-9247c8080324" containerID="af14f4ebe48f81041c177272c8f28db150a3203bfcd885990de1c54cea6f66cf" exitCode=0 Jul 09 15:12:07.582444 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:07.582286 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zp4v" Jul 09 15:12:07.583183 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:07.582321 2567 scope.go:117] "RemoveContainer" containerID="af14f4ebe48f81041c177272c8f28db150a3203bfcd885990de1c54cea6f66cf" Jul 09 15:12:07.584139 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:07.584059 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgq47" event={"ID":"af627aa6-399d-4829-b536-9b2672872846","Type":"ContainerStarted","Data":"5384f8d53c93e5b89a618d31cf9c3e84b163f3c1526935e1dfeb96fac1a24395"} Jul 09 15:12:07.584205 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:07.584146 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgq47" Jul 09 15:12:07.584205 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:07.584182 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgq47" event={"ID":"af627aa6-399d-4829-b536-9b2672872846","Type":"ContainerStarted","Data":"a5d160f5b98572efe976939e559c2dad38a4e2e40ae944f1173990ed8758670a"} Jul 09 15:12:07.585142 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:12:07.585110 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7zp4v\" is forbidden: User \"system:node:ip-10-0-130-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-114.ec2.internal' and this object" podUID="76b43023-a8d9-4c2f-91a5-9247c8080324" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zp4v" Jul 09 15:12:07.589313 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:12:07.589284 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7zp4v\" is forbidden: User \"system:node:ip-10-0-130-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-114.ec2.internal' and this object" podUID="76b43023-a8d9-4c2f-91a5-9247c8080324" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zp4v" Jul 09 15:12:07.592243 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:07.592213 2567 scope.go:117] "RemoveContainer" containerID="af14f4ebe48f81041c177272c8f28db150a3203bfcd885990de1c54cea6f66cf" Jul 09 15:12:07.592599 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:12:07.592571 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af14f4ebe48f81041c177272c8f28db150a3203bfcd885990de1c54cea6f66cf\": container with ID starting with af14f4ebe48f81041c177272c8f28db150a3203bfcd885990de1c54cea6f66cf not found: ID does not exist" containerID="af14f4ebe48f81041c177272c8f28db150a3203bfcd885990de1c54cea6f66cf" Jul 09 15:12:07.592713 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:07.592608 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af14f4ebe48f81041c177272c8f28db150a3203bfcd885990de1c54cea6f66cf"} err="failed to get container status \"af14f4ebe48f81041c177272c8f28db150a3203bfcd885990de1c54cea6f66cf\": rpc error: code = NotFound desc = could not find container \"af14f4ebe48f81041c177272c8f28db150a3203bfcd885990de1c54cea6f66cf\": container with ID starting with af14f4ebe48f81041c177272c8f28db150a3203bfcd885990de1c54cea6f66cf not found: ID does not exist" Jul 09 15:12:07.647128 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:07.647065 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgq47" podStartSLOduration=1.647043322 podStartE2EDuration="1.647043322s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:12:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:12:07.644274393 +0000 UTC m=+574.374780885" watchObservedRunningTime="2026-07-09 15:12:07.647043322 +0000 UTC m=+574.377549819" Jul 09 15:12:07.647643 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:12:07.647615 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7zp4v\" is forbidden: User \"system:node:ip-10-0-130-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-114.ec2.internal' and this object" podUID="76b43023-a8d9-4c2f-91a5-9247c8080324" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zp4v" Jul 09 15:12:07.828464 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:07.828427 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76b43023-a8d9-4c2f-91a5-9247c8080324" path="/var/lib/kubelet/pods/76b43023-a8d9-4c2f-91a5-9247c8080324/volumes" Jul 09 15:12:08.570378 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:08.570346 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9pg4"] Jul 09 15:12:08.573811 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:08.573792 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9pg4" Jul 09 15:12:08.598203 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:08.598173 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9pg4"] Jul 09 15:12:08.714554 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:08.714514 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cf979\" (UniqueName: \"kubernetes.io/projected/9ce8088a-526c-42c6-b948-c4b0be715572-kube-api-access-cf979\") pod \"kuadrant-operator-controller-manager-86878c769f-s9pg4\" (UID: \"9ce8088a-526c-42c6-b948-c4b0be715572\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9pg4" Jul 09 15:12:08.714986 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:08.714644 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9ce8088a-526c-42c6-b948-c4b0be715572-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-s9pg4\" (UID: \"9ce8088a-526c-42c6-b948-c4b0be715572\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9pg4" Jul 09 15:12:08.815908 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:08.815861 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9ce8088a-526c-42c6-b948-c4b0be715572-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-s9pg4\" (UID: \"9ce8088a-526c-42c6-b948-c4b0be715572\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9pg4" Jul 09 15:12:08.816093 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:08.815971 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf979\" (UniqueName: \"kubernetes.io/projected/9ce8088a-526c-42c6-b948-c4b0be715572-kube-api-access-cf979\") pod \"kuadrant-operator-controller-manager-86878c769f-s9pg4\" (UID: \"9ce8088a-526c-42c6-b948-c4b0be715572\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9pg4" Jul 09 15:12:08.816308 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:08.816286 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9ce8088a-526c-42c6-b948-c4b0be715572-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-s9pg4\" (UID: \"9ce8088a-526c-42c6-b948-c4b0be715572\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9pg4" Jul 09 15:12:08.832119 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:08.832048 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cf979\" (UniqueName: \"kubernetes.io/projected/9ce8088a-526c-42c6-b948-c4b0be715572-kube-api-access-cf979\") pod \"kuadrant-operator-controller-manager-86878c769f-s9pg4\" (UID: \"9ce8088a-526c-42c6-b948-c4b0be715572\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9pg4" Jul 09 15:12:08.885127 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:08.885089 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9pg4" Jul 09 15:12:09.038416 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:09.038384 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9pg4"] Jul 09 15:12:09.041725 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:12:09.041700 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ce8088a_526c_42c6_b948_c4b0be715572.slice/crio-57424dc0012cc6f9eeb900cc9135dc9d8d5df6bc82eaf0cba0cef43aba65ee1e WatchSource:0}: Error finding container 57424dc0012cc6f9eeb900cc9135dc9d8d5df6bc82eaf0cba0cef43aba65ee1e: Status 404 returned error can't find the container with id 57424dc0012cc6f9eeb900cc9135dc9d8d5df6bc82eaf0cba0cef43aba65ee1e Jul 09 15:12:09.598197 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:09.598164 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9pg4" event={"ID":"9ce8088a-526c-42c6-b948-c4b0be715572","Type":"ContainerStarted","Data":"4766cf8560a6f860da04322c8a9ec69d1e20e9a51f5462f5cf5f213247a37a48"} Jul 09 15:12:09.598382 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:09.598220 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9pg4" Jul 09 15:12:09.598382 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:09.598237 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9pg4" event={"ID":"9ce8088a-526c-42c6-b948-c4b0be715572","Type":"ContainerStarted","Data":"57424dc0012cc6f9eeb900cc9135dc9d8d5df6bc82eaf0cba0cef43aba65ee1e"} Jul 09 15:12:10.933664 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:10.933632 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:12:11.087703 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:11.087661 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:12:14.551020 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:14.550984 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:12:14.698023 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:14.697984 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:12:17.376181 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:17.376143 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:12:17.556204 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:17.556164 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:12:18.592403 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:18.592373 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgq47" Jul 09 15:12:18.693989 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:18.693923 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9pg4" podStartSLOduration=10.693903588 podStartE2EDuration="10.693903588s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:12:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:12:09.649209353 +0000 UTC m=+576.379715863" watchObservedRunningTime="2026-07-09 15:12:18.693903588 +0000 UTC m=+585.424410085" Jul 09 15:12:20.605866 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:20.605812 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9pg4" Jul 09 15:12:20.696319 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:20.696284 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgq47"] Jul 09 15:12:20.696524 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:20.696500 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgq47" podUID="af627aa6-399d-4829-b536-9b2672872846" containerName="manager" containerID="cri-o://5384f8d53c93e5b89a618d31cf9c3e84b163f3c1526935e1dfeb96fac1a24395" gracePeriod=10 Jul 09 15:12:20.948472 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:20.948448 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgq47" Jul 09 15:12:21.030267 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:21.030233 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af627aa6-399d-4829-b536-9b2672872846-extensions-socket-volume\") pod \"af627aa6-399d-4829-b536-9b2672872846\" (UID: \"af627aa6-399d-4829-b536-9b2672872846\") " Jul 09 15:12:21.030464 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:21.030300 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrkqw\" (UniqueName: \"kubernetes.io/projected/af627aa6-399d-4829-b536-9b2672872846-kube-api-access-zrkqw\") pod \"af627aa6-399d-4829-b536-9b2672872846\" (UID: \"af627aa6-399d-4829-b536-9b2672872846\") " Jul 09 15:12:21.030625 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:21.030598 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af627aa6-399d-4829-b536-9b2672872846-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "af627aa6-399d-4829-b536-9b2672872846" (UID: "af627aa6-399d-4829-b536-9b2672872846"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:12:21.032476 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:21.032449 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af627aa6-399d-4829-b536-9b2672872846-kube-api-access-zrkqw" (OuterVolumeSpecName: "kube-api-access-zrkqw") pod "af627aa6-399d-4829-b536-9b2672872846" (UID: "af627aa6-399d-4829-b536-9b2672872846"). InnerVolumeSpecName "kube-api-access-zrkqw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:12:21.131052 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:21.130969 2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af627aa6-399d-4829-b536-9b2672872846-extensions-socket-volume\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:12:21.131052 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:21.130996 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrkqw\" (UniqueName: \"kubernetes.io/projected/af627aa6-399d-4829-b536-9b2672872846-kube-api-access-zrkqw\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:12:21.658215 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:21.658180 2567 generic.go:356] "Generic (PLEG): container finished" podID="af627aa6-399d-4829-b536-9b2672872846" containerID="5384f8d53c93e5b89a618d31cf9c3e84b163f3c1526935e1dfeb96fac1a24395" exitCode=0 Jul 09 15:12:21.658629 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:21.658228 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgq47" Jul 09 15:12:21.658629 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:21.658269 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgq47" event={"ID":"af627aa6-399d-4829-b536-9b2672872846","Type":"ContainerDied","Data":"5384f8d53c93e5b89a618d31cf9c3e84b163f3c1526935e1dfeb96fac1a24395"} Jul 09 15:12:21.658629 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:21.658323 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgq47" event={"ID":"af627aa6-399d-4829-b536-9b2672872846","Type":"ContainerDied","Data":"a5d160f5b98572efe976939e559c2dad38a4e2e40ae944f1173990ed8758670a"} Jul 09 15:12:21.658629 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:21.658361 2567 scope.go:117] "RemoveContainer" containerID="5384f8d53c93e5b89a618d31cf9c3e84b163f3c1526935e1dfeb96fac1a24395" Jul 09 15:12:21.667094 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:21.667075 2567 scope.go:117] "RemoveContainer" containerID="5384f8d53c93e5b89a618d31cf9c3e84b163f3c1526935e1dfeb96fac1a24395" Jul 09 15:12:21.667361 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:12:21.667344 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5384f8d53c93e5b89a618d31cf9c3e84b163f3c1526935e1dfeb96fac1a24395\": container with ID starting with 5384f8d53c93e5b89a618d31cf9c3e84b163f3c1526935e1dfeb96fac1a24395 not found: ID does not exist" containerID="5384f8d53c93e5b89a618d31cf9c3e84b163f3c1526935e1dfeb96fac1a24395" Jul 09 15:12:21.667405 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:21.667376 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5384f8d53c93e5b89a618d31cf9c3e84b163f3c1526935e1dfeb96fac1a24395"} err="failed to get container status \"5384f8d53c93e5b89a618d31cf9c3e84b163f3c1526935e1dfeb96fac1a24395\": rpc error: code = NotFound desc = could not find container \"5384f8d53c93e5b89a618d31cf9c3e84b163f3c1526935e1dfeb96fac1a24395\": container with ID starting with 5384f8d53c93e5b89a618d31cf9c3e84b163f3c1526935e1dfeb96fac1a24395 not found: ID does not exist" Jul 09 15:12:21.699097 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:21.699065 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgq47"] Jul 09 15:12:21.708419 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:21.708390 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgq47"] Jul 09 15:12:21.828704 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:21.828668 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af627aa6-399d-4829-b536-9b2672872846" path="/var/lib/kubelet/pods/af627aa6-399d-4829-b536-9b2672872846/volumes" Jul 09 15:12:25.641048 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:25.641017 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:12:25.750681 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:25.750625 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:12:33.700070 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:33.700038 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hml7m_84a10578-4913-456b-9fb7-e6440914b677/ovn-acl-logging/0.log" Jul 09 15:12:33.700532 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:33.700511 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hml7m_84a10578-4913-456b-9fb7-e6440914b677/ovn-acl-logging/0.log" Jul 09 15:12:37.325835 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.325785 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg"] Jul 09 15:12:37.326259 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.326252 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af627aa6-399d-4829-b536-9b2672872846" containerName="manager" Jul 09 15:12:37.326329 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.326264 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="af627aa6-399d-4829-b536-9b2672872846" containerName="manager" Jul 09 15:12:37.326329 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.326327 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="af627aa6-399d-4829-b536-9b2672872846" containerName="manager" Jul 09 15:12:37.331436 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.331411 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" Jul 09 15:12:37.334233 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.334215 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-t4l8j\"" Jul 09 15:12:37.342901 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.342875 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg"] Jul 09 15:12:37.357425 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.357393 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:12:37.399659 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.399624 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-b4hrg\" (UID: \"62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" Jul 09 15:12:37.399659 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.399673 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lz47m\" (UniqueName: \"kubernetes.io/projected/62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978-kube-api-access-lz47m\") pod \"maas-default-gateway-openshift-default-56654dcf7-b4hrg\" (UID: \"62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" Jul 09 15:12:37.399965 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.399767 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-b4hrg\" (UID: \"62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" Jul 09 15:12:37.399965 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.399839 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-b4hrg\" (UID: \"62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" Jul 09 15:12:37.399965 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.399876 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-b4hrg\" (UID: \"62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" Jul 09 15:12:37.399965 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.399909 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-b4hrg\" (UID: \"62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" Jul 09 15:12:37.399965 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.399944 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-b4hrg\" (UID: \"62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" Jul 09 15:12:37.400146 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.399969 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-b4hrg\" (UID: \"62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" Jul 09 15:12:37.400146 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.400004 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-b4hrg\" (UID: \"62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" Jul 09 15:12:37.500851 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.500792 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-b4hrg\" (UID: \"62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" Jul 09 15:12:37.500851 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.500857 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-b4hrg\" (UID: \"62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" Jul 09 15:12:37.501073 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.500953 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-b4hrg\" (UID: \"62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" Jul 09 15:12:37.501073 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.500977 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-b4hrg\" (UID: \"62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" Jul 09 15:12:37.501073 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.501002 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-b4hrg\" (UID: \"62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" Jul 09 15:12:37.501218 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.501165 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-b4hrg\" (UID: \"62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" Jul 09 15:12:37.501277 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.501233 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-b4hrg\" (UID: \"62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" Jul 09 15:12:37.501277 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.501269 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-b4hrg\" (UID: \"62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" Jul 09 15:12:37.501381 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.501307 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz47m\" (UniqueName: \"kubernetes.io/projected/62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978-kube-api-access-lz47m\") pod \"maas-default-gateway-openshift-default-56654dcf7-b4hrg\" (UID: \"62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" Jul 09 15:12:37.501381 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.501355 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-b4hrg\" (UID: \"62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" Jul 09 15:12:37.501381 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.501367 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-b4hrg\" (UID: \"62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" Jul 09 15:12:37.501624 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.501597 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-b4hrg\" (UID: \"62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" Jul 09 15:12:37.501758 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.501686 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-b4hrg\" (UID: \"62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" Jul 09 15:12:37.501758 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.501689 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-b4hrg\" (UID: \"62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" Jul 09 15:12:37.503349 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.503318 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-b4hrg\" (UID: \"62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" Jul 09 15:12:37.503540 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.503522 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-b4hrg\" (UID: \"62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" Jul 09 15:12:37.515089 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.515059 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-b4hrg\" (UID: \"62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" Jul 09 15:12:37.515213 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.515166 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lz47m\" (UniqueName: \"kubernetes.io/projected/62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978-kube-api-access-lz47m\") pod \"maas-default-gateway-openshift-default-56654dcf7-b4hrg\" (UID: \"62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" Jul 09 15:12:37.534158 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.534119 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:12:37.642192 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.642101 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" Jul 09 15:12:37.795148 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.795113 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg"] Jul 09 15:12:37.798186 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:12:37.798148 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62cc6ea3_b3e5_40c5_ad1f_b2dd27e84978.slice/crio-8b841cb338b083ea7e6da89a7e5032f034a512f8109a184d085c979cee6fa1ef WatchSource:0}: Error finding container 8b841cb338b083ea7e6da89a7e5032f034a512f8109a184d085c979cee6fa1ef: Status 404 returned error can't find the container with id 8b841cb338b083ea7e6da89a7e5032f034a512f8109a184d085c979cee6fa1ef Jul 09 15:12:37.800395 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.800367 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:12:37.800469 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.800458 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:12:37.800509 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:37.800489 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:12:38.745360 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:38.745320 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" event={"ID":"62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978","Type":"ContainerStarted","Data":"e7d25429cc23d3cd291b383a035d2e4d98d08db05e2c0abd9b5bbeb72a5ca621"} Jul 09 15:12:38.745810 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:38.745374 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" event={"ID":"62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978","Type":"ContainerStarted","Data":"8b841cb338b083ea7e6da89a7e5032f034a512f8109a184d085c979cee6fa1ef"} Jul 09 15:12:38.775766 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:38.775704 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" podStartSLOduration=1.775687316 podStartE2EDuration="1.775687316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:12:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:12:38.774708777 +0000 UTC m=+605.505215287" watchObservedRunningTime="2026-07-09 15:12:38.775687316 +0000 UTC m=+605.506193854" Jul 09 15:12:39.643200 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:39.643163 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" Jul 09 15:12:39.647974 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:39.647943 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" Jul 09 15:12:39.751210 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:39.751179 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" Jul 09 15:12:39.752396 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:39.752372 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" Jul 09 15:12:39.983469 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:39.983427 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:12:40.124555 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:40.124522 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:12:40.259750 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:40.259663 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:12:40.971638 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:40.971603 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:12:41.152448 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:41.152404 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:12:41.293439 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:41.293348 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-9nrmr"] Jul 09 15:12:41.297192 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:41.297167 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9nrmr" Jul 09 15:12:41.306494 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:41.306474 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-w9ks5\"" Jul 09 15:12:41.323072 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:41.323039 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9nrmr"] Jul 09 15:12:41.341339 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:41.341304 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fvvx\" (UniqueName: \"kubernetes.io/projected/90e8f643-e033-4165-8f0a-2f8791a91a66-kube-api-access-7fvvx\") pod \"authorino-75d556bd77-9nrmr\" (UID: \"90e8f643-e033-4165-8f0a-2f8791a91a66\") " pod="kuadrant-system/authorino-75d556bd77-9nrmr" Jul 09 15:12:41.441929 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:41.441891 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fvvx\" (UniqueName: \"kubernetes.io/projected/90e8f643-e033-4165-8f0a-2f8791a91a66-kube-api-access-7fvvx\") pod \"authorino-75d556bd77-9nrmr\" (UID: \"90e8f643-e033-4165-8f0a-2f8791a91a66\") " pod="kuadrant-system/authorino-75d556bd77-9nrmr" Jul 09 15:12:41.455880 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:41.455793 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fvvx\" (UniqueName: \"kubernetes.io/projected/90e8f643-e033-4165-8f0a-2f8791a91a66-kube-api-access-7fvvx\") pod \"authorino-75d556bd77-9nrmr\" (UID: \"90e8f643-e033-4165-8f0a-2f8791a91a66\") " pod="kuadrant-system/authorino-75d556bd77-9nrmr" Jul 09 15:12:41.607137 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:41.607053 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9nrmr" Jul 09 15:12:41.763231 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:41.763198 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9nrmr"] Jul 09 15:12:41.767286 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:12:41.767259 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod90e8f643_e033_4165_8f0a_2f8791a91a66.slice/crio-b8235239d21e5870b050861c114968d7d09d28c49d11d2681bff0837e1ad1773 WatchSource:0}: Error finding container b8235239d21e5870b050861c114968d7d09d28c49d11d2681bff0837e1ad1773: Status 404 returned error can't find the container with id b8235239d21e5870b050861c114968d7d09d28c49d11d2681bff0837e1ad1773 Jul 09 15:12:42.766348 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:42.766311 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9nrmr" event={"ID":"90e8f643-e033-4165-8f0a-2f8791a91a66","Type":"ContainerStarted","Data":"b8235239d21e5870b050861c114968d7d09d28c49d11d2681bff0837e1ad1773"} Jul 09 15:12:46.790951 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:46.790906 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9nrmr" event={"ID":"90e8f643-e033-4165-8f0a-2f8791a91a66","Type":"ContainerStarted","Data":"09c94b144604894791fa3e23906605f5ee842f0c1769cbda254655cba2986836"} Jul 09 15:12:46.820686 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:46.820622 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-9nrmr" podStartSLOduration=1.795814848 podStartE2EDuration="5.820602366s" totalImagesPullingTime="4.024787518s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:12:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:12:46.819877855 +0000 UTC m=+613.550384352" watchObservedRunningTime="2026-07-09 15:12:46.820602366 +0000 UTC m=+613.551108868" Jul 09 15:12:52.501047 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:12:52.501014 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:13:01.261494 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:01.261411 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:13:01.519474 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:01.519377 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:13:01.839708 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:01.839631 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-4zs2j"] Jul 09 15:13:01.843057 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:01.843041 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4zs2j" Jul 09 15:13:01.846770 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:01.846749 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-4kqpx\"" Jul 09 15:13:01.846930 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:01.846750 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 09 15:13:01.847645 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:01.847630 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 09 15:13:01.847713 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:01.847651 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 09 15:13:01.865514 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:01.865485 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-4zs2j"] Jul 09 15:13:01.948908 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:01.948854 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/3dfc95d2-cc3d-4e8f-a879-070487f9bcbd-data\") pod \"postgres-69fb4594cf-4zs2j\" (UID: \"3dfc95d2-cc3d-4e8f-a879-070487f9bcbd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4zs2j" Jul 09 15:13:01.948908 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:01.948905 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxgpq\" (UniqueName: \"kubernetes.io/projected/3dfc95d2-cc3d-4e8f-a879-070487f9bcbd-kube-api-access-fxgpq\") pod \"postgres-69fb4594cf-4zs2j\" (UID: \"3dfc95d2-cc3d-4e8f-a879-070487f9bcbd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4zs2j" Jul 09 15:13:02.050264 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:02.050225 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/3dfc95d2-cc3d-4e8f-a879-070487f9bcbd-data\") pod \"postgres-69fb4594cf-4zs2j\" (UID: \"3dfc95d2-cc3d-4e8f-a879-070487f9bcbd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4zs2j" Jul 09 15:13:02.050264 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:02.050266 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxgpq\" (UniqueName: \"kubernetes.io/projected/3dfc95d2-cc3d-4e8f-a879-070487f9bcbd-kube-api-access-fxgpq\") pod \"postgres-69fb4594cf-4zs2j\" (UID: \"3dfc95d2-cc3d-4e8f-a879-070487f9bcbd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4zs2j" Jul 09 15:13:02.050596 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:02.050577 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/3dfc95d2-cc3d-4e8f-a879-070487f9bcbd-data\") pod \"postgres-69fb4594cf-4zs2j\" (UID: \"3dfc95d2-cc3d-4e8f-a879-070487f9bcbd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4zs2j" Jul 09 15:13:02.066016 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:02.065983 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxgpq\" (UniqueName: \"kubernetes.io/projected/3dfc95d2-cc3d-4e8f-a879-070487f9bcbd-kube-api-access-fxgpq\") pod \"postgres-69fb4594cf-4zs2j\" (UID: \"3dfc95d2-cc3d-4e8f-a879-070487f9bcbd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4zs2j" Jul 09 15:13:02.154054 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:02.153961 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4zs2j" Jul 09 15:13:02.303316 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:02.303291 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-4zs2j"] Jul 09 15:13:02.305979 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:13:02.305936 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dfc95d2_cc3d_4e8f_a879_070487f9bcbd.slice/crio-7a0c93e0b5bf7bf0a38f425bc9aabd8862d5b65c355c22aa14ab12587a855b81 WatchSource:0}: Error finding container 7a0c93e0b5bf7bf0a38f425bc9aabd8862d5b65c355c22aa14ab12587a855b81: Status 404 returned error can't find the container with id 7a0c93e0b5bf7bf0a38f425bc9aabd8862d5b65c355c22aa14ab12587a855b81 Jul 09 15:13:02.871988 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:02.871941 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4zs2j" event={"ID":"3dfc95d2-cc3d-4e8f-a879-070487f9bcbd","Type":"ContainerStarted","Data":"7a0c93e0b5bf7bf0a38f425bc9aabd8862d5b65c355c22aa14ab12587a855b81"} Jul 09 15:13:02.897978 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:02.897944 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:13:07.815429 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:07.815395 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:13:07.908125 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:07.908076 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4zs2j" event={"ID":"3dfc95d2-cc3d-4e8f-a879-070487f9bcbd","Type":"ContainerStarted","Data":"373019e30a939dd5253b34532b3e78dfde4ba9a4f5a6c0d586c8efbaae4b2c67"} Jul 09 15:13:07.908304 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:07.908141 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4zs2j" Jul 09 15:13:07.948900 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:07.948845 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4zs2j" podStartSLOduration=1.442857199 podStartE2EDuration="6.948802577s" totalImagesPullingTime="5.505945378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:13:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:13:07.946251366 +0000 UTC m=+634.676757857" watchObservedRunningTime="2026-07-09 15:13:07.948802577 +0000 UTC m=+634.679309105" Jul 09 15:13:07.989169 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:07.989120 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:13:13.944224 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:13.944191 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4zs2j" Jul 09 15:13:14.427342 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:14.427301 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:13:14.703782 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:14.703750 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:13:16.969186 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:16.969152 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9nrmr"] Jul 09 15:13:16.969650 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:16.969352 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-9nrmr" podUID="90e8f643-e033-4165-8f0a-2f8791a91a66" containerName="authorino" containerID="cri-o://09c94b144604894791fa3e23906605f5ee842f0c1769cbda254655cba2986836" gracePeriod=30 Jul 09 15:13:17.256584 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:17.256553 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9nrmr" Jul 09 15:13:17.299583 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:17.299541 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7fvvx\" (UniqueName: \"kubernetes.io/projected/90e8f643-e033-4165-8f0a-2f8791a91a66-kube-api-access-7fvvx\") pod \"90e8f643-e033-4165-8f0a-2f8791a91a66\" (UID: \"90e8f643-e033-4165-8f0a-2f8791a91a66\") " Jul 09 15:13:17.301684 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:17.301656 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90e8f643-e033-4165-8f0a-2f8791a91a66-kube-api-access-7fvvx" (OuterVolumeSpecName: "kube-api-access-7fvvx") pod "90e8f643-e033-4165-8f0a-2f8791a91a66" (UID: "90e8f643-e033-4165-8f0a-2f8791a91a66"). InnerVolumeSpecName "kube-api-access-7fvvx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:13:17.400658 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:17.400620 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fvvx\" (UniqueName: \"kubernetes.io/projected/90e8f643-e033-4165-8f0a-2f8791a91a66-kube-api-access-7fvvx\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:13:17.957216 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:17.957181 2567 generic.go:356] "Generic (PLEG): container finished" podID="90e8f643-e033-4165-8f0a-2f8791a91a66" containerID="09c94b144604894791fa3e23906605f5ee842f0c1769cbda254655cba2986836" exitCode=0 Jul 09 15:13:17.957388 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:17.957223 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9nrmr" Jul 09 15:13:17.957388 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:17.957240 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9nrmr" event={"ID":"90e8f643-e033-4165-8f0a-2f8791a91a66","Type":"ContainerDied","Data":"09c94b144604894791fa3e23906605f5ee842f0c1769cbda254655cba2986836"} Jul 09 15:13:17.957388 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:17.957280 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9nrmr" event={"ID":"90e8f643-e033-4165-8f0a-2f8791a91a66","Type":"ContainerDied","Data":"b8235239d21e5870b050861c114968d7d09d28c49d11d2681bff0837e1ad1773"} Jul 09 15:13:17.957388 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:17.957298 2567 scope.go:117] "RemoveContainer" containerID="09c94b144604894791fa3e23906605f5ee842f0c1769cbda254655cba2986836" Jul 09 15:13:17.965983 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:17.965961 2567 scope.go:117] "RemoveContainer" containerID="09c94b144604894791fa3e23906605f5ee842f0c1769cbda254655cba2986836" Jul 09 15:13:17.966245 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:13:17.966225 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09c94b144604894791fa3e23906605f5ee842f0c1769cbda254655cba2986836\": container with ID starting with 09c94b144604894791fa3e23906605f5ee842f0c1769cbda254655cba2986836 not found: ID does not exist" containerID="09c94b144604894791fa3e23906605f5ee842f0c1769cbda254655cba2986836" Jul 09 15:13:17.966311 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:17.966253 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09c94b144604894791fa3e23906605f5ee842f0c1769cbda254655cba2986836"} err="failed to get container status \"09c94b144604894791fa3e23906605f5ee842f0c1769cbda254655cba2986836\": rpc error: code = NotFound desc = could not find container \"09c94b144604894791fa3e23906605f5ee842f0c1769cbda254655cba2986836\": container with ID starting with 09c94b144604894791fa3e23906605f5ee842f0c1769cbda254655cba2986836 not found: ID does not exist" Jul 09 15:13:18.030088 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:18.030058 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9nrmr"] Jul 09 15:13:18.042653 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:18.042621 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9nrmr"] Jul 09 15:13:19.828641 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:19.828595 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90e8f643-e033-4165-8f0a-2f8791a91a66" path="/var/lib/kubelet/pods/90e8f643-e033-4165-8f0a-2f8791a91a66/volumes" Jul 09 15:13:22.506116 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:22.506077 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:13:22.536806 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:22.536772 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-85566d4fc9-j47xz"] Jul 09 15:13:22.537198 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:22.537185 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90e8f643-e033-4165-8f0a-2f8791a91a66" containerName="authorino" Jul 09 15:13:22.537259 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:22.537200 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="90e8f643-e033-4165-8f0a-2f8791a91a66" containerName="authorino" Jul 09 15:13:22.537294 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:22.537277 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="90e8f643-e033-4165-8f0a-2f8791a91a66" containerName="authorino" Jul 09 15:13:22.759938 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:22.759847 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-85566d4fc9-j47xz"] Jul 09 15:13:22.760081 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:22.759966 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85566d4fc9-j47xz" Jul 09 15:13:22.764483 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:22.764463 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5g87n\"" Jul 09 15:13:22.764762 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:22.764748 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 09 15:13:22.765443 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:22.765422 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 09 15:13:22.857290 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:22.857258 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0b7815d3-3eb5-4bba-9670-b0282e50677f-webhook-cert\") pod \"maas-controller-85566d4fc9-j47xz\" (UID: \"0b7815d3-3eb5-4bba-9670-b0282e50677f\") " pod="opendatahub/maas-controller-85566d4fc9-j47xz" Jul 09 15:13:22.857460 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:22.857331 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg9m2\" (UniqueName: \"kubernetes.io/projected/0b7815d3-3eb5-4bba-9670-b0282e50677f-kube-api-access-rg9m2\") pod \"maas-controller-85566d4fc9-j47xz\" (UID: \"0b7815d3-3eb5-4bba-9670-b0282e50677f\") " pod="opendatahub/maas-controller-85566d4fc9-j47xz" Jul 09 15:13:22.875752 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:22.875722 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-58d8c4744b-lbvjp"] Jul 09 15:13:22.896983 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:22.896955 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58d8c4744b-lbvjp" Jul 09 15:13:22.906059 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:22.906031 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58d8c4744b-lbvjp"] Jul 09 15:13:22.957912 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:22.957877 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9afbd50e-a1fe-4c7b-85a8-212e8496c259-webhook-cert\") pod \"maas-controller-58d8c4744b-lbvjp\" (UID: \"9afbd50e-a1fe-4c7b-85a8-212e8496c259\") " pod="opendatahub/maas-controller-58d8c4744b-lbvjp" Jul 09 15:13:22.958093 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:22.957929 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg9m2\" (UniqueName: \"kubernetes.io/projected/0b7815d3-3eb5-4bba-9670-b0282e50677f-kube-api-access-rg9m2\") pod \"maas-controller-85566d4fc9-j47xz\" (UID: \"0b7815d3-3eb5-4bba-9670-b0282e50677f\") " pod="opendatahub/maas-controller-85566d4fc9-j47xz" Jul 09 15:13:22.958093 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:22.958053 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lg86\" (UniqueName: \"kubernetes.io/projected/9afbd50e-a1fe-4c7b-85a8-212e8496c259-kube-api-access-8lg86\") pod \"maas-controller-58d8c4744b-lbvjp\" (UID: \"9afbd50e-a1fe-4c7b-85a8-212e8496c259\") " pod="opendatahub/maas-controller-58d8c4744b-lbvjp" Jul 09 15:13:22.958234 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:22.958137 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0b7815d3-3eb5-4bba-9670-b0282e50677f-webhook-cert\") pod \"maas-controller-85566d4fc9-j47xz\" (UID: \"0b7815d3-3eb5-4bba-9670-b0282e50677f\") " pod="opendatahub/maas-controller-85566d4fc9-j47xz" Jul 09 15:13:22.960503 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:22.960481 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0b7815d3-3eb5-4bba-9670-b0282e50677f-webhook-cert\") pod \"maas-controller-85566d4fc9-j47xz\" (UID: \"0b7815d3-3eb5-4bba-9670-b0282e50677f\") " pod="opendatahub/maas-controller-85566d4fc9-j47xz" Jul 09 15:13:22.974377 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:22.974355 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg9m2\" (UniqueName: \"kubernetes.io/projected/0b7815d3-3eb5-4bba-9670-b0282e50677f-kube-api-access-rg9m2\") pod \"maas-controller-85566d4fc9-j47xz\" (UID: \"0b7815d3-3eb5-4bba-9670-b0282e50677f\") " pod="opendatahub/maas-controller-85566d4fc9-j47xz" Jul 09 15:13:23.058928 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:23.058804 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9afbd50e-a1fe-4c7b-85a8-212e8496c259-webhook-cert\") pod \"maas-controller-58d8c4744b-lbvjp\" (UID: \"9afbd50e-a1fe-4c7b-85a8-212e8496c259\") " pod="opendatahub/maas-controller-58d8c4744b-lbvjp" Jul 09 15:13:23.059087 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:23.058961 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lg86\" (UniqueName: \"kubernetes.io/projected/9afbd50e-a1fe-4c7b-85a8-212e8496c259-kube-api-access-8lg86\") pod \"maas-controller-58d8c4744b-lbvjp\" (UID: \"9afbd50e-a1fe-4c7b-85a8-212e8496c259\") " pod="opendatahub/maas-controller-58d8c4744b-lbvjp" Jul 09 15:13:23.061274 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:23.061251 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9afbd50e-a1fe-4c7b-85a8-212e8496c259-webhook-cert\") pod \"maas-controller-58d8c4744b-lbvjp\" (UID: \"9afbd50e-a1fe-4c7b-85a8-212e8496c259\") " pod="opendatahub/maas-controller-58d8c4744b-lbvjp" Jul 09 15:13:23.070131 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:23.070111 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85566d4fc9-j47xz" Jul 09 15:13:23.081091 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:23.081066 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lg86\" (UniqueName: \"kubernetes.io/projected/9afbd50e-a1fe-4c7b-85a8-212e8496c259-kube-api-access-8lg86\") pod \"maas-controller-58d8c4744b-lbvjp\" (UID: \"9afbd50e-a1fe-4c7b-85a8-212e8496c259\") " pod="opendatahub/maas-controller-58d8c4744b-lbvjp" Jul 09 15:13:23.208362 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:23.208328 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58d8c4744b-lbvjp" Jul 09 15:13:23.230127 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:23.230046 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-85566d4fc9-j47xz"] Jul 09 15:13:23.230497 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:13:23.230470 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b7815d3_3eb5_4bba_9670_b0282e50677f.slice/crio-3d6866ea2252b2508b4be8d9a40bf6b87edc8c0a2fb7c85dbe5d456c87870e9d WatchSource:0}: Error finding container 3d6866ea2252b2508b4be8d9a40bf6b87edc8c0a2fb7c85dbe5d456c87870e9d: Status 404 returned error can't find the container with id 3d6866ea2252b2508b4be8d9a40bf6b87edc8c0a2fb7c85dbe5d456c87870e9d Jul 09 15:13:23.354639 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:23.354612 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58d8c4744b-lbvjp"] Jul 09 15:13:23.357252 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:13:23.357225 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9afbd50e_a1fe_4c7b_85a8_212e8496c259.slice/crio-9dd490f1de803521b233f7a46be1eaa59995c66773477d9485a5ddf9f7da85c0 WatchSource:0}: Error finding container 9dd490f1de803521b233f7a46be1eaa59995c66773477d9485a5ddf9f7da85c0: Status 404 returned error can't find the container with id 9dd490f1de803521b233f7a46be1eaa59995c66773477d9485a5ddf9f7da85c0 Jul 09 15:13:23.994892 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:23.994739 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58d8c4744b-lbvjp" event={"ID":"9afbd50e-a1fe-4c7b-85a8-212e8496c259","Type":"ContainerStarted","Data":"9dd490f1de803521b233f7a46be1eaa59995c66773477d9485a5ddf9f7da85c0"} Jul 09 15:13:24.001309 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:24.001217 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85566d4fc9-j47xz" event={"ID":"0b7815d3-3eb5-4bba-9670-b0282e50677f","Type":"ContainerStarted","Data":"3d6866ea2252b2508b4be8d9a40bf6b87edc8c0a2fb7c85dbe5d456c87870e9d"} Jul 09 15:13:24.205566 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:24.205419 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:13:25.878968 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:25.878867 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:13:26.015160 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:26.015124 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58d8c4744b-lbvjp" event={"ID":"9afbd50e-a1fe-4c7b-85a8-212e8496c259","Type":"ContainerStarted","Data":"fa82d13f4499690dfc67c45a5c0fed6d2c08a4fe68f7cfaf6e5b02c81acbbe7d"} Jul 09 15:13:26.015357 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:26.015243 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-58d8c4744b-lbvjp" Jul 09 15:13:26.016845 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:26.016801 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85566d4fc9-j47xz" event={"ID":"0b7815d3-3eb5-4bba-9670-b0282e50677f","Type":"ContainerStarted","Data":"06514546ec1e8476c5f1318e2266a5cc5cedcfa7edf673520794814862656bb0"} Jul 09 15:13:26.016969 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:26.016936 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-85566d4fc9-j47xz" Jul 09 15:13:26.051766 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:26.051705 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-58d8c4744b-lbvjp" podStartSLOduration=1.827465205 podStartE2EDuration="4.051688591s" totalImagesPullingTime="2.224223386s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:13:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:13:26.050196297 +0000 UTC m=+652.780702809" watchObservedRunningTime="2026-07-09 15:13:26.051688591 +0000 UTC m=+652.782195101" Jul 09 15:13:26.085515 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:26.085470 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:13:26.115461 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:26.115403 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-85566d4fc9-j47xz" podStartSLOduration=1.7749950700000001 podStartE2EDuration="4.115386081s" totalImagesPullingTime="2.340391011s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:13:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:13:26.114950566 +0000 UTC m=+652.845457059" watchObservedRunningTime="2026-07-09 15:13:26.115386081 +0000 UTC m=+652.845892572" Jul 09 15:13:37.030287 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:37.030256 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-58d8c4744b-lbvjp" Jul 09 15:13:37.030697 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:37.030327 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-85566d4fc9-j47xz" Jul 09 15:13:37.179581 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:37.179544 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-85566d4fc9-j47xz"] Jul 09 15:13:37.179852 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:37.179801 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-85566d4fc9-j47xz" podUID="0b7815d3-3eb5-4bba-9670-b0282e50677f" containerName="manager" containerID="cri-o://06514546ec1e8476c5f1318e2266a5cc5cedcfa7edf673520794814862656bb0" gracePeriod=30 Jul 09 15:13:37.436255 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:37.436229 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85566d4fc9-j47xz" Jul 09 15:13:37.603208 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:37.603099 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rg9m2\" (UniqueName: \"kubernetes.io/projected/0b7815d3-3eb5-4bba-9670-b0282e50677f-kube-api-access-rg9m2\") pod \"0b7815d3-3eb5-4bba-9670-b0282e50677f\" (UID: \"0b7815d3-3eb5-4bba-9670-b0282e50677f\") " Jul 09 15:13:37.603375 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:37.603233 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0b7815d3-3eb5-4bba-9670-b0282e50677f-webhook-cert\") pod \"0b7815d3-3eb5-4bba-9670-b0282e50677f\" (UID: \"0b7815d3-3eb5-4bba-9670-b0282e50677f\") " Jul 09 15:13:37.605537 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:37.605510 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b7815d3-3eb5-4bba-9670-b0282e50677f-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "0b7815d3-3eb5-4bba-9670-b0282e50677f" (UID: "0b7815d3-3eb5-4bba-9670-b0282e50677f"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 15:13:37.605620 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:37.605510 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b7815d3-3eb5-4bba-9670-b0282e50677f-kube-api-access-rg9m2" (OuterVolumeSpecName: "kube-api-access-rg9m2") pod "0b7815d3-3eb5-4bba-9670-b0282e50677f" (UID: "0b7815d3-3eb5-4bba-9670-b0282e50677f"). InnerVolumeSpecName "kube-api-access-rg9m2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:13:37.704327 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:37.704284 2567 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0b7815d3-3eb5-4bba-9670-b0282e50677f-webhook-cert\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:13:37.704327 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:37.704319 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rg9m2\" (UniqueName: \"kubernetes.io/projected/0b7815d3-3eb5-4bba-9670-b0282e50677f-kube-api-access-rg9m2\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:13:38.081558 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:38.081518 2567 generic.go:356] "Generic (PLEG): container finished" podID="0b7815d3-3eb5-4bba-9670-b0282e50677f" containerID="06514546ec1e8476c5f1318e2266a5cc5cedcfa7edf673520794814862656bb0" exitCode=0 Jul 09 15:13:38.081994 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:38.081569 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85566d4fc9-j47xz" Jul 09 15:13:38.081994 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:38.081609 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85566d4fc9-j47xz" event={"ID":"0b7815d3-3eb5-4bba-9670-b0282e50677f","Type":"ContainerDied","Data":"06514546ec1e8476c5f1318e2266a5cc5cedcfa7edf673520794814862656bb0"} Jul 09 15:13:38.081994 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:38.081666 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85566d4fc9-j47xz" event={"ID":"0b7815d3-3eb5-4bba-9670-b0282e50677f","Type":"ContainerDied","Data":"3d6866ea2252b2508b4be8d9a40bf6b87edc8c0a2fb7c85dbe5d456c87870e9d"} Jul 09 15:13:38.081994 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:38.081693 2567 scope.go:117] "RemoveContainer" containerID="06514546ec1e8476c5f1318e2266a5cc5cedcfa7edf673520794814862656bb0" Jul 09 15:13:38.090255 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:38.090222 2567 scope.go:117] "RemoveContainer" containerID="06514546ec1e8476c5f1318e2266a5cc5cedcfa7edf673520794814862656bb0" Jul 09 15:13:38.090504 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:13:38.090487 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06514546ec1e8476c5f1318e2266a5cc5cedcfa7edf673520794814862656bb0\": container with ID starting with 06514546ec1e8476c5f1318e2266a5cc5cedcfa7edf673520794814862656bb0 not found: ID does not exist" containerID="06514546ec1e8476c5f1318e2266a5cc5cedcfa7edf673520794814862656bb0" Jul 09 15:13:38.090580 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:38.090514 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06514546ec1e8476c5f1318e2266a5cc5cedcfa7edf673520794814862656bb0"} err="failed to get container status \"06514546ec1e8476c5f1318e2266a5cc5cedcfa7edf673520794814862656bb0\": rpc error: code = NotFound desc = could not find container \"06514546ec1e8476c5f1318e2266a5cc5cedcfa7edf673520794814862656bb0\": container with ID starting with 06514546ec1e8476c5f1318e2266a5cc5cedcfa7edf673520794814862656bb0 not found: ID does not exist" Jul 09 15:13:38.103882 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:38.103847 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-85566d4fc9-j47xz"] Jul 09 15:13:38.110744 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:38.110719 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-85566d4fc9-j47xz"] Jul 09 15:13:39.828741 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:39.828709 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b7815d3-3eb5-4bba-9670-b0282e50677f" path="/var/lib/kubelet/pods/0b7815d3-3eb5-4bba-9670-b0282e50677f/volumes" Jul 09 15:13:58.769915 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:58.769872 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:13:59.048622 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:59.048511 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:13:59.371473 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:13:59.371383 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:14:00.030371 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:00.030322 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:14:00.134037 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:00.134002 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:14:00.311799 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:00.311720 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-9zt6p"] Jul 09 15:14:00.312177 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:00.312158 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b7815d3-3eb5-4bba-9670-b0282e50677f" containerName="manager" Jul 09 15:14:00.312294 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:00.312179 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b7815d3-3eb5-4bba-9670-b0282e50677f" containerName="manager" Jul 09 15:14:00.312294 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:00.312288 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b7815d3-3eb5-4bba-9670-b0282e50677f" containerName="manager" Jul 09 15:14:00.317658 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:00.317633 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-9zt6p" Jul 09 15:14:00.320525 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:00.320503 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 09 15:14:00.320694 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:00.320613 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-cbkll\"" Jul 09 15:14:00.341251 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:00.341220 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-9zt6p"] Jul 09 15:14:00.437424 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:00.437390 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4007bf49-82d0-4310-b0f4-21bf752c912e-plugins-config-volume\") pod \"payload-processing-58f86c7798-9zt6p\" (UID: \"4007bf49-82d0-4310-b0f4-21bf752c912e\") " pod="openshift-ingress/payload-processing-58f86c7798-9zt6p" Jul 09 15:14:00.437581 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:00.437436 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnh59\" (UniqueName: \"kubernetes.io/projected/4007bf49-82d0-4310-b0f4-21bf752c912e-kube-api-access-wnh59\") pod \"payload-processing-58f86c7798-9zt6p\" (UID: \"4007bf49-82d0-4310-b0f4-21bf752c912e\") " pod="openshift-ingress/payload-processing-58f86c7798-9zt6p" Jul 09 15:14:00.538685 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:00.538647 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4007bf49-82d0-4310-b0f4-21bf752c912e-plugins-config-volume\") pod \"payload-processing-58f86c7798-9zt6p\" (UID: \"4007bf49-82d0-4310-b0f4-21bf752c912e\") " pod="openshift-ingress/payload-processing-58f86c7798-9zt6p" Jul 09 15:14:00.538886 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:00.538708 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnh59\" (UniqueName: \"kubernetes.io/projected/4007bf49-82d0-4310-b0f4-21bf752c912e-kube-api-access-wnh59\") pod \"payload-processing-58f86c7798-9zt6p\" (UID: \"4007bf49-82d0-4310-b0f4-21bf752c912e\") " pod="openshift-ingress/payload-processing-58f86c7798-9zt6p" Jul 09 15:14:00.539861 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:00.539810 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4007bf49-82d0-4310-b0f4-21bf752c912e-plugins-config-volume\") pod \"payload-processing-58f86c7798-9zt6p\" (UID: \"4007bf49-82d0-4310-b0f4-21bf752c912e\") " pod="openshift-ingress/payload-processing-58f86c7798-9zt6p" Jul 09 15:14:00.566427 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:00.566354 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnh59\" (UniqueName: \"kubernetes.io/projected/4007bf49-82d0-4310-b0f4-21bf752c912e-kube-api-access-wnh59\") pod \"payload-processing-58f86c7798-9zt6p\" (UID: \"4007bf49-82d0-4310-b0f4-21bf752c912e\") " pod="openshift-ingress/payload-processing-58f86c7798-9zt6p" Jul 09 15:14:00.628094 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:00.628060 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-9zt6p" Jul 09 15:14:00.766358 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:00.766333 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-9zt6p"] Jul 09 15:14:00.768339 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:14:00.768312 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4007bf49_82d0_4310_b0f4_21bf752c912e.slice/crio-fcfba624a9455a9ba4d321e75b7ba333136a79b7e184eeaee8a8d9ecba44d711 WatchSource:0}: Error finding container fcfba624a9455a9ba4d321e75b7ba333136a79b7e184eeaee8a8d9ecba44d711: Status 404 returned error can't find the container with id fcfba624a9455a9ba4d321e75b7ba333136a79b7e184eeaee8a8d9ecba44d711 Jul 09 15:14:00.770350 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:00.770333 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 15:14:01.205611 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:01.205557 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-9zt6p" event={"ID":"4007bf49-82d0-4310-b0f4-21bf752c912e","Type":"ContainerStarted","Data":"fcfba624a9455a9ba4d321e75b7ba333136a79b7e184eeaee8a8d9ecba44d711"} Jul 09 15:14:03.227890 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:03.227852 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-9zt6p" event={"ID":"4007bf49-82d0-4310-b0f4-21bf752c912e","Type":"ContainerStarted","Data":"8abc408a8f533427bac632f4ec2bd79b46965eb8e107b6c2b148604c62d1f13e"} Jul 09 15:14:03.258925 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:03.258870 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-9zt6p" podStartSLOduration=1.384700751 podStartE2EDuration="3.258853344s" totalImagesPullingTime="1.874152593s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:14:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:14:03.255358112 +0000 UTC m=+689.985864600" watchObservedRunningTime="2026-07-09 15:14:03.258853344 +0000 UTC m=+689.989359835" Jul 09 15:14:04.233120 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:04.233091 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-9zt6p" Jul 09 15:14:08.254876 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:08.254771 2567 generic.go:356] "Generic (PLEG): container finished" podID="62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978" containerID="e7d25429cc23d3cd291b383a035d2e4d98d08db05e2c0abd9b5bbeb72a5ca621" exitCode=0 Jul 09 15:14:08.254876 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:08.254852 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" event={"ID":"62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978","Type":"ContainerDied","Data":"e7d25429cc23d3cd291b383a035d2e4d98d08db05e2c0abd9b5bbeb72a5ca621"} Jul 09 15:14:08.255305 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:08.255266 2567 scope.go:117] "RemoveContainer" containerID="e7d25429cc23d3cd291b383a035d2e4d98d08db05e2c0abd9b5bbeb72a5ca621" Jul 09 15:14:08.256281 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:08.256253 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:14:08.256339 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:08.256326 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:14:08.256375 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:08.256359 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:14:09.164422 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:09.164392 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:14:09.261656 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:09.261613 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" event={"ID":"62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978","Type":"ContainerStarted","Data":"df7c204e38f511e912624c73f22e39dd491d4c841b977a0b4c9d95e560621cb2"} Jul 09 15:14:09.331131 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:09.331102 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:14:09.642803 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:09.642769 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" Jul 09 15:14:09.643912 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:09.643890 2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-b4hrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 09 15:14:09.643973 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:09.643941 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" podUID="62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 09 15:14:09.751943 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:09.751911 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" Jul 09 15:14:10.642477 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:10.642442 2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-b4hrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 09 15:14:10.642908 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:10.642509 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" podUID="62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 09 15:14:11.643433 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:11.643393 2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-b4hrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 09 15:14:11.643879 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:11.643452 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" podUID="62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 09 15:14:12.642517 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:12.642479 2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-b4hrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 09 15:14:12.642702 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:12.642537 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" podUID="62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 09 15:14:13.646897 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:13.646865 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" Jul 09 15:14:13.647911 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:13.647892 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b4hrg" Jul 09 15:14:15.241226 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:15.241192 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-9zt6p" Jul 09 15:14:24.357009 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:24.356970 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9"] Jul 09 15:14:24.360769 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:24.360747 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9" Jul 09 15:14:24.365098 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:24.365071 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 09 15:14:24.365263 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:24.365187 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 09 15:14:24.366143 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:24.366126 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-f2vzq\"" Jul 09 15:14:24.366218 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:24.366189 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 09 15:14:24.378951 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:24.378918 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9"] Jul 09 15:14:24.400318 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:24.400279 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:14:24.471739 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:24.471659 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj2pw\" (UniqueName: \"kubernetes.io/projected/b5e394ed-ab9b-4e85-a315-2dfbeb449ec7-kube-api-access-vj2pw\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9\" (UID: \"b5e394ed-ab9b-4e85-a315-2dfbeb449ec7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9" Jul 09 15:14:24.471739 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:24.471704 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5e394ed-ab9b-4e85-a315-2dfbeb449ec7-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9\" (UID: \"b5e394ed-ab9b-4e85-a315-2dfbeb449ec7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9" Jul 09 15:14:24.471739 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:24.471731 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5e394ed-ab9b-4e85-a315-2dfbeb449ec7-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9\" (UID: \"b5e394ed-ab9b-4e85-a315-2dfbeb449ec7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9" Jul 09 15:14:24.471996 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:24.471779 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5e394ed-ab9b-4e85-a315-2dfbeb449ec7-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9\" (UID: \"b5e394ed-ab9b-4e85-a315-2dfbeb449ec7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9" Jul 09 15:14:24.471996 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:24.471799 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5e394ed-ab9b-4e85-a315-2dfbeb449ec7-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9\" (UID: \"b5e394ed-ab9b-4e85-a315-2dfbeb449ec7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9" Jul 09 15:14:24.471996 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:24.471870 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5e394ed-ab9b-4e85-a315-2dfbeb449ec7-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9\" (UID: \"b5e394ed-ab9b-4e85-a315-2dfbeb449ec7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9" Jul 09 15:14:24.572710 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:24.572676 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vj2pw\" (UniqueName: \"kubernetes.io/projected/b5e394ed-ab9b-4e85-a315-2dfbeb449ec7-kube-api-access-vj2pw\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9\" (UID: \"b5e394ed-ab9b-4e85-a315-2dfbeb449ec7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9" Jul 09 15:14:24.572925 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:24.572721 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5e394ed-ab9b-4e85-a315-2dfbeb449ec7-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9\" (UID: \"b5e394ed-ab9b-4e85-a315-2dfbeb449ec7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9" Jul 09 15:14:24.572925 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:24.572871 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5e394ed-ab9b-4e85-a315-2dfbeb449ec7-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9\" (UID: \"b5e394ed-ab9b-4e85-a315-2dfbeb449ec7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9" Jul 09 15:14:24.572925 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:24.572920 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5e394ed-ab9b-4e85-a315-2dfbeb449ec7-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9\" (UID: \"b5e394ed-ab9b-4e85-a315-2dfbeb449ec7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9" Jul 09 15:14:24.573101 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:24.572964 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5e394ed-ab9b-4e85-a315-2dfbeb449ec7-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9\" (UID: \"b5e394ed-ab9b-4e85-a315-2dfbeb449ec7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9" Jul 09 15:14:24.573101 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:24.573010 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5e394ed-ab9b-4e85-a315-2dfbeb449ec7-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9\" (UID: \"b5e394ed-ab9b-4e85-a315-2dfbeb449ec7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9" Jul 09 15:14:24.573419 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:24.573385 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5e394ed-ab9b-4e85-a315-2dfbeb449ec7-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9\" (UID: \"b5e394ed-ab9b-4e85-a315-2dfbeb449ec7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9" Jul 09 15:14:24.573419 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:24.573409 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5e394ed-ab9b-4e85-a315-2dfbeb449ec7-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9\" (UID: \"b5e394ed-ab9b-4e85-a315-2dfbeb449ec7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9" Jul 09 15:14:24.573567 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:24.573426 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5e394ed-ab9b-4e85-a315-2dfbeb449ec7-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9\" (UID: \"b5e394ed-ab9b-4e85-a315-2dfbeb449ec7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9" Jul 09 15:14:24.575539 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:24.575515 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5e394ed-ab9b-4e85-a315-2dfbeb449ec7-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9\" (UID: \"b5e394ed-ab9b-4e85-a315-2dfbeb449ec7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9" Jul 09 15:14:24.575848 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:24.575810 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5e394ed-ab9b-4e85-a315-2dfbeb449ec7-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9\" (UID: \"b5e394ed-ab9b-4e85-a315-2dfbeb449ec7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9" Jul 09 15:14:24.582051 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:24.582022 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj2pw\" (UniqueName: \"kubernetes.io/projected/b5e394ed-ab9b-4e85-a315-2dfbeb449ec7-kube-api-access-vj2pw\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9\" (UID: \"b5e394ed-ab9b-4e85-a315-2dfbeb449ec7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9" Jul 09 15:14:24.672388 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:24.672288 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9" Jul 09 15:14:24.819669 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:24.819635 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9"] Jul 09 15:14:24.822374 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:14:24.822334 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5e394ed_ab9b_4e85_a315_2dfbeb449ec7.slice/crio-9ebe828f5e6788dd3d5f1bded29cec12ce7a08d7ba969796c356feeff13d9e47 WatchSource:0}: Error finding container 9ebe828f5e6788dd3d5f1bded29cec12ce7a08d7ba969796c356feeff13d9e47: Status 404 returned error can't find the container with id 9ebe828f5e6788dd3d5f1bded29cec12ce7a08d7ba969796c356feeff13d9e47 Jul 09 15:14:25.347769 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:25.347723 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9" event={"ID":"b5e394ed-ab9b-4e85-a315-2dfbeb449ec7","Type":"ContainerStarted","Data":"9ebe828f5e6788dd3d5f1bded29cec12ce7a08d7ba969796c356feeff13d9e47"} Jul 09 15:14:26.353608 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:26.353576 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9" event={"ID":"b5e394ed-ab9b-4e85-a315-2dfbeb449ec7","Type":"ContainerStarted","Data":"c322cae2015f59412956d4c744ff2ca40e0b9588bbdba5943bdb27d729e2a6c4"} Jul 09 15:14:26.376754 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:26.376697 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9" podStartSLOduration=1.033545098 podStartE2EDuration="2.376679844s" totalImagesPullingTime="1.343134746s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:14:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:14:26.374608672 +0000 UTC m=+713.105115163" watchObservedRunningTime="2026-07-09 15:14:26.376679844 +0000 UTC m=+713.107186335" Jul 09 15:14:34.673016 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:34.672971 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9" Jul 09 15:14:34.673479 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:34.673046 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9" Jul 09 15:14:34.685564 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:34.685535 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9" Jul 09 15:14:35.409435 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:35.409403 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9" Jul 09 15:14:49.133589 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:49.133554 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:14:58.073070 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:58.073037 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-bg222"] Jul 09 15:14:58.076256 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:58.076239 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bg222" Jul 09 15:14:58.079119 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:58.079097 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 09 15:14:58.087324 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:58.087300 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-bg222"] Jul 09 15:14:58.101936 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:58.101895 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:14:58.113967 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:58.113930 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23488e80-cefa-4039-86e5-a1fdb1d73838-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bg222\" (UID: \"23488e80-cefa-4039-86e5-a1fdb1d73838\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bg222" Jul 09 15:14:58.114120 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:58.113979 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23488e80-cefa-4039-86e5-a1fdb1d73838-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bg222\" (UID: \"23488e80-cefa-4039-86e5-a1fdb1d73838\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bg222" Jul 09 15:14:58.114120 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:58.114003 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23488e80-cefa-4039-86e5-a1fdb1d73838-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bg222\" (UID: \"23488e80-cefa-4039-86e5-a1fdb1d73838\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bg222" Jul 09 15:14:58.114120 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:58.114063 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23488e80-cefa-4039-86e5-a1fdb1d73838-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bg222\" (UID: \"23488e80-cefa-4039-86e5-a1fdb1d73838\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bg222" Jul 09 15:14:58.114244 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:58.114129 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23488e80-cefa-4039-86e5-a1fdb1d73838-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bg222\" (UID: \"23488e80-cefa-4039-86e5-a1fdb1d73838\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bg222" Jul 09 15:14:58.114244 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:58.114159 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbvm4\" (UniqueName: \"kubernetes.io/projected/23488e80-cefa-4039-86e5-a1fdb1d73838-kube-api-access-hbvm4\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bg222\" (UID: \"23488e80-cefa-4039-86e5-a1fdb1d73838\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bg222" Jul 09 15:14:58.215438 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:58.215403 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23488e80-cefa-4039-86e5-a1fdb1d73838-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bg222\" (UID: \"23488e80-cefa-4039-86e5-a1fdb1d73838\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bg222" Jul 09 15:14:58.215438 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:58.215446 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23488e80-cefa-4039-86e5-a1fdb1d73838-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bg222\" (UID: \"23488e80-cefa-4039-86e5-a1fdb1d73838\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bg222" Jul 09 15:14:58.215680 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:58.215495 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23488e80-cefa-4039-86e5-a1fdb1d73838-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bg222\" (UID: \"23488e80-cefa-4039-86e5-a1fdb1d73838\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bg222" Jul 09 15:14:58.215680 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:58.215552 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23488e80-cefa-4039-86e5-a1fdb1d73838-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bg222\" (UID: \"23488e80-cefa-4039-86e5-a1fdb1d73838\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bg222" Jul 09 15:14:58.215680 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:58.215580 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbvm4\" (UniqueName: \"kubernetes.io/projected/23488e80-cefa-4039-86e5-a1fdb1d73838-kube-api-access-hbvm4\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bg222\" (UID: \"23488e80-cefa-4039-86e5-a1fdb1d73838\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bg222" Jul 09 15:14:58.215858 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:58.215686 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23488e80-cefa-4039-86e5-a1fdb1d73838-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bg222\" (UID: \"23488e80-cefa-4039-86e5-a1fdb1d73838\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bg222" Jul 09 15:14:58.216118 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:58.216087 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23488e80-cefa-4039-86e5-a1fdb1d73838-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bg222\" (UID: \"23488e80-cefa-4039-86e5-a1fdb1d73838\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bg222" Jul 09 15:14:58.216248 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:58.216128 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23488e80-cefa-4039-86e5-a1fdb1d73838-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bg222\" (UID: \"23488e80-cefa-4039-86e5-a1fdb1d73838\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bg222" Jul 09 15:14:58.216248 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:58.216142 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23488e80-cefa-4039-86e5-a1fdb1d73838-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bg222\" (UID: \"23488e80-cefa-4039-86e5-a1fdb1d73838\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bg222" Jul 09 15:14:58.217902 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:58.217886 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23488e80-cefa-4039-86e5-a1fdb1d73838-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bg222\" (UID: \"23488e80-cefa-4039-86e5-a1fdb1d73838\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bg222" Jul 09 15:14:58.218178 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:58.218157 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23488e80-cefa-4039-86e5-a1fdb1d73838-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bg222\" (UID: \"23488e80-cefa-4039-86e5-a1fdb1d73838\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bg222" Jul 09 15:14:58.225069 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:58.225045 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbvm4\" (UniqueName: \"kubernetes.io/projected/23488e80-cefa-4039-86e5-a1fdb1d73838-kube-api-access-hbvm4\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bg222\" (UID: \"23488e80-cefa-4039-86e5-a1fdb1d73838\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bg222" Jul 09 15:14:58.387890 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:58.387786 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bg222" Jul 09 15:14:58.525340 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:58.525312 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-bg222"] Jul 09 15:14:58.528031 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:14:58.528002 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23488e80_cefa_4039_86e5_a1fdb1d73838.slice/crio-09b7af1adc1c96cc2b82c44a63036047618faf84016c19abbc4264ffd8673f68 WatchSource:0}: Error finding container 09b7af1adc1c96cc2b82c44a63036047618faf84016c19abbc4264ffd8673f68: Status 404 returned error can't find the container with id 09b7af1adc1c96cc2b82c44a63036047618faf84016c19abbc4264ffd8673f68 Jul 09 15:14:59.518873 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:59.518835 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bg222" event={"ID":"23488e80-cefa-4039-86e5-a1fdb1d73838","Type":"ContainerStarted","Data":"ec5972271d595b17a05d4b0231d7980e8bc7ac18c596f4ae36374207d73ebed9"} Jul 09 15:14:59.519276 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:59.518901 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bg222" event={"ID":"23488e80-cefa-4039-86e5-a1fdb1d73838","Type":"ContainerStarted","Data":"09b7af1adc1c96cc2b82c44a63036047618faf84016c19abbc4264ffd8673f68"} Jul 09 15:14:59.542213 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:14:59.542156 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bg222" podStartSLOduration=1.260446963 podStartE2EDuration="1.542138265s" totalImagesPullingTime="281.691302ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:14:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:14:59.53946412 +0000 UTC m=+746.269970637" watchObservedRunningTime="2026-07-09 15:14:59.542138265 +0000 UTC m=+746.272644756" Jul 09 15:15:02.613214 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:02.613175 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:15:03.320910 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:03.320865 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:15:08.388125 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:08.388084 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bg222" Jul 09 15:15:08.388699 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:08.388147 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bg222" Jul 09 15:15:08.401038 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:08.401010 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bg222" Jul 09 15:15:08.577139 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:08.577108 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bg222" Jul 09 15:15:11.649313 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:11.649271 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:15:11.709960 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:11.709925 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:15:14.876329 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:14.876289 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-8nhwg"] Jul 09 15:15:14.879073 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:14.879055 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nhwg" Jul 09 15:15:14.881989 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:14.881970 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 09 15:15:14.890955 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:14.890932 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-8nhwg"] Jul 09 15:15:14.972582 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:14.972545 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:15:14.982775 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:14.982748 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/710b727a-4d18-4f43-9273-4b11afd32afd-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8nhwg\" (UID: \"710b727a-4d18-4f43-9273-4b11afd32afd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nhwg" Jul 09 15:15:14.982946 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:14.982792 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/710b727a-4d18-4f43-9273-4b11afd32afd-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8nhwg\" (UID: \"710b727a-4d18-4f43-9273-4b11afd32afd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nhwg" Jul 09 15:15:14.982946 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:14.982846 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncsmp\" (UniqueName: \"kubernetes.io/projected/710b727a-4d18-4f43-9273-4b11afd32afd-kube-api-access-ncsmp\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8nhwg\" (UID: \"710b727a-4d18-4f43-9273-4b11afd32afd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nhwg" Jul 09 15:15:14.983131 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:14.982946 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/710b727a-4d18-4f43-9273-4b11afd32afd-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8nhwg\" (UID: \"710b727a-4d18-4f43-9273-4b11afd32afd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nhwg" Jul 09 15:15:14.983131 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:14.982992 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/710b727a-4d18-4f43-9273-4b11afd32afd-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8nhwg\" (UID: \"710b727a-4d18-4f43-9273-4b11afd32afd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nhwg" Jul 09 15:15:14.983131 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:14.983035 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/710b727a-4d18-4f43-9273-4b11afd32afd-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8nhwg\" (UID: \"710b727a-4d18-4f43-9273-4b11afd32afd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nhwg" Jul 09 15:15:15.084156 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:15.084120 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/710b727a-4d18-4f43-9273-4b11afd32afd-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8nhwg\" (UID: \"710b727a-4d18-4f43-9273-4b11afd32afd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nhwg" Jul 09 15:15:15.084355 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:15.084167 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/710b727a-4d18-4f43-9273-4b11afd32afd-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8nhwg\" (UID: \"710b727a-4d18-4f43-9273-4b11afd32afd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nhwg" Jul 09 15:15:15.084355 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:15.084185 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncsmp\" (UniqueName: \"kubernetes.io/projected/710b727a-4d18-4f43-9273-4b11afd32afd-kube-api-access-ncsmp\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8nhwg\" (UID: \"710b727a-4d18-4f43-9273-4b11afd32afd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nhwg" Jul 09 15:15:15.084355 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:15.084222 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/710b727a-4d18-4f43-9273-4b11afd32afd-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8nhwg\" (UID: \"710b727a-4d18-4f43-9273-4b11afd32afd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nhwg" Jul 09 15:15:15.084355 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:15.084241 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/710b727a-4d18-4f43-9273-4b11afd32afd-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8nhwg\" (UID: \"710b727a-4d18-4f43-9273-4b11afd32afd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nhwg" Jul 09 15:15:15.084355 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:15.084264 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/710b727a-4d18-4f43-9273-4b11afd32afd-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8nhwg\" (UID: \"710b727a-4d18-4f43-9273-4b11afd32afd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nhwg" Jul 09 15:15:15.084631 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:15.084582 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/710b727a-4d18-4f43-9273-4b11afd32afd-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8nhwg\" (UID: \"710b727a-4d18-4f43-9273-4b11afd32afd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nhwg" Jul 09 15:15:15.084631 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:15.084620 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/710b727a-4d18-4f43-9273-4b11afd32afd-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8nhwg\" (UID: \"710b727a-4d18-4f43-9273-4b11afd32afd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nhwg" Jul 09 15:15:15.084728 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:15.084683 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/710b727a-4d18-4f43-9273-4b11afd32afd-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8nhwg\" (UID: \"710b727a-4d18-4f43-9273-4b11afd32afd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nhwg" Jul 09 15:15:15.086607 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:15.086583 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/710b727a-4d18-4f43-9273-4b11afd32afd-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8nhwg\" (UID: \"710b727a-4d18-4f43-9273-4b11afd32afd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nhwg" Jul 09 15:15:15.086796 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:15.086779 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/710b727a-4d18-4f43-9273-4b11afd32afd-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8nhwg\" (UID: \"710b727a-4d18-4f43-9273-4b11afd32afd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nhwg" Jul 09 15:15:15.094423 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:15.094398 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncsmp\" (UniqueName: \"kubernetes.io/projected/710b727a-4d18-4f43-9273-4b11afd32afd-kube-api-access-ncsmp\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8nhwg\" (UID: \"710b727a-4d18-4f43-9273-4b11afd32afd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nhwg" Jul 09 15:15:15.190945 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:15.190854 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nhwg" Jul 09 15:15:15.329995 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:15.329970 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-8nhwg"] Jul 09 15:15:15.332484 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:15:15.332457 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod710b727a_4d18_4f43_9273_4b11afd32afd.slice/crio-981b492e639bb8fd129d2df9fe4c7390834ffb29f59552dd95d20ddd0afc9369 WatchSource:0}: Error finding container 981b492e639bb8fd129d2df9fe4c7390834ffb29f59552dd95d20ddd0afc9369: Status 404 returned error can't find the container with id 981b492e639bb8fd129d2df9fe4c7390834ffb29f59552dd95d20ddd0afc9369 Jul 09 15:15:15.601498 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:15.601463 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nhwg" event={"ID":"710b727a-4d18-4f43-9273-4b11afd32afd","Type":"ContainerStarted","Data":"cbd80458720a4e0af6a88a95bf2a845a748a4ecc00d03a6576dc56d3becbf3cd"} Jul 09 15:15:15.601658 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:15.601519 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nhwg" event={"ID":"710b727a-4d18-4f43-9273-4b11afd32afd","Type":"ContainerStarted","Data":"981b492e639bb8fd129d2df9fe4c7390834ffb29f59552dd95d20ddd0afc9369"} Jul 09 15:15:15.629933 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:15.629799 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nhwg" podStartSLOduration=1.429928249 podStartE2EDuration="1.629783749s" totalImagesPullingTime="199.8555ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:15:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:15:15.628353 +0000 UTC m=+762.358859515" watchObservedRunningTime="2026-07-09 15:15:15.629783749 +0000 UTC m=+762.360290240" Jul 09 15:15:19.947447 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:19.947410 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:15:21.807830 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:21.807783 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:15:25.191310 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:25.191269 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nhwg" Jul 09 15:15:25.191800 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:25.191555 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nhwg" Jul 09 15:15:25.204238 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:25.204210 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nhwg" Jul 09 15:15:25.668149 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:25.668122 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nhwg" Jul 09 15:15:29.693333 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:15:29.693287 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:17:05.002363 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:17:05.002325 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:17:05.090621 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:17:05.090590 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:17:06.142360 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:17:06.142322 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:17:06.239619 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:17:06.239582 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:17:33.732633 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:17:33.732556 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hml7m_84a10578-4913-456b-9fb7-e6440914b677/ovn-acl-logging/0.log" Jul 09 15:17:33.735673 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:17:33.735648 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hml7m_84a10578-4913-456b-9fb7-e6440914b677/ovn-acl-logging/0.log" Jul 09 15:17:36.030396 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:17:36.030360 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:18:18.657845 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:18.657782 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:18:18.758146 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:18.758107 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-58d8c4744b-lbvjp"] Jul 09 15:18:18.758351 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:18.758316 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-58d8c4744b-lbvjp" podUID="9afbd50e-a1fe-4c7b-85a8-212e8496c259" containerName="manager" containerID="cri-o://fa82d13f4499690dfc67c45a5c0fed6d2c08a4fe68f7cfaf6e5b02c81acbbe7d" gracePeriod=30 Jul 09 15:18:19.020275 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:19.020247 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58d8c4744b-lbvjp" Jul 09 15:18:19.077748 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:19.077710 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9afbd50e-a1fe-4c7b-85a8-212e8496c259-webhook-cert\") pod \"9afbd50e-a1fe-4c7b-85a8-212e8496c259\" (UID: \"9afbd50e-a1fe-4c7b-85a8-212e8496c259\") " Jul 09 15:18:19.077966 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:19.077881 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8lg86\" (UniqueName: \"kubernetes.io/projected/9afbd50e-a1fe-4c7b-85a8-212e8496c259-kube-api-access-8lg86\") pod \"9afbd50e-a1fe-4c7b-85a8-212e8496c259\" (UID: \"9afbd50e-a1fe-4c7b-85a8-212e8496c259\") " Jul 09 15:18:19.079960 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:19.079924 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9afbd50e-a1fe-4c7b-85a8-212e8496c259-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9afbd50e-a1fe-4c7b-85a8-212e8496c259" (UID: "9afbd50e-a1fe-4c7b-85a8-212e8496c259"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 15:18:19.080105 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:19.080023 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9afbd50e-a1fe-4c7b-85a8-212e8496c259-kube-api-access-8lg86" (OuterVolumeSpecName: "kube-api-access-8lg86") pod "9afbd50e-a1fe-4c7b-85a8-212e8496c259" (UID: "9afbd50e-a1fe-4c7b-85a8-212e8496c259"). InnerVolumeSpecName "kube-api-access-8lg86". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:18:19.179386 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:19.179303 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lg86\" (UniqueName: \"kubernetes.io/projected/9afbd50e-a1fe-4c7b-85a8-212e8496c259-kube-api-access-8lg86\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:18:19.179386 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:19.179333 2567 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9afbd50e-a1fe-4c7b-85a8-212e8496c259-webhook-cert\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:18:19.567623 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:19.567588 2567 generic.go:356] "Generic (PLEG): container finished" podID="9afbd50e-a1fe-4c7b-85a8-212e8496c259" containerID="fa82d13f4499690dfc67c45a5c0fed6d2c08a4fe68f7cfaf6e5b02c81acbbe7d" exitCode=0 Jul 09 15:18:19.567789 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:19.567637 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58d8c4744b-lbvjp" Jul 09 15:18:19.567789 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:19.567663 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58d8c4744b-lbvjp" event={"ID":"9afbd50e-a1fe-4c7b-85a8-212e8496c259","Type":"ContainerDied","Data":"fa82d13f4499690dfc67c45a5c0fed6d2c08a4fe68f7cfaf6e5b02c81acbbe7d"} Jul 09 15:18:19.567789 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:19.567703 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58d8c4744b-lbvjp" event={"ID":"9afbd50e-a1fe-4c7b-85a8-212e8496c259","Type":"ContainerDied","Data":"9dd490f1de803521b233f7a46be1eaa59995c66773477d9485a5ddf9f7da85c0"} Jul 09 15:18:19.567789 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:19.567724 2567 scope.go:117] "RemoveContainer" containerID="fa82d13f4499690dfc67c45a5c0fed6d2c08a4fe68f7cfaf6e5b02c81acbbe7d" Jul 09 15:18:19.577189 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:19.577166 2567 scope.go:117] "RemoveContainer" containerID="fa82d13f4499690dfc67c45a5c0fed6d2c08a4fe68f7cfaf6e5b02c81acbbe7d" Jul 09 15:18:19.577465 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:18:19.577445 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa82d13f4499690dfc67c45a5c0fed6d2c08a4fe68f7cfaf6e5b02c81acbbe7d\": container with ID starting with fa82d13f4499690dfc67c45a5c0fed6d2c08a4fe68f7cfaf6e5b02c81acbbe7d not found: ID does not exist" containerID="fa82d13f4499690dfc67c45a5c0fed6d2c08a4fe68f7cfaf6e5b02c81acbbe7d" Jul 09 15:18:19.577539 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:19.577472 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa82d13f4499690dfc67c45a5c0fed6d2c08a4fe68f7cfaf6e5b02c81acbbe7d"} err="failed to get container status \"fa82d13f4499690dfc67c45a5c0fed6d2c08a4fe68f7cfaf6e5b02c81acbbe7d\": rpc error: code = NotFound desc = could not find container \"fa82d13f4499690dfc67c45a5c0fed6d2c08a4fe68f7cfaf6e5b02c81acbbe7d\": container with ID starting with fa82d13f4499690dfc67c45a5c0fed6d2c08a4fe68f7cfaf6e5b02c81acbbe7d not found: ID does not exist" Jul 09 15:18:19.590354 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:19.590327 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-58d8c4744b-lbvjp"] Jul 09 15:18:19.594285 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:19.594260 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-58d8c4744b-lbvjp"] Jul 09 15:18:19.828903 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:19.828795 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9afbd50e-a1fe-4c7b-85a8-212e8496c259" path="/var/lib/kubelet/pods/9afbd50e-a1fe-4c7b-85a8-212e8496c259/volumes" Jul 09 15:18:20.279245 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:20.279214 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:18:20.397077 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:20.397047 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-58d8c4744b-nrkr4"] Jul 09 15:18:20.397725 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:20.397700 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9afbd50e-a1fe-4c7b-85a8-212e8496c259" containerName="manager" Jul 09 15:18:20.397725 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:20.397722 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9afbd50e-a1fe-4c7b-85a8-212e8496c259" containerName="manager" Jul 09 15:18:20.397959 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:20.397882 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9afbd50e-a1fe-4c7b-85a8-212e8496c259" containerName="manager" Jul 09 15:18:20.402547 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:20.402525 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58d8c4744b-nrkr4" Jul 09 15:18:20.406235 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:20.406214 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 09 15:18:20.406342 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:20.406241 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 09 15:18:20.406342 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:20.406215 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5g87n\"" Jul 09 15:18:20.411459 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:20.411425 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58d8c4744b-nrkr4"] Jul 09 15:18:20.490248 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:20.490209 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbr4q\" (UniqueName: \"kubernetes.io/projected/1c767830-cb4b-44a3-9b9f-3bb5f0e55ecc-kube-api-access-nbr4q\") pod \"maas-controller-58d8c4744b-nrkr4\" (UID: \"1c767830-cb4b-44a3-9b9f-3bb5f0e55ecc\") " pod="opendatahub/maas-controller-58d8c4744b-nrkr4" Jul 09 15:18:20.490421 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:20.490258 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1c767830-cb4b-44a3-9b9f-3bb5f0e55ecc-webhook-cert\") pod \"maas-controller-58d8c4744b-nrkr4\" (UID: \"1c767830-cb4b-44a3-9b9f-3bb5f0e55ecc\") " pod="opendatahub/maas-controller-58d8c4744b-nrkr4" Jul 09 15:18:20.591575 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:20.591463 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1c767830-cb4b-44a3-9b9f-3bb5f0e55ecc-webhook-cert\") pod \"maas-controller-58d8c4744b-nrkr4\" (UID: \"1c767830-cb4b-44a3-9b9f-3bb5f0e55ecc\") " pod="opendatahub/maas-controller-58d8c4744b-nrkr4" Jul 09 15:18:20.591765 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:20.591579 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbr4q\" (UniqueName: \"kubernetes.io/projected/1c767830-cb4b-44a3-9b9f-3bb5f0e55ecc-kube-api-access-nbr4q\") pod \"maas-controller-58d8c4744b-nrkr4\" (UID: \"1c767830-cb4b-44a3-9b9f-3bb5f0e55ecc\") " pod="opendatahub/maas-controller-58d8c4744b-nrkr4" Jul 09 15:18:20.594096 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:20.594065 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1c767830-cb4b-44a3-9b9f-3bb5f0e55ecc-webhook-cert\") pod \"maas-controller-58d8c4744b-nrkr4\" (UID: \"1c767830-cb4b-44a3-9b9f-3bb5f0e55ecc\") " pod="opendatahub/maas-controller-58d8c4744b-nrkr4" Jul 09 15:18:20.601015 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:20.600989 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbr4q\" (UniqueName: \"kubernetes.io/projected/1c767830-cb4b-44a3-9b9f-3bb5f0e55ecc-kube-api-access-nbr4q\") pod \"maas-controller-58d8c4744b-nrkr4\" (UID: \"1c767830-cb4b-44a3-9b9f-3bb5f0e55ecc\") " pod="opendatahub/maas-controller-58d8c4744b-nrkr4" Jul 09 15:18:20.713811 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:20.713762 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58d8c4744b-nrkr4" Jul 09 15:18:20.855380 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:20.855354 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58d8c4744b-nrkr4"] Jul 09 15:18:20.857641 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:18:20.857612 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c767830_cb4b_44a3_9b9f_3bb5f0e55ecc.slice/crio-74c8c07031e99c7cbb5f40eb1e73b26e062c9f3096fc41502e2c196b52646a39 WatchSource:0}: Error finding container 74c8c07031e99c7cbb5f40eb1e73b26e062c9f3096fc41502e2c196b52646a39: Status 404 returned error can't find the container with id 74c8c07031e99c7cbb5f40eb1e73b26e062c9f3096fc41502e2c196b52646a39 Jul 09 15:18:21.580413 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:21.580309 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58d8c4744b-nrkr4" event={"ID":"1c767830-cb4b-44a3-9b9f-3bb5f0e55ecc","Type":"ContainerStarted","Data":"1d9e33c7a06b6412bf3e25a458dec94f6f0cb47379935a7af6c379787cf4406d"} Jul 09 15:18:21.580413 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:21.580386 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58d8c4744b-nrkr4" event={"ID":"1c767830-cb4b-44a3-9b9f-3bb5f0e55ecc","Type":"ContainerStarted","Data":"74c8c07031e99c7cbb5f40eb1e73b26e062c9f3096fc41502e2c196b52646a39"} Jul 09 15:18:21.580636 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:21.580424 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-58d8c4744b-nrkr4" Jul 09 15:18:21.600079 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:21.600026 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-58d8c4744b-nrkr4" podStartSLOduration=1.188835642 podStartE2EDuration="1.600011166s" totalImagesPullingTime="411.175524ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:18:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:18:21.598037198 +0000 UTC m=+948.328543689" watchObservedRunningTime="2026-07-09 15:18:21.600011166 +0000 UTC m=+948.330517688" Jul 09 15:18:32.591033 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:32.591001 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-58d8c4744b-nrkr4" Jul 09 15:18:43.774452 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:18:43.774415 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:19:03.884515 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:19:03.884475 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:19:04.076777 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:19:04.076744 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:19:04.276286 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:19:04.276247 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:19:13.108794 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:19:13.108752 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:19:13.365094 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:19:13.364999 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:19:13.560160 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:19:13.560119 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:19:13.720525 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:19:13.720493 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:19:21.253932 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:19:21.253897 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:19:21.372744 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:19:21.372703 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:21:53.697970 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:21:53.697933 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:21:53.767018 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:21:53.766970 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:22:33.766190 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:22:33.766154 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hml7m_84a10578-4913-456b-9fb7-e6440914b677/ovn-acl-logging/0.log" Jul 09 15:22:33.773514 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:22:33.773490 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hml7m_84a10578-4913-456b-9fb7-e6440914b677/ovn-acl-logging/0.log" Jul 09 15:23:04.878928 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:23:04.878893 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:23:25.473328 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:23:25.473239 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9pg4"] Jul 09 15:23:25.473735 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:23:25.473512 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9pg4" podUID="9ce8088a-526c-42c6-b948-c4b0be715572" containerName="manager" containerID="cri-o://4766cf8560a6f860da04322c8a9ec69d1e20e9a51f5462f5cf5f213247a37a48" gracePeriod=10 Jul 09 15:23:26.209578 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:23:26.209547 2567 generic.go:356] "Generic (PLEG): container finished" podID="9ce8088a-526c-42c6-b948-c4b0be715572" containerID="4766cf8560a6f860da04322c8a9ec69d1e20e9a51f5462f5cf5f213247a37a48" exitCode=0 Jul 09 15:23:26.209738 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:23:26.209625 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9pg4" event={"ID":"9ce8088a-526c-42c6-b948-c4b0be715572","Type":"ContainerDied","Data":"4766cf8560a6f860da04322c8a9ec69d1e20e9a51f5462f5cf5f213247a37a48"} Jul 09 15:23:26.832950 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:23:26.832922 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9pg4" Jul 09 15:23:26.931555 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:23:26.931522 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9ce8088a-526c-42c6-b948-c4b0be715572-extensions-socket-volume\") pod \"9ce8088a-526c-42c6-b948-c4b0be715572\" (UID: \"9ce8088a-526c-42c6-b948-c4b0be715572\") " Jul 09 15:23:26.931759 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:23:26.931580 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cf979\" (UniqueName: \"kubernetes.io/projected/9ce8088a-526c-42c6-b948-c4b0be715572-kube-api-access-cf979\") pod \"9ce8088a-526c-42c6-b948-c4b0be715572\" (UID: \"9ce8088a-526c-42c6-b948-c4b0be715572\") " Jul 09 15:23:26.932011 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:23:26.931981 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ce8088a-526c-42c6-b948-c4b0be715572-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9ce8088a-526c-42c6-b948-c4b0be715572" (UID: "9ce8088a-526c-42c6-b948-c4b0be715572"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:23:26.933644 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:23:26.933620 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ce8088a-526c-42c6-b948-c4b0be715572-kube-api-access-cf979" (OuterVolumeSpecName: "kube-api-access-cf979") pod "9ce8088a-526c-42c6-b948-c4b0be715572" (UID: "9ce8088a-526c-42c6-b948-c4b0be715572"). InnerVolumeSpecName "kube-api-access-cf979". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:23:27.032994 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:23:27.032958 2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9ce8088a-526c-42c6-b948-c4b0be715572-extensions-socket-volume\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:23:27.032994 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:23:27.032989 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cf979\" (UniqueName: \"kubernetes.io/projected/9ce8088a-526c-42c6-b948-c4b0be715572-kube-api-access-cf979\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:23:27.216300 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:23:27.216271 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9pg4" Jul 09 15:23:27.216300 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:23:27.216294 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9pg4" event={"ID":"9ce8088a-526c-42c6-b948-c4b0be715572","Type":"ContainerDied","Data":"57424dc0012cc6f9eeb900cc9135dc9d8d5df6bc82eaf0cba0cef43aba65ee1e"} Jul 09 15:23:27.216510 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:23:27.216350 2567 scope.go:117] "RemoveContainer" containerID="4766cf8560a6f860da04322c8a9ec69d1e20e9a51f5462f5cf5f213247a37a48" Jul 09 15:23:27.239429 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:23:27.239393 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9pg4"] Jul 09 15:23:27.241393 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:23:27.241368 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9pg4"] Jul 09 15:23:27.829213 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:23:27.829180 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ce8088a-526c-42c6-b948-c4b0be715572" path="/var/lib/kubelet/pods/9ce8088a-526c-42c6-b948-c4b0be715572/volumes" Jul 09 15:24:11.841058 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:24:11.841018 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:24:27.365061 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:24:27.365024 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5wr6x"] Jul 09 15:24:27.367265 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:24:27.365457 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ce8088a-526c-42c6-b948-c4b0be715572" containerName="manager" Jul 09 15:24:27.367265 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:24:27.365468 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ce8088a-526c-42c6-b948-c4b0be715572" containerName="manager" Jul 09 15:24:27.367265 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:24:27.365563 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ce8088a-526c-42c6-b948-c4b0be715572" containerName="manager" Jul 09 15:24:27.368522 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:24:27.368505 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5wr6x" Jul 09 15:24:27.376045 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:24:27.376013 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 09 15:24:27.376232 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:24:27.376216 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-xrlb8\"" Jul 09 15:24:27.376478 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:24:27.376455 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 09 15:24:27.400368 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:24:27.400336 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5wr6x"] Jul 09 15:24:27.442875 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:24:27.442840 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpbbs\" (UniqueName: \"kubernetes.io/projected/9ec5bec4-baca-4251-9fdc-0085ccbc5776-kube-api-access-bpbbs\") pod \"kuadrant-operator-controller-manager-86878c769f-5wr6x\" (UID: \"9ec5bec4-baca-4251-9fdc-0085ccbc5776\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5wr6x" Jul 09 15:24:27.443066 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:24:27.442924 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9ec5bec4-baca-4251-9fdc-0085ccbc5776-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-5wr6x\" (UID: \"9ec5bec4-baca-4251-9fdc-0085ccbc5776\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5wr6x" Jul 09 15:24:27.543636 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:24:27.543594 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpbbs\" (UniqueName: \"kubernetes.io/projected/9ec5bec4-baca-4251-9fdc-0085ccbc5776-kube-api-access-bpbbs\") pod \"kuadrant-operator-controller-manager-86878c769f-5wr6x\" (UID: \"9ec5bec4-baca-4251-9fdc-0085ccbc5776\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5wr6x" Jul 09 15:24:27.543869 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:24:27.543705 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9ec5bec4-baca-4251-9fdc-0085ccbc5776-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-5wr6x\" (UID: \"9ec5bec4-baca-4251-9fdc-0085ccbc5776\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5wr6x" Jul 09 15:24:27.544099 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:24:27.544079 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9ec5bec4-baca-4251-9fdc-0085ccbc5776-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-5wr6x\" (UID: \"9ec5bec4-baca-4251-9fdc-0085ccbc5776\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5wr6x" Jul 09 15:24:27.558405 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:24:27.558378 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpbbs\" (UniqueName: \"kubernetes.io/projected/9ec5bec4-baca-4251-9fdc-0085ccbc5776-kube-api-access-bpbbs\") pod \"kuadrant-operator-controller-manager-86878c769f-5wr6x\" (UID: \"9ec5bec4-baca-4251-9fdc-0085ccbc5776\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5wr6x" Jul 09 15:24:27.678850 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:24:27.678743 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5wr6x" Jul 09 15:24:27.842713 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:24:27.842681 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5wr6x"] Jul 09 15:24:27.844552 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:24:27.844526 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ec5bec4_baca_4251_9fdc_0085ccbc5776.slice/crio-e6d41ff80e9e16033d8c54e6c2202f5478d0dc32e6896370b279c36c11ade2f5 WatchSource:0}: Error finding container e6d41ff80e9e16033d8c54e6c2202f5478d0dc32e6896370b279c36c11ade2f5: Status 404 returned error can't find the container with id e6d41ff80e9e16033d8c54e6c2202f5478d0dc32e6896370b279c36c11ade2f5 Jul 09 15:24:28.539545 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:24:28.539504 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5wr6x" event={"ID":"9ec5bec4-baca-4251-9fdc-0085ccbc5776","Type":"ContainerStarted","Data":"958fbdb28ff5001ee04219388343a6c16bd679c049548618b7a2fc57dc4a0565"} Jul 09 15:24:28.539981 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:24:28.539571 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5wr6x" event={"ID":"9ec5bec4-baca-4251-9fdc-0085ccbc5776","Type":"ContainerStarted","Data":"e6d41ff80e9e16033d8c54e6c2202f5478d0dc32e6896370b279c36c11ade2f5"} Jul 09 15:24:28.539981 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:24:28.539665 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5wr6x" Jul 09 15:24:28.584590 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:24:28.584535 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5wr6x" podStartSLOduration=1.5845168250000001 podStartE2EDuration="1.584516825s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:24:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:24:28.582107241 +0000 UTC m=+1315.312613733" watchObservedRunningTime="2026-07-09 15:24:28.584516825 +0000 UTC m=+1315.315023320" Jul 09 15:24:39.547358 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:24:39.547321 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5wr6x" Jul 09 15:26:09.150467 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:26:09.150432 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:26:09.219888 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:26:09.219851 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:27:33.800514 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:27:33.800480 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hml7m_84a10578-4913-456b-9fb7-e6440914b677/ovn-acl-logging/0.log" Jul 09 15:27:33.808255 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:27:33.808232 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hml7m_84a10578-4913-456b-9fb7-e6440914b677/ovn-acl-logging/0.log" Jul 09 15:28:07.121504 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:28:07.121426 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:29:01.939905 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:29:01.939867 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:30:32.475488 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:32.475452 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:30:34.526643 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:34.526601 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:30:40.185492 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.185454 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:30:40.316325 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.316284 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:30:40.495995 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.495961 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn"] Jul 09 15:30:40.499650 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.499624 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" Jul 09 15:30:40.502882 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.502857 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-1829e532-openshift-default-dockercfg-wpcwz\"" Jul 09 15:30:40.526314 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.526280 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn"] Jul 09 15:30:40.530636 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.530604 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12f4570b-e830-4f29-ab27-6be60f4e433b-credential-socket\") pod \"e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn\" (UID: \"12f4570b-e830-4f29-ab27-6be60f4e433b\") " pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" Jul 09 15:30:40.530843 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.530678 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12f4570b-e830-4f29-ab27-6be60f4e433b-istio-podinfo\") pod \"e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn\" (UID: \"12f4570b-e830-4f29-ab27-6be60f4e433b\") " pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" Jul 09 15:30:40.530843 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.530703 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12f4570b-e830-4f29-ab27-6be60f4e433b-istiod-ca-cert\") pod \"e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn\" (UID: \"12f4570b-e830-4f29-ab27-6be60f4e433b\") " pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" Jul 09 15:30:40.530843 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.530727 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkx9w\" (UniqueName: \"kubernetes.io/projected/12f4570b-e830-4f29-ab27-6be60f4e433b-kube-api-access-tkx9w\") pod \"e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn\" (UID: \"12f4570b-e830-4f29-ab27-6be60f4e433b\") " pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" Jul 09 15:30:40.530843 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.530789 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12f4570b-e830-4f29-ab27-6be60f4e433b-workload-certs\") pod \"e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn\" (UID: \"12f4570b-e830-4f29-ab27-6be60f4e433b\") " pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" Jul 09 15:30:40.531072 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.530891 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12f4570b-e830-4f29-ab27-6be60f4e433b-istio-data\") pod \"e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn\" (UID: \"12f4570b-e830-4f29-ab27-6be60f4e433b\") " pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" Jul 09 15:30:40.531072 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.530924 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12f4570b-e830-4f29-ab27-6be60f4e433b-workload-socket\") pod \"e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn\" (UID: \"12f4570b-e830-4f29-ab27-6be60f4e433b\") " pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" Jul 09 15:30:40.531072 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.530964 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12f4570b-e830-4f29-ab27-6be60f4e433b-istio-envoy\") pod \"e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn\" (UID: \"12f4570b-e830-4f29-ab27-6be60f4e433b\") " pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" Jul 09 15:30:40.531189 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.531083 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12f4570b-e830-4f29-ab27-6be60f4e433b-istio-token\") pod \"e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn\" (UID: \"12f4570b-e830-4f29-ab27-6be60f4e433b\") " pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" Jul 09 15:30:40.557098 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.557057 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:30:40.632238 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.632206 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12f4570b-e830-4f29-ab27-6be60f4e433b-credential-socket\") pod \"e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn\" (UID: \"12f4570b-e830-4f29-ab27-6be60f4e433b\") " pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" Jul 09 15:30:40.632437 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.632262 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12f4570b-e830-4f29-ab27-6be60f4e433b-istio-podinfo\") pod \"e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn\" (UID: \"12f4570b-e830-4f29-ab27-6be60f4e433b\") " pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" Jul 09 15:30:40.632437 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.632281 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12f4570b-e830-4f29-ab27-6be60f4e433b-istiod-ca-cert\") pod \"e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn\" (UID: \"12f4570b-e830-4f29-ab27-6be60f4e433b\") " pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" Jul 09 15:30:40.632437 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.632299 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkx9w\" (UniqueName: \"kubernetes.io/projected/12f4570b-e830-4f29-ab27-6be60f4e433b-kube-api-access-tkx9w\") pod \"e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn\" (UID: \"12f4570b-e830-4f29-ab27-6be60f4e433b\") " pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" Jul 09 15:30:40.632437 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.632320 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12f4570b-e830-4f29-ab27-6be60f4e433b-workload-certs\") pod \"e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn\" (UID: \"12f4570b-e830-4f29-ab27-6be60f4e433b\") " pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" Jul 09 15:30:40.632437 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.632358 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12f4570b-e830-4f29-ab27-6be60f4e433b-istio-data\") pod \"e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn\" (UID: \"12f4570b-e830-4f29-ab27-6be60f4e433b\") " pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" Jul 09 15:30:40.632437 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.632379 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12f4570b-e830-4f29-ab27-6be60f4e433b-workload-socket\") pod \"e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn\" (UID: \"12f4570b-e830-4f29-ab27-6be60f4e433b\") " pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" Jul 09 15:30:40.632437 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.632416 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12f4570b-e830-4f29-ab27-6be60f4e433b-istio-envoy\") pod \"e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn\" (UID: \"12f4570b-e830-4f29-ab27-6be60f4e433b\") " pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" Jul 09 15:30:40.632800 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.632504 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12f4570b-e830-4f29-ab27-6be60f4e433b-istio-token\") pod \"e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn\" (UID: \"12f4570b-e830-4f29-ab27-6be60f4e433b\") " pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" Jul 09 15:30:40.632800 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.632763 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12f4570b-e830-4f29-ab27-6be60f4e433b-credential-socket\") pod \"e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn\" (UID: \"12f4570b-e830-4f29-ab27-6be60f4e433b\") " pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" Jul 09 15:30:40.632967 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.632803 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12f4570b-e830-4f29-ab27-6be60f4e433b-istio-data\") pod \"e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn\" (UID: \"12f4570b-e830-4f29-ab27-6be60f4e433b\") " pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" Jul 09 15:30:40.632967 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.632864 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12f4570b-e830-4f29-ab27-6be60f4e433b-workload-socket\") pod \"e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn\" (UID: \"12f4570b-e830-4f29-ab27-6be60f4e433b\") " pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" Jul 09 15:30:40.632967 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.632899 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12f4570b-e830-4f29-ab27-6be60f4e433b-workload-certs\") pod \"e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn\" (UID: \"12f4570b-e830-4f29-ab27-6be60f4e433b\") " pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" Jul 09 15:30:40.633375 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.633352 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12f4570b-e830-4f29-ab27-6be60f4e433b-istiod-ca-cert\") pod \"e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn\" (UID: \"12f4570b-e830-4f29-ab27-6be60f4e433b\") " pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" Jul 09 15:30:40.634698 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.634680 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12f4570b-e830-4f29-ab27-6be60f4e433b-istio-envoy\") pod \"e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn\" (UID: \"12f4570b-e830-4f29-ab27-6be60f4e433b\") " pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" Jul 09 15:30:40.634873 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.634854 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12f4570b-e830-4f29-ab27-6be60f4e433b-istio-podinfo\") pod \"e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn\" (UID: \"12f4570b-e830-4f29-ab27-6be60f4e433b\") " pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" Jul 09 15:30:40.643359 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.643330 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkx9w\" (UniqueName: \"kubernetes.io/projected/12f4570b-e830-4f29-ab27-6be60f4e433b-kube-api-access-tkx9w\") pod \"e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn\" (UID: \"12f4570b-e830-4f29-ab27-6be60f4e433b\") " pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" Jul 09 15:30:40.643695 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.643670 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12f4570b-e830-4f29-ab27-6be60f4e433b-istio-token\") pod \"e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn\" (UID: \"12f4570b-e830-4f29-ab27-6be60f4e433b\") " pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" Jul 09 15:30:40.686356 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.686320 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:30:40.810923 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.810805 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" Jul 09 15:30:40.974590 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.974561 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn"] Jul 09 15:30:40.975882 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:30:40.975855 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12f4570b_e830_4f29_ab27_6be60f4e433b.slice/crio-9f6797f9306c4173e19b453bf999c61789b88099e032100fb85555de73a98a27 WatchSource:0}: Error finding container 9f6797f9306c4173e19b453bf999c61789b88099e032100fb85555de73a98a27: Status 404 returned error can't find the container with id 9f6797f9306c4173e19b453bf999c61789b88099e032100fb85555de73a98a27 Jul 09 15:30:40.978046 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.978014 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:30:40.978131 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.978086 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:30:40.978131 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:40.978117 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:30:41.505004 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:41.504961 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" event={"ID":"12f4570b-e830-4f29-ab27-6be60f4e433b","Type":"ContainerStarted","Data":"8c49365821aef4554f33438664f1dd96bb27155e0d897f10ed24cd7e8275fca6"} Jul 09 15:30:41.505441 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:41.505033 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" event={"ID":"12f4570b-e830-4f29-ab27-6be60f4e433b","Type":"ContainerStarted","Data":"9f6797f9306c4173e19b453bf999c61789b88099e032100fb85555de73a98a27"} Jul 09 15:30:41.540802 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:41.540739 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" podStartSLOduration=1.540721453 podStartE2EDuration="1.540721453s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:30:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:30:41.537481071 +0000 UTC m=+1688.267987561" watchObservedRunningTime="2026-07-09 15:30:41.540721453 +0000 UTC m=+1688.271227934" Jul 09 15:30:41.721209 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:41.719637 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:30:41.811005 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:41.810911 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" Jul 09 15:30:42.013423 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:42.013392 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:30:42.184412 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:42.184298 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:30:42.325850 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:42.325793 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:30:42.816398 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:42.816368 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" Jul 09 15:30:42.930252 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:42.930210 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:30:43.047165 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:43.047123 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:30:43.518149 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:43.518118 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" Jul 09 15:30:43.519258 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:43.519241 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" Jul 09 15:30:44.652170 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:44.652136 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:30:46.924453 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:46.924414 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:30:47.153849 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:47.153782 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn"] Jul 09 15:30:47.154077 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:47.154045 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" podUID="12f4570b-e830-4f29-ab27-6be60f4e433b" containerName="istio-proxy" containerID="cri-o://8c49365821aef4554f33438664f1dd96bb27155e0d897f10ed24cd7e8275fca6" gracePeriod=30 Jul 09 15:30:47.252758 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:47.252725 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:30:47.491934 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:47.491897 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:30:47.833125 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:47.833090 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:30:49.241048 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:49.241010 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:30:52.907404 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:52.907342 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" Jul 09 15:30:53.080939 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.080899 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12f4570b-e830-4f29-ab27-6be60f4e433b-workload-certs\") pod \"12f4570b-e830-4f29-ab27-6be60f4e433b\" (UID: \"12f4570b-e830-4f29-ab27-6be60f4e433b\") " Jul 09 15:30:53.081145 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.080984 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12f4570b-e830-4f29-ab27-6be60f4e433b-workload-socket\") pod \"12f4570b-e830-4f29-ab27-6be60f4e433b\" (UID: \"12f4570b-e830-4f29-ab27-6be60f4e433b\") " Jul 09 15:30:53.081145 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.081040 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12f4570b-e830-4f29-ab27-6be60f4e433b-istiod-ca-cert\") pod \"12f4570b-e830-4f29-ab27-6be60f4e433b\" (UID: \"12f4570b-e830-4f29-ab27-6be60f4e433b\") " Jul 09 15:30:53.081145 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.081069 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12f4570b-e830-4f29-ab27-6be60f4e433b-istio-envoy\") pod \"12f4570b-e830-4f29-ab27-6be60f4e433b\" (UID: \"12f4570b-e830-4f29-ab27-6be60f4e433b\") " Jul 09 15:30:53.081145 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.081108 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkx9w\" (UniqueName: \"kubernetes.io/projected/12f4570b-e830-4f29-ab27-6be60f4e433b-kube-api-access-tkx9w\") pod \"12f4570b-e830-4f29-ab27-6be60f4e433b\" (UID: \"12f4570b-e830-4f29-ab27-6be60f4e433b\") " Jul 09 15:30:53.081369 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.081227 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12f4570b-e830-4f29-ab27-6be60f4e433b-istio-data\") pod \"12f4570b-e830-4f29-ab27-6be60f4e433b\" (UID: \"12f4570b-e830-4f29-ab27-6be60f4e433b\") " Jul 09 15:30:53.081369 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.081326 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12f4570b-e830-4f29-ab27-6be60f4e433b-istio-token\") pod \"12f4570b-e830-4f29-ab27-6be60f4e433b\" (UID: \"12f4570b-e830-4f29-ab27-6be60f4e433b\") " Jul 09 15:30:53.081369 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.081353 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12f4570b-e830-4f29-ab27-6be60f4e433b-credential-socket\") pod \"12f4570b-e830-4f29-ab27-6be60f4e433b\" (UID: \"12f4570b-e830-4f29-ab27-6be60f4e433b\") " Jul 09 15:30:53.081521 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.081229 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12f4570b-e830-4f29-ab27-6be60f4e433b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "12f4570b-e830-4f29-ab27-6be60f4e433b" (UID: "12f4570b-e830-4f29-ab27-6be60f4e433b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:30:53.081521 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.081358 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12f4570b-e830-4f29-ab27-6be60f4e433b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "12f4570b-e830-4f29-ab27-6be60f4e433b" (UID: "12f4570b-e830-4f29-ab27-6be60f4e433b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:30:53.081521 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.081396 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12f4570b-e830-4f29-ab27-6be60f4e433b-istio-podinfo\") pod \"12f4570b-e830-4f29-ab27-6be60f4e433b\" (UID: \"12f4570b-e830-4f29-ab27-6be60f4e433b\") " Jul 09 15:30:53.081521 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.081424 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12f4570b-e830-4f29-ab27-6be60f4e433b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "12f4570b-e830-4f29-ab27-6be60f4e433b" (UID: "12f4570b-e830-4f29-ab27-6be60f4e433b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:30:53.081760 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.081722 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12f4570b-e830-4f29-ab27-6be60f4e433b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "12f4570b-e830-4f29-ab27-6be60f4e433b" (UID: "12f4570b-e830-4f29-ab27-6be60f4e433b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:30:53.081983 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.081965 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12f4570b-e830-4f29-ab27-6be60f4e433b-istiod-ca-cert\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:30:53.082059 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.081987 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12f4570b-e830-4f29-ab27-6be60f4e433b-credential-socket\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:30:53.082059 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.082001 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12f4570b-e830-4f29-ab27-6be60f4e433b-workload-certs\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:30:53.082059 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.082016 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12f4570b-e830-4f29-ab27-6be60f4e433b-workload-socket\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:30:53.082241 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.082203 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12f4570b-e830-4f29-ab27-6be60f4e433b-istio-data" (OuterVolumeSpecName: "istio-data") pod "12f4570b-e830-4f29-ab27-6be60f4e433b" (UID: "12f4570b-e830-4f29-ab27-6be60f4e433b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:30:53.083608 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.083585 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/12f4570b-e830-4f29-ab27-6be60f4e433b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "12f4570b-e830-4f29-ab27-6be60f4e433b" (UID: "12f4570b-e830-4f29-ab27-6be60f4e433b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 15:30:53.083716 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.083659 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12f4570b-e830-4f29-ab27-6be60f4e433b-kube-api-access-tkx9w" (OuterVolumeSpecName: "kube-api-access-tkx9w") pod "12f4570b-e830-4f29-ab27-6be60f4e433b" (UID: "12f4570b-e830-4f29-ab27-6be60f4e433b"). InnerVolumeSpecName "kube-api-access-tkx9w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:30:53.083767 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.083728 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12f4570b-e830-4f29-ab27-6be60f4e433b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "12f4570b-e830-4f29-ab27-6be60f4e433b" (UID: "12f4570b-e830-4f29-ab27-6be60f4e433b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:30:53.083767 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.083745 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12f4570b-e830-4f29-ab27-6be60f4e433b-istio-token" (OuterVolumeSpecName: "istio-token") pod "12f4570b-e830-4f29-ab27-6be60f4e433b" (UID: "12f4570b-e830-4f29-ab27-6be60f4e433b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:30:53.183129 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.183047 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12f4570b-e830-4f29-ab27-6be60f4e433b-istio-envoy\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:30:53.183129 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.183077 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkx9w\" (UniqueName: \"kubernetes.io/projected/12f4570b-e830-4f29-ab27-6be60f4e433b-kube-api-access-tkx9w\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:30:53.183129 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.183088 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12f4570b-e830-4f29-ab27-6be60f4e433b-istio-data\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:30:53.183129 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.183097 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12f4570b-e830-4f29-ab27-6be60f4e433b-istio-token\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:30:53.183129 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.183105 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12f4570b-e830-4f29-ab27-6be60f4e433b-istio-podinfo\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:30:53.577366 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.577327 2567 generic.go:356] "Generic (PLEG): container finished" podID="12f4570b-e830-4f29-ab27-6be60f4e433b" containerID="8c49365821aef4554f33438664f1dd96bb27155e0d897f10ed24cd7e8275fca6" exitCode=0 Jul 09 15:30:53.577519 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.577380 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" Jul 09 15:30:53.577519 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.577418 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" event={"ID":"12f4570b-e830-4f29-ab27-6be60f4e433b","Type":"ContainerDied","Data":"8c49365821aef4554f33438664f1dd96bb27155e0d897f10ed24cd7e8275fca6"} Jul 09 15:30:53.577519 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.577506 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn" event={"ID":"12f4570b-e830-4f29-ab27-6be60f4e433b","Type":"ContainerDied","Data":"9f6797f9306c4173e19b453bf999c61789b88099e032100fb85555de73a98a27"} Jul 09 15:30:53.577644 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.577530 2567 scope.go:117] "RemoveContainer" containerID="8c49365821aef4554f33438664f1dd96bb27155e0d897f10ed24cd7e8275fca6" Jul 09 15:30:53.587998 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.587973 2567 scope.go:117] "RemoveContainer" containerID="8c49365821aef4554f33438664f1dd96bb27155e0d897f10ed24cd7e8275fca6" Jul 09 15:30:53.588401 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:30:53.588368 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c49365821aef4554f33438664f1dd96bb27155e0d897f10ed24cd7e8275fca6\": container with ID starting with 8c49365821aef4554f33438664f1dd96bb27155e0d897f10ed24cd7e8275fca6 not found: ID does not exist" containerID="8c49365821aef4554f33438664f1dd96bb27155e0d897f10ed24cd7e8275fca6" Jul 09 15:30:53.588509 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.588412 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c49365821aef4554f33438664f1dd96bb27155e0d897f10ed24cd7e8275fca6"} err="failed to get container status \"8c49365821aef4554f33438664f1dd96bb27155e0d897f10ed24cd7e8275fca6\": rpc error: code = NotFound desc = could not find container \"8c49365821aef4554f33438664f1dd96bb27155e0d897f10ed24cd7e8275fca6\": container with ID starting with 8c49365821aef4554f33438664f1dd96bb27155e0d897f10ed24cd7e8275fca6 not found: ID does not exist" Jul 09 15:30:53.604079 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.604043 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn"] Jul 09 15:30:53.608647 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.608616 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-1829e532-openshift-default-bcbb556f7-twnvn"] Jul 09 15:30:53.623807 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.623770 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:30:53.700462 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.700424 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:30:53.832066 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.831979 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12f4570b-e830-4f29-ab27-6be60f4e433b" path="/var/lib/kubelet/pods/12f4570b-e830-4f29-ab27-6be60f4e433b/volumes" Jul 09 15:30:53.845488 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.845458 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk"] Jul 09 15:30:53.845954 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.845940 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12f4570b-e830-4f29-ab27-6be60f4e433b" containerName="istio-proxy" Jul 09 15:30:53.845954 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.845955 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="12f4570b-e830-4f29-ab27-6be60f4e433b" containerName="istio-proxy" Jul 09 15:30:53.846084 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.846016 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="12f4570b-e830-4f29-ab27-6be60f4e433b" containerName="istio-proxy" Jul 09 15:30:53.849881 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.849866 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" Jul 09 15:30:53.853010 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.852986 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-40e15ce5-openshift-default-dockercfg-r8cj7\"" Jul 09 15:30:53.870670 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.870638 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk"] Jul 09 15:30:53.928208 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.928173 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:30:53.991435 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.991394 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2z6k\" (UniqueName: \"kubernetes.io/projected/21790e81-c441-4c03-ad1b-bcb324657a72-kube-api-access-h2z6k\") pod \"e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk\" (UID: \"21790e81-c441-4c03-ad1b-bcb324657a72\") " pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" Jul 09 15:30:53.991435 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.991439 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21790e81-c441-4c03-ad1b-bcb324657a72-istiod-ca-cert\") pod \"e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk\" (UID: \"21790e81-c441-4c03-ad1b-bcb324657a72\") " pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" Jul 09 15:30:53.991706 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.991513 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21790e81-c441-4c03-ad1b-bcb324657a72-istio-token\") pod \"e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk\" (UID: \"21790e81-c441-4c03-ad1b-bcb324657a72\") " pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" Jul 09 15:30:53.991706 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.991574 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21790e81-c441-4c03-ad1b-bcb324657a72-istio-data\") pod \"e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk\" (UID: \"21790e81-c441-4c03-ad1b-bcb324657a72\") " pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" Jul 09 15:30:53.991706 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.991607 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21790e81-c441-4c03-ad1b-bcb324657a72-istio-podinfo\") pod \"e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk\" (UID: \"21790e81-c441-4c03-ad1b-bcb324657a72\") " pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" Jul 09 15:30:53.991706 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.991639 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21790e81-c441-4c03-ad1b-bcb324657a72-credential-socket\") pod \"e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk\" (UID: \"21790e81-c441-4c03-ad1b-bcb324657a72\") " pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" Jul 09 15:30:53.991873 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.991743 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21790e81-c441-4c03-ad1b-bcb324657a72-istio-envoy\") pod \"e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk\" (UID: \"21790e81-c441-4c03-ad1b-bcb324657a72\") " pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" Jul 09 15:30:53.991873 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.991780 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21790e81-c441-4c03-ad1b-bcb324657a72-workload-socket\") pod \"e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk\" (UID: \"21790e81-c441-4c03-ad1b-bcb324657a72\") " pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" Jul 09 15:30:53.991873 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:53.991836 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21790e81-c441-4c03-ad1b-bcb324657a72-workload-certs\") pod \"e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk\" (UID: \"21790e81-c441-4c03-ad1b-bcb324657a72\") " pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" Jul 09 15:30:54.029786 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:54.029747 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:30:54.092556 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:54.092451 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2z6k\" (UniqueName: \"kubernetes.io/projected/21790e81-c441-4c03-ad1b-bcb324657a72-kube-api-access-h2z6k\") pod \"e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk\" (UID: \"21790e81-c441-4c03-ad1b-bcb324657a72\") " pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" Jul 09 15:30:54.092556 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:54.092496 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21790e81-c441-4c03-ad1b-bcb324657a72-istiod-ca-cert\") pod \"e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk\" (UID: \"21790e81-c441-4c03-ad1b-bcb324657a72\") " pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" Jul 09 15:30:54.092556 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:54.092544 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21790e81-c441-4c03-ad1b-bcb324657a72-istio-token\") pod \"e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk\" (UID: \"21790e81-c441-4c03-ad1b-bcb324657a72\") " pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" Jul 09 15:30:54.092868 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:54.092578 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21790e81-c441-4c03-ad1b-bcb324657a72-istio-data\") pod \"e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk\" (UID: \"21790e81-c441-4c03-ad1b-bcb324657a72\") " pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" Jul 09 15:30:54.092868 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:54.092604 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21790e81-c441-4c03-ad1b-bcb324657a72-istio-podinfo\") pod \"e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk\" (UID: \"21790e81-c441-4c03-ad1b-bcb324657a72\") " pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" Jul 09 15:30:54.092868 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:54.092634 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21790e81-c441-4c03-ad1b-bcb324657a72-credential-socket\") pod \"e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk\" (UID: \"21790e81-c441-4c03-ad1b-bcb324657a72\") " pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" Jul 09 15:30:54.092868 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:54.092707 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21790e81-c441-4c03-ad1b-bcb324657a72-istio-envoy\") pod \"e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk\" (UID: \"21790e81-c441-4c03-ad1b-bcb324657a72\") " pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" Jul 09 15:30:54.092868 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:54.092739 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21790e81-c441-4c03-ad1b-bcb324657a72-workload-socket\") pod \"e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk\" (UID: \"21790e81-c441-4c03-ad1b-bcb324657a72\") " pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" Jul 09 15:30:54.092868 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:54.092788 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21790e81-c441-4c03-ad1b-bcb324657a72-workload-certs\") pod \"e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk\" (UID: \"21790e81-c441-4c03-ad1b-bcb324657a72\") " pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" Jul 09 15:30:54.093158 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:54.093016 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21790e81-c441-4c03-ad1b-bcb324657a72-istio-data\") pod \"e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk\" (UID: \"21790e81-c441-4c03-ad1b-bcb324657a72\") " pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" Jul 09 15:30:54.093292 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:54.093271 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21790e81-c441-4c03-ad1b-bcb324657a72-workload-socket\") pod \"e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk\" (UID: \"21790e81-c441-4c03-ad1b-bcb324657a72\") " pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" Jul 09 15:30:54.093363 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:54.093299 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21790e81-c441-4c03-ad1b-bcb324657a72-workload-certs\") pod \"e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk\" (UID: \"21790e81-c441-4c03-ad1b-bcb324657a72\") " pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" Jul 09 15:30:54.093401 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:54.093382 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21790e81-c441-4c03-ad1b-bcb324657a72-istiod-ca-cert\") pod \"e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk\" (UID: \"21790e81-c441-4c03-ad1b-bcb324657a72\") " pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" Jul 09 15:30:54.093445 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:54.093397 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21790e81-c441-4c03-ad1b-bcb324657a72-credential-socket\") pod \"e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk\" (UID: \"21790e81-c441-4c03-ad1b-bcb324657a72\") " pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" Jul 09 15:30:54.095112 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:54.095091 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21790e81-c441-4c03-ad1b-bcb324657a72-istio-envoy\") pod \"e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk\" (UID: \"21790e81-c441-4c03-ad1b-bcb324657a72\") " pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" Jul 09 15:30:54.095426 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:54.095407 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21790e81-c441-4c03-ad1b-bcb324657a72-istio-podinfo\") pod \"e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk\" (UID: \"21790e81-c441-4c03-ad1b-bcb324657a72\") " pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" Jul 09 15:30:54.102102 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:54.102070 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21790e81-c441-4c03-ad1b-bcb324657a72-istio-token\") pod \"e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk\" (UID: \"21790e81-c441-4c03-ad1b-bcb324657a72\") " pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" Jul 09 15:30:54.102389 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:54.102373 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2z6k\" (UniqueName: \"kubernetes.io/projected/21790e81-c441-4c03-ad1b-bcb324657a72-kube-api-access-h2z6k\") pod \"e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk\" (UID: \"21790e81-c441-4c03-ad1b-bcb324657a72\") " pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" Jul 09 15:30:54.162387 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:54.162355 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" Jul 09 15:30:54.318986 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:54.318958 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk"] Jul 09 15:30:54.321311 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:30:54.321274 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21790e81_c441_4c03_ad1b_bcb324657a72.slice/crio-61fd00c8c83e35267a5c39a6bc9b12310255b739cf586c5fa39eca0861fa30db WatchSource:0}: Error finding container 61fd00c8c83e35267a5c39a6bc9b12310255b739cf586c5fa39eca0861fa30db: Status 404 returned error can't find the container with id 61fd00c8c83e35267a5c39a6bc9b12310255b739cf586c5fa39eca0861fa30db Jul 09 15:30:54.323490 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:54.323458 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:30:54.323602 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:54.323531 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:30:54.323602 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:54.323560 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:30:54.585570 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:54.585481 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" event={"ID":"21790e81-c441-4c03-ad1b-bcb324657a72","Type":"ContainerStarted","Data":"b6282a488a6f177bd3d99a6387b89c3f8944c97bc594a0bdcca2af2bc2bb6638"} Jul 09 15:30:54.585570 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:54.585535 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" event={"ID":"21790e81-c441-4c03-ad1b-bcb324657a72","Type":"ContainerStarted","Data":"61fd00c8c83e35267a5c39a6bc9b12310255b739cf586c5fa39eca0861fa30db"} Jul 09 15:30:54.611309 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:54.611191 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" podStartSLOduration=1.611166316 podStartE2EDuration="1.611166316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:30:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:30:54.608729641 +0000 UTC m=+1701.339236161" watchObservedRunningTime="2026-07-09 15:30:54.611166316 +0000 UTC m=+1701.341672812" Jul 09 15:30:55.162993 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:55.162956 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" Jul 09 15:30:55.168291 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:55.168262 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" Jul 09 15:30:55.560169 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:55.560119 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:30:55.592539 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:55.592504 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" Jul 09 15:30:55.593552 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:55.593530 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" Jul 09 15:30:55.904358 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:55.904264 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:30:56.253602 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:56.253561 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:30:56.370876 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:56.370836 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:30:58.038968 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:30:58.038933 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:31:00.257182 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:00.257145 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:31:00.815842 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:00.814876 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:31:06.432440 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.432339 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk"] Jul 09 15:31:06.432905 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.432546 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" podUID="21790e81-c441-4c03-ad1b-bcb324657a72" containerName="istio-proxy" containerID="cri-o://b6282a488a6f177bd3d99a6387b89c3f8944c97bc594a0bdcca2af2bc2bb6638" gracePeriod=30 Jul 09 15:31:06.466768 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.466726 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:31:06.574724 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.574688 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:31:06.587983 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.587950 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" Jul 09 15:31:06.619178 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.619144 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21790e81-c441-4c03-ad1b-bcb324657a72-credential-socket\") pod \"21790e81-c441-4c03-ad1b-bcb324657a72\" (UID: \"21790e81-c441-4c03-ad1b-bcb324657a72\") " Jul 09 15:31:06.619351 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.619244 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21790e81-c441-4c03-ad1b-bcb324657a72-workload-certs\") pod \"21790e81-c441-4c03-ad1b-bcb324657a72\" (UID: \"21790e81-c441-4c03-ad1b-bcb324657a72\") " Jul 09 15:31:06.619351 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.619285 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21790e81-c441-4c03-ad1b-bcb324657a72-istio-token\") pod \"21790e81-c441-4c03-ad1b-bcb324657a72\" (UID: \"21790e81-c441-4c03-ad1b-bcb324657a72\") " Jul 09 15:31:06.619351 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.619314 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2z6k\" (UniqueName: \"kubernetes.io/projected/21790e81-c441-4c03-ad1b-bcb324657a72-kube-api-access-h2z6k\") pod \"21790e81-c441-4c03-ad1b-bcb324657a72\" (UID: \"21790e81-c441-4c03-ad1b-bcb324657a72\") " Jul 09 15:31:06.619351 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.619347 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21790e81-c441-4c03-ad1b-bcb324657a72-istio-data\") pod \"21790e81-c441-4c03-ad1b-bcb324657a72\" (UID: \"21790e81-c441-4c03-ad1b-bcb324657a72\") " Jul 09 15:31:06.619576 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.619372 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21790e81-c441-4c03-ad1b-bcb324657a72-istio-envoy\") pod \"21790e81-c441-4c03-ad1b-bcb324657a72\" (UID: \"21790e81-c441-4c03-ad1b-bcb324657a72\") " Jul 09 15:31:06.619576 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.619433 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21790e81-c441-4c03-ad1b-bcb324657a72-workload-socket\") pod \"21790e81-c441-4c03-ad1b-bcb324657a72\" (UID: \"21790e81-c441-4c03-ad1b-bcb324657a72\") " Jul 09 15:31:06.619576 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.619468 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21790e81-c441-4c03-ad1b-bcb324657a72-istiod-ca-cert\") pod \"21790e81-c441-4c03-ad1b-bcb324657a72\" (UID: \"21790e81-c441-4c03-ad1b-bcb324657a72\") " Jul 09 15:31:06.619576 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.619477 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21790e81-c441-4c03-ad1b-bcb324657a72-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "21790e81-c441-4c03-ad1b-bcb324657a72" (UID: "21790e81-c441-4c03-ad1b-bcb324657a72"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:31:06.619576 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.619504 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21790e81-c441-4c03-ad1b-bcb324657a72-istio-podinfo\") pod \"21790e81-c441-4c03-ad1b-bcb324657a72\" (UID: \"21790e81-c441-4c03-ad1b-bcb324657a72\") " Jul 09 15:31:06.619870 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.619596 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21790e81-c441-4c03-ad1b-bcb324657a72-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "21790e81-c441-4c03-ad1b-bcb324657a72" (UID: "21790e81-c441-4c03-ad1b-bcb324657a72"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:31:06.619870 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.619785 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21790e81-c441-4c03-ad1b-bcb324657a72-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "21790e81-c441-4c03-ad1b-bcb324657a72" (UID: "21790e81-c441-4c03-ad1b-bcb324657a72"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:31:06.619999 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.619901 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21790e81-c441-4c03-ad1b-bcb324657a72-workload-socket\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:31:06.619999 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.619922 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21790e81-c441-4c03-ad1b-bcb324657a72-credential-socket\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:31:06.619999 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.619935 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21790e81-c441-4c03-ad1b-bcb324657a72-workload-certs\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:31:06.620595 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.620534 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21790e81-c441-4c03-ad1b-bcb324657a72-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "21790e81-c441-4c03-ad1b-bcb324657a72" (UID: "21790e81-c441-4c03-ad1b-bcb324657a72"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:31:06.621376 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.621331 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21790e81-c441-4c03-ad1b-bcb324657a72-istio-data" (OuterVolumeSpecName: "istio-data") pod "21790e81-c441-4c03-ad1b-bcb324657a72" (UID: "21790e81-c441-4c03-ad1b-bcb324657a72"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:31:06.623105 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.622992 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/21790e81-c441-4c03-ad1b-bcb324657a72-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "21790e81-c441-4c03-ad1b-bcb324657a72" (UID: "21790e81-c441-4c03-ad1b-bcb324657a72"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 15:31:06.623105 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.623073 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21790e81-c441-4c03-ad1b-bcb324657a72-kube-api-access-h2z6k" (OuterVolumeSpecName: "kube-api-access-h2z6k") pod "21790e81-c441-4c03-ad1b-bcb324657a72" (UID: "21790e81-c441-4c03-ad1b-bcb324657a72"). InnerVolumeSpecName "kube-api-access-h2z6k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:31:06.623412 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.623380 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21790e81-c441-4c03-ad1b-bcb324657a72-istio-token" (OuterVolumeSpecName: "istio-token") pod "21790e81-c441-4c03-ad1b-bcb324657a72" (UID: "21790e81-c441-4c03-ad1b-bcb324657a72"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:31:06.624197 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.623796 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21790e81-c441-4c03-ad1b-bcb324657a72-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "21790e81-c441-4c03-ad1b-bcb324657a72" (UID: "21790e81-c441-4c03-ad1b-bcb324657a72"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:31:06.661249 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.661212 2567 generic.go:356] "Generic (PLEG): container finished" podID="21790e81-c441-4c03-ad1b-bcb324657a72" containerID="b6282a488a6f177bd3d99a6387b89c3f8944c97bc594a0bdcca2af2bc2bb6638" exitCode=0 Jul 09 15:31:06.661432 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.661277 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" Jul 09 15:31:06.661432 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.661292 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" event={"ID":"21790e81-c441-4c03-ad1b-bcb324657a72","Type":"ContainerDied","Data":"b6282a488a6f177bd3d99a6387b89c3f8944c97bc594a0bdcca2af2bc2bb6638"} Jul 09 15:31:06.661432 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.661373 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk" event={"ID":"21790e81-c441-4c03-ad1b-bcb324657a72","Type":"ContainerDied","Data":"61fd00c8c83e35267a5c39a6bc9b12310255b739cf586c5fa39eca0861fa30db"} Jul 09 15:31:06.661432 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.661396 2567 scope.go:117] "RemoveContainer" containerID="b6282a488a6f177bd3d99a6387b89c3f8944c97bc594a0bdcca2af2bc2bb6638" Jul 09 15:31:06.670615 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.670598 2567 scope.go:117] "RemoveContainer" containerID="b6282a488a6f177bd3d99a6387b89c3f8944c97bc594a0bdcca2af2bc2bb6638" Jul 09 15:31:06.670967 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:31:06.670938 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6282a488a6f177bd3d99a6387b89c3f8944c97bc594a0bdcca2af2bc2bb6638\": container with ID starting with b6282a488a6f177bd3d99a6387b89c3f8944c97bc594a0bdcca2af2bc2bb6638 not found: ID does not exist" containerID="b6282a488a6f177bd3d99a6387b89c3f8944c97bc594a0bdcca2af2bc2bb6638" Jul 09 15:31:06.671054 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.670979 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6282a488a6f177bd3d99a6387b89c3f8944c97bc594a0bdcca2af2bc2bb6638"} err="failed to get container status \"b6282a488a6f177bd3d99a6387b89c3f8944c97bc594a0bdcca2af2bc2bb6638\": rpc error: code = NotFound desc = could not find container \"b6282a488a6f177bd3d99a6387b89c3f8944c97bc594a0bdcca2af2bc2bb6638\": container with ID starting with b6282a488a6f177bd3d99a6387b89c3f8944c97bc594a0bdcca2af2bc2bb6638 not found: ID does not exist" Jul 09 15:31:06.691916 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.691808 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk"] Jul 09 15:31:06.697352 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.697324 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-40e15ce5-openshift-default-6fb7d8d7f5-mvwnk"] Jul 09 15:31:06.721048 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.721005 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21790e81-c441-4c03-ad1b-bcb324657a72-istio-token\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:31:06.721048 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.721040 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2z6k\" (UniqueName: \"kubernetes.io/projected/21790e81-c441-4c03-ad1b-bcb324657a72-kube-api-access-h2z6k\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:31:06.721048 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.721055 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21790e81-c441-4c03-ad1b-bcb324657a72-istio-data\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:31:06.721412 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.721068 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21790e81-c441-4c03-ad1b-bcb324657a72-istio-envoy\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:31:06.721412 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.721079 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21790e81-c441-4c03-ad1b-bcb324657a72-istiod-ca-cert\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:31:06.721412 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:06.721090 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21790e81-c441-4c03-ad1b-bcb324657a72-istio-podinfo\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:31:07.829311 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:07.829270 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21790e81-c441-4c03-ad1b-bcb324657a72" path="/var/lib/kubelet/pods/21790e81-c441-4c03-ad1b-bcb324657a72/volumes" Jul 09 15:31:08.918121 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:08.918083 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:31:13.457297 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.457257 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:31:13.550938 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.550900 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:31:13.637109 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.637071 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt"] Jul 09 15:31:13.637981 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.637949 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21790e81-c441-4c03-ad1b-bcb324657a72" containerName="istio-proxy" Jul 09 15:31:13.637981 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.637984 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="21790e81-c441-4c03-ad1b-bcb324657a72" containerName="istio-proxy" Jul 09 15:31:13.638206 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.638170 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="21790e81-c441-4c03-ad1b-bcb324657a72" containerName="istio-proxy" Jul 09 15:31:13.641656 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.641630 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" Jul 09 15:31:13.644316 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.644295 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-14931b17-openshift-default-dockercfg-sm5pl\"" Jul 09 15:31:13.657513 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.657473 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt"] Jul 09 15:31:13.676277 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.676242 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:31:13.693714 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.693678 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4a36ac3-26ed-42cf-9175-a68ac9d24315-istio-data\") pod \"e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt\" (UID: \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\") " pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" Jul 09 15:31:13.693949 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.693744 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4a36ac3-26ed-42cf-9175-a68ac9d24315-istio-token\") pod \"e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt\" (UID: \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\") " pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" Jul 09 15:31:13.693949 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.693783 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4a36ac3-26ed-42cf-9175-a68ac9d24315-istiod-ca-cert\") pod \"e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt\" (UID: \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\") " pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" Jul 09 15:31:13.693949 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.693844 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p82px\" (UniqueName: \"kubernetes.io/projected/f4a36ac3-26ed-42cf-9175-a68ac9d24315-kube-api-access-p82px\") pod \"e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt\" (UID: \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\") " pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" Jul 09 15:31:13.693949 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.693883 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4a36ac3-26ed-42cf-9175-a68ac9d24315-workload-socket\") pod \"e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt\" (UID: \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\") " pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" Jul 09 15:31:13.694194 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.694038 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4a36ac3-26ed-42cf-9175-a68ac9d24315-workload-certs\") pod \"e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt\" (UID: \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\") " pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" Jul 09 15:31:13.694194 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.694082 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4a36ac3-26ed-42cf-9175-a68ac9d24315-credential-socket\") pod \"e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt\" (UID: \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\") " pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" Jul 09 15:31:13.694194 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.694112 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4a36ac3-26ed-42cf-9175-a68ac9d24315-istio-envoy\") pod \"e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt\" (UID: \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\") " pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" Jul 09 15:31:13.694194 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.694143 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4a36ac3-26ed-42cf-9175-a68ac9d24315-istio-podinfo\") pod \"e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt\" (UID: \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\") " pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" Jul 09 15:31:13.774363 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.774273 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:31:13.794962 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.794917 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4a36ac3-26ed-42cf-9175-a68ac9d24315-istio-data\") pod \"e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt\" (UID: \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\") " pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" Jul 09 15:31:13.795166 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.794976 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4a36ac3-26ed-42cf-9175-a68ac9d24315-istio-token\") pod \"e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt\" (UID: \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\") " pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" Jul 09 15:31:13.795166 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.794999 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4a36ac3-26ed-42cf-9175-a68ac9d24315-istiod-ca-cert\") pod \"e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt\" (UID: \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\") " pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" Jul 09 15:31:13.795166 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.795034 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p82px\" (UniqueName: \"kubernetes.io/projected/f4a36ac3-26ed-42cf-9175-a68ac9d24315-kube-api-access-p82px\") pod \"e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt\" (UID: \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\") " pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" Jul 09 15:31:13.795166 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.795063 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4a36ac3-26ed-42cf-9175-a68ac9d24315-workload-socket\") pod \"e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt\" (UID: \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\") " pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" Jul 09 15:31:13.795382 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.795283 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4a36ac3-26ed-42cf-9175-a68ac9d24315-workload-certs\") pod \"e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt\" (UID: \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\") " pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" Jul 09 15:31:13.795382 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.795330 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4a36ac3-26ed-42cf-9175-a68ac9d24315-credential-socket\") pod \"e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt\" (UID: \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\") " pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" Jul 09 15:31:13.795382 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.795357 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4a36ac3-26ed-42cf-9175-a68ac9d24315-istio-envoy\") pod \"e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt\" (UID: \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\") " pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" Jul 09 15:31:13.795540 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.795388 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4a36ac3-26ed-42cf-9175-a68ac9d24315-istio-podinfo\") pod \"e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt\" (UID: \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\") " pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" Jul 09 15:31:13.795540 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.795400 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4a36ac3-26ed-42cf-9175-a68ac9d24315-istio-data\") pod \"e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt\" (UID: \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\") " pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" Jul 09 15:31:13.795540 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.795521 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4a36ac3-26ed-42cf-9175-a68ac9d24315-workload-socket\") pod \"e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt\" (UID: \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\") " pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" Jul 09 15:31:13.795757 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.795737 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4a36ac3-26ed-42cf-9175-a68ac9d24315-credential-socket\") pod \"e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt\" (UID: \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\") " pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" Jul 09 15:31:13.795870 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.795795 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4a36ac3-26ed-42cf-9175-a68ac9d24315-workload-certs\") pod \"e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt\" (UID: \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\") " pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" Jul 09 15:31:13.795948 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.795869 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4a36ac3-26ed-42cf-9175-a68ac9d24315-istiod-ca-cert\") pod \"e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt\" (UID: \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\") " pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" Jul 09 15:31:13.797945 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.797920 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4a36ac3-26ed-42cf-9175-a68ac9d24315-istio-envoy\") pod \"e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt\" (UID: \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\") " pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" Jul 09 15:31:13.798079 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.798020 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4a36ac3-26ed-42cf-9175-a68ac9d24315-istio-podinfo\") pod \"e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt\" (UID: \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\") " pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" Jul 09 15:31:13.805803 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.805767 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p82px\" (UniqueName: \"kubernetes.io/projected/f4a36ac3-26ed-42cf-9175-a68ac9d24315-kube-api-access-p82px\") pod \"e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt\" (UID: \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\") " pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" Jul 09 15:31:13.808966 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.808947 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4a36ac3-26ed-42cf-9175-a68ac9d24315-istio-token\") pod \"e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt\" (UID: \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\") " pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" Jul 09 15:31:13.952797 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:13.952763 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" Jul 09 15:31:14.104655 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:14.104623 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt"] Jul 09 15:31:14.105607 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:31:14.105583 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4a36ac3_26ed_42cf_9175_a68ac9d24315.slice/crio-ce5326196a973663c41b929bdaa2280b282cd2062b4394f5456a66b24ec3bef2 WatchSource:0}: Error finding container ce5326196a973663c41b929bdaa2280b282cd2062b4394f5456a66b24ec3bef2: Status 404 returned error can't find the container with id ce5326196a973663c41b929bdaa2280b282cd2062b4394f5456a66b24ec3bef2 Jul 09 15:31:14.107981 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:14.107941 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:31:14.108084 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:14.108012 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:31:14.108084 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:14.108042 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:31:14.705835 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:14.705772 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" event={"ID":"f4a36ac3-26ed-42cf-9175-a68ac9d24315","Type":"ContainerStarted","Data":"9aafb4e04adeebb60c33a3522c7593e980915a3fb215ef0b941dafda5737ba1f"} Jul 09 15:31:14.706294 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:14.705854 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" event={"ID":"f4a36ac3-26ed-42cf-9175-a68ac9d24315","Type":"ContainerStarted","Data":"ce5326196a973663c41b929bdaa2280b282cd2062b4394f5456a66b24ec3bef2"} Jul 09 15:31:14.752086 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:14.752036 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" podStartSLOduration=1.752017662 podStartE2EDuration="1.752017662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:31:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:31:14.75195478 +0000 UTC m=+1721.482461270" watchObservedRunningTime="2026-07-09 15:31:14.752017662 +0000 UTC m=+1721.482524152" Jul 09 15:31:14.924223 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:14.924189 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:31:14.953035 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:14.952996 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" Jul 09 15:31:15.057087 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:15.056996 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:31:15.167658 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:15.167628 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:31:15.316451 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:15.316348 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:31:15.627157 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:15.627042 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:31:15.724606 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:15.724575 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:31:15.958251 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:15.958223 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" Jul 09 15:31:16.717230 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:16.717189 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" Jul 09 15:31:16.718331 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:16.718309 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" Jul 09 15:31:16.802713 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:16.802660 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:31:19.301667 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:19.301629 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:31:19.444226 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:19.444189 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt"] Jul 09 15:31:19.507586 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:19.507547 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:31:19.669846 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:19.669718 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:31:19.732555 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:19.732517 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" podUID="f4a36ac3-26ed-42cf-9175-a68ac9d24315" containerName="istio-proxy" containerID="cri-o://9aafb4e04adeebb60c33a3522c7593e980915a3fb215ef0b941dafda5737ba1f" gracePeriod=30 Jul 09 15:31:20.025112 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:20.025076 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:31:22.853420 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:22.853385 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:31:26.368016 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.367973 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:31:26.488078 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.488053 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" Jul 09 15:31:26.536054 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.535936 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:31:26.633949 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.633903 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4a36ac3-26ed-42cf-9175-a68ac9d24315-istiod-ca-cert\") pod \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\" (UID: \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\") " Jul 09 15:31:26.634165 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.633967 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4a36ac3-26ed-42cf-9175-a68ac9d24315-workload-certs\") pod \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\" (UID: \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\") " Jul 09 15:31:26.634165 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.633996 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4a36ac3-26ed-42cf-9175-a68ac9d24315-istio-token\") pod \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\" (UID: \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\") " Jul 09 15:31:26.634165 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.634027 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4a36ac3-26ed-42cf-9175-a68ac9d24315-credential-socket\") pod \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\" (UID: \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\") " Jul 09 15:31:26.634165 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.634056 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4a36ac3-26ed-42cf-9175-a68ac9d24315-istio-data\") pod \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\" (UID: \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\") " Jul 09 15:31:26.634165 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.634076 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p82px\" (UniqueName: \"kubernetes.io/projected/f4a36ac3-26ed-42cf-9175-a68ac9d24315-kube-api-access-p82px\") pod \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\" (UID: \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\") " Jul 09 15:31:26.634165 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.634100 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4a36ac3-26ed-42cf-9175-a68ac9d24315-workload-socket\") pod \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\" (UID: \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\") " Jul 09 15:31:26.634481 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.634176 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4a36ac3-26ed-42cf-9175-a68ac9d24315-istio-envoy\") pod \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\" (UID: \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\") " Jul 09 15:31:26.634481 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.634245 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4a36ac3-26ed-42cf-9175-a68ac9d24315-istio-podinfo\") pod \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\" (UID: \"f4a36ac3-26ed-42cf-9175-a68ac9d24315\") " Jul 09 15:31:26.634481 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.634357 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4a36ac3-26ed-42cf-9175-a68ac9d24315-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f4a36ac3-26ed-42cf-9175-a68ac9d24315" (UID: "f4a36ac3-26ed-42cf-9175-a68ac9d24315"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:31:26.634637 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.634474 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4a36ac3-26ed-42cf-9175-a68ac9d24315-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f4a36ac3-26ed-42cf-9175-a68ac9d24315" (UID: "f4a36ac3-26ed-42cf-9175-a68ac9d24315"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:31:26.634637 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.634402 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4a36ac3-26ed-42cf-9175-a68ac9d24315-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f4a36ac3-26ed-42cf-9175-a68ac9d24315" (UID: "f4a36ac3-26ed-42cf-9175-a68ac9d24315"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:31:26.634637 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.634538 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4a36ac3-26ed-42cf-9175-a68ac9d24315-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f4a36ac3-26ed-42cf-9175-a68ac9d24315" (UID: "f4a36ac3-26ed-42cf-9175-a68ac9d24315"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:31:26.634792 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.634734 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4a36ac3-26ed-42cf-9175-a68ac9d24315-workload-certs\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:31:26.634792 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.634752 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4a36ac3-26ed-42cf-9175-a68ac9d24315-credential-socket\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:31:26.634792 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.634767 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4a36ac3-26ed-42cf-9175-a68ac9d24315-workload-socket\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:31:26.634792 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.634780 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4a36ac3-26ed-42cf-9175-a68ac9d24315-istiod-ca-cert\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:31:26.635130 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.635107 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4a36ac3-26ed-42cf-9175-a68ac9d24315-istio-data" (OuterVolumeSpecName: "istio-data") pod "f4a36ac3-26ed-42cf-9175-a68ac9d24315" (UID: "f4a36ac3-26ed-42cf-9175-a68ac9d24315"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:31:26.636782 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.636749 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4a36ac3-26ed-42cf-9175-a68ac9d24315-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f4a36ac3-26ed-42cf-9175-a68ac9d24315" (UID: "f4a36ac3-26ed-42cf-9175-a68ac9d24315"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:31:26.636782 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.636772 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4a36ac3-26ed-42cf-9175-a68ac9d24315-kube-api-access-p82px" (OuterVolumeSpecName: "kube-api-access-p82px") pod "f4a36ac3-26ed-42cf-9175-a68ac9d24315" (UID: "f4a36ac3-26ed-42cf-9175-a68ac9d24315"). InnerVolumeSpecName "kube-api-access-p82px". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:31:26.637040 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.636842 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4a36ac3-26ed-42cf-9175-a68ac9d24315-istio-token" (OuterVolumeSpecName: "istio-token") pod "f4a36ac3-26ed-42cf-9175-a68ac9d24315" (UID: "f4a36ac3-26ed-42cf-9175-a68ac9d24315"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:31:26.637040 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.636871 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f4a36ac3-26ed-42cf-9175-a68ac9d24315-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f4a36ac3-26ed-42cf-9175-a68ac9d24315" (UID: "f4a36ac3-26ed-42cf-9175-a68ac9d24315"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 15:31:26.735992 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.735953 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4a36ac3-26ed-42cf-9175-a68ac9d24315-istio-token\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:31:26.735992 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.735984 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4a36ac3-26ed-42cf-9175-a68ac9d24315-istio-data\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:31:26.735992 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.735993 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p82px\" (UniqueName: \"kubernetes.io/projected/f4a36ac3-26ed-42cf-9175-a68ac9d24315-kube-api-access-p82px\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:31:26.735992 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.736003 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4a36ac3-26ed-42cf-9175-a68ac9d24315-istio-envoy\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:31:26.736265 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.736012 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4a36ac3-26ed-42cf-9175-a68ac9d24315-istio-podinfo\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:31:26.770970 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.770930 2567 generic.go:356] "Generic (PLEG): container finished" podID="f4a36ac3-26ed-42cf-9175-a68ac9d24315" containerID="9aafb4e04adeebb60c33a3522c7593e980915a3fb215ef0b941dafda5737ba1f" exitCode=0 Jul 09 15:31:26.771121 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.770987 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" Jul 09 15:31:26.771121 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.771011 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" event={"ID":"f4a36ac3-26ed-42cf-9175-a68ac9d24315","Type":"ContainerDied","Data":"9aafb4e04adeebb60c33a3522c7593e980915a3fb215ef0b941dafda5737ba1f"} Jul 09 15:31:26.771121 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.771068 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt" event={"ID":"f4a36ac3-26ed-42cf-9175-a68ac9d24315","Type":"ContainerDied","Data":"ce5326196a973663c41b929bdaa2280b282cd2062b4394f5456a66b24ec3bef2"} Jul 09 15:31:26.771121 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.771090 2567 scope.go:117] "RemoveContainer" containerID="9aafb4e04adeebb60c33a3522c7593e980915a3fb215ef0b941dafda5737ba1f" Jul 09 15:31:26.781176 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.781158 2567 scope.go:117] "RemoveContainer" containerID="9aafb4e04adeebb60c33a3522c7593e980915a3fb215ef0b941dafda5737ba1f" Jul 09 15:31:26.781466 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:31:26.781444 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9aafb4e04adeebb60c33a3522c7593e980915a3fb215ef0b941dafda5737ba1f\": container with ID starting with 9aafb4e04adeebb60c33a3522c7593e980915a3fb215ef0b941dafda5737ba1f not found: ID does not exist" containerID="9aafb4e04adeebb60c33a3522c7593e980915a3fb215ef0b941dafda5737ba1f" Jul 09 15:31:26.781513 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.781478 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9aafb4e04adeebb60c33a3522c7593e980915a3fb215ef0b941dafda5737ba1f"} err="failed to get container status \"9aafb4e04adeebb60c33a3522c7593e980915a3fb215ef0b941dafda5737ba1f\": rpc error: code = NotFound desc = could not find container \"9aafb4e04adeebb60c33a3522c7593e980915a3fb215ef0b941dafda5737ba1f\": container with ID starting with 9aafb4e04adeebb60c33a3522c7593e980915a3fb215ef0b941dafda5737ba1f not found: ID does not exist" Jul 09 15:31:26.815564 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.815473 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt"] Jul 09 15:31:26.825844 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:26.825801 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-14931b17-openshift-default-7476cc9c9-78vwt"] Jul 09 15:31:27.076686 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:27.076584 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:31:27.243715 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:27.243663 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:31:27.829570 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:27.829531 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4a36ac3-26ed-42cf-9175-a68ac9d24315" path="/var/lib/kubelet/pods/f4a36ac3-26ed-42cf-9175-a68ac9d24315/volumes" Jul 09 15:31:28.835546 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:28.835498 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:31:35.594675 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:35.594635 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:31:35.703882 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:35.703842 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:31:35.811105 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:35.811068 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:31:35.918852 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:35.918731 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:31:36.384939 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:31:36.384890 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:32:02.532869 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:32:02.532805 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:32:09.334330 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:32:09.334291 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:32:09.584799 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:32:09.584680 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:32:09.757166 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:32:09.757129 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:32:25.718462 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:32:25.718360 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:32:32.481459 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:32:32.481415 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:32:32.604724 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:32:32.604686 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:32:32.721189 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:32:32.721156 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:32:33.835396 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:32:33.835363 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hml7m_84a10578-4913-456b-9fb7-e6440914b677/ovn-acl-logging/0.log" Jul 09 15:32:33.844160 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:32:33.844138 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hml7m_84a10578-4913-456b-9fb7-e6440914b677/ovn-acl-logging/0.log" Jul 09 15:32:43.351907 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:32:43.351861 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:32:44.256145 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:32:44.256086 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:32:50.787279 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:32:50.787237 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:32:50.926017 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:32:50.925981 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:32:51.026784 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:32:51.026748 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:32:51.980487 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:32:51.980444 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:32:52.332689 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:32:52.332595 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:32:53.952375 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:32:53.952343 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:33:00.310897 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:00.310860 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:33:00.403889 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:00.403853 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:33:01.073587 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:01.073552 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:33:07.063893 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:07.063847 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:33:07.705372 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:07.705331 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc"] Jul 09 15:33:07.705883 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:07.705864 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4a36ac3-26ed-42cf-9175-a68ac9d24315" containerName="istio-proxy" Jul 09 15:33:07.705991 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:07.705885 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4a36ac3-26ed-42cf-9175-a68ac9d24315" containerName="istio-proxy" Jul 09 15:33:07.706054 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:07.706030 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4a36ac3-26ed-42cf-9175-a68ac9d24315" containerName="istio-proxy" Jul 09 15:33:07.708175 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:07.708156 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:07.712948 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:07.712926 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-cd893609-openshift-default-dockercfg-tzdph\"" Jul 09 15:33:07.757410 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:07.757372 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc"] Jul 09 15:33:07.776522 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:07.776488 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-istiod-ca-cert\") pod \"e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc\" (UID: \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\") " pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:07.776712 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:07.776540 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-istio-token\") pod \"e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc\" (UID: \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\") " pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:07.776712 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:07.776617 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-istio-envoy\") pod \"e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc\" (UID: \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\") " pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:07.776712 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:07.776688 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-workload-socket\") pod \"e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc\" (UID: \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\") " pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:07.776909 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:07.776722 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9dsm\" (UniqueName: \"kubernetes.io/projected/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-kube-api-access-t9dsm\") pod \"e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc\" (UID: \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\") " pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:07.776909 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:07.776774 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-istio-data\") pod \"e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc\" (UID: \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\") " pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:07.777024 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:07.776915 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-workload-certs\") pod \"e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc\" (UID: \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\") " pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:07.777024 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:07.776958 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-istio-podinfo\") pod \"e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc\" (UID: \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\") " pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:07.777024 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:07.777007 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-credential-socket\") pod \"e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc\" (UID: \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\") " pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:07.878503 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:07.878461 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-workload-socket\") pod \"e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc\" (UID: \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\") " pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:07.878503 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:07.878507 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9dsm\" (UniqueName: \"kubernetes.io/projected/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-kube-api-access-t9dsm\") pod \"e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc\" (UID: \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\") " pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:07.878723 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:07.878550 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-istio-data\") pod \"e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc\" (UID: \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\") " pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:07.878723 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:07.878598 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-workload-certs\") pod \"e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc\" (UID: \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\") " pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:07.878798 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:07.878719 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-istio-podinfo\") pod \"e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc\" (UID: \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\") " pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:07.878798 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:07.878787 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-credential-socket\") pod \"e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc\" (UID: \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\") " pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:07.878985 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:07.878857 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-istiod-ca-cert\") pod \"e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc\" (UID: \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\") " pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:07.878985 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:07.878919 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-istio-token\") pod \"e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc\" (UID: \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\") " pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:07.878985 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:07.878927 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-workload-socket\") pod \"e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc\" (UID: \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\") " pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:07.878985 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:07.878946 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-istio-envoy\") pod \"e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc\" (UID: \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\") " pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:07.879185 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:07.879066 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-istio-data\") pod \"e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc\" (UID: \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\") " pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:07.879185 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:07.879170 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-credential-socket\") pod \"e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc\" (UID: \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\") " pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:07.879328 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:07.879309 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-workload-certs\") pod \"e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc\" (UID: \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\") " pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:07.879511 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:07.879492 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-istiod-ca-cert\") pod \"e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc\" (UID: \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\") " pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:07.881091 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:07.881071 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-istio-envoy\") pod \"e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc\" (UID: \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\") " pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:07.881339 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:07.881319 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-istio-podinfo\") pod \"e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc\" (UID: \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\") " pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:07.887576 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:07.887552 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-istio-token\") pod \"e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc\" (UID: \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\") " pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:07.887783 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:07.887760 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9dsm\" (UniqueName: \"kubernetes.io/projected/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-kube-api-access-t9dsm\") pod \"e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc\" (UID: \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\") " pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:08.019405 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:08.019304 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:08.222031 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:08.222006 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc"] Jul 09 15:33:08.223794 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:33:08.223764 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3175f218_ebab_4bdb_9c7a_3f5e90f9814c.slice/crio-dbb0f7096b4d3e51be0a72f9bf620412cd060723bfaf4f4b67cd50f19b83dfc7 WatchSource:0}: Error finding container dbb0f7096b4d3e51be0a72f9bf620412cd060723bfaf4f4b67cd50f19b83dfc7: Status 404 returned error can't find the container with id dbb0f7096b4d3e51be0a72f9bf620412cd060723bfaf4f4b67cd50f19b83dfc7 Jul 09 15:33:08.230876 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:08.226889 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:33:08.230876 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:08.227055 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:33:08.230876 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:08.227143 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:33:08.309078 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:08.309045 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" event={"ID":"3175f218-ebab-4bdb-9c7a-3f5e90f9814c","Type":"ContainerStarted","Data":"628b53963c8aeabd5ff8bd12b034a1274c3b476e7643d2eff73062bcdc3545bb"} Jul 09 15:33:08.309256 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:08.309125 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" event={"ID":"3175f218-ebab-4bdb-9c7a-3f5e90f9814c","Type":"ContainerStarted","Data":"dbb0f7096b4d3e51be0a72f9bf620412cd060723bfaf4f4b67cd50f19b83dfc7"} Jul 09 15:33:08.360259 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:08.360201 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" podStartSLOduration=1.360184883 podStartE2EDuration="1.360184883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:33:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:33:08.357648051 +0000 UTC m=+1835.088154555" watchObservedRunningTime="2026-07-09 15:33:08.360184883 +0000 UTC m=+1835.090691373" Jul 09 15:33:09.020287 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:09.020247 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:09.026033 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:09.026000 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:09.315320 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:09.315229 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:09.316109 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:09.316090 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:10.196919 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:10.196887 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:33:10.297428 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:10.297393 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:33:10.928620 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:10.928585 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:33:11.084148 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:11.084107 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:33:11.487792 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:11.487744 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:33:15.016001 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:15.015958 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:33:15.117000 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:15.116958 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:33:15.582687 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:15.582655 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:33:16.203558 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:16.203523 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:33:16.329568 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:16.329527 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:33:16.793497 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:16.793457 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:33:24.321268 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:24.321227 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:33:24.425136 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:24.425089 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:33:24.811092 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:24.811044 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:33:24.936932 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:24.936472 2567 patch_prober.go:28] interesting pod/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": read tcp 10.134.0.2:40544->10.134.0.48:15021: read: connection reset by peer" start-of-body= Jul 09 15:33:24.936932 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:24.936545 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" podUID="3175f218-ebab-4bdb-9c7a-3f5e90f9814c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": read tcp 10.134.0.2:40544->10.134.0.48:15021: read: connection reset by peer" Jul 09 15:33:25.205209 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:25.205171 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:33:25.403308 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:25.403270 2567 generic.go:356] "Generic (PLEG): container finished" podID="3175f218-ebab-4bdb-9c7a-3f5e90f9814c" containerID="628b53963c8aeabd5ff8bd12b034a1274c3b476e7643d2eff73062bcdc3545bb" exitCode=0 Jul 09 15:33:25.403726 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:25.403350 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" event={"ID":"3175f218-ebab-4bdb-9c7a-3f5e90f9814c","Type":"ContainerDied","Data":"628b53963c8aeabd5ff8bd12b034a1274c3b476e7643d2eff73062bcdc3545bb"} Jul 09 15:33:25.403851 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:25.403806 2567 scope.go:117] "RemoveContainer" containerID="628b53963c8aeabd5ff8bd12b034a1274c3b476e7643d2eff73062bcdc3545bb" Jul 09 15:33:25.405078 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:25.405046 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:33:25.405168 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:25.405117 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:33:25.405168 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:25.405156 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:33:26.411414 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:26.411378 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" event={"ID":"3175f218-ebab-4bdb-9c7a-3f5e90f9814c","Type":"ContainerStarted","Data":"3f490f4748d638c664bb35dfe2cc4e66ee6198095b510a6a6a20abe19e133c7a"} Jul 09 15:33:27.020282 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:27.020238 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:27.020512 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:27.020465 2567 patch_prober.go:28] interesting pod/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 09 15:33:27.020586 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:27.020516 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" podUID="3175f218-ebab-4bdb-9c7a-3f5e90f9814c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 09 15:33:28.020417 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:28.020376 2567 patch_prober.go:28] interesting pod/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 09 15:33:28.021080 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:28.021044 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" podUID="3175f218-ebab-4bdb-9c7a-3f5e90f9814c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 09 15:33:28.110801 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:28.110763 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:33:29.020050 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:29.020013 2567 patch_prober.go:28] interesting pod/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 09 15:33:29.020232 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:29.020084 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" podUID="3175f218-ebab-4bdb-9c7a-3f5e90f9814c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 09 15:33:30.019746 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:30.019695 2567 patch_prober.go:28] interesting pod/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 09 15:33:30.020291 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:30.019775 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" podUID="3175f218-ebab-4bdb-9c7a-3f5e90f9814c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 09 15:33:31.024461 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:31.024428 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:31.024943 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:31.024721 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:31.025418 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:31.025401 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:32.882787 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:32.882754 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:33:33.040500 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:33.040457 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:33:33.115704 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:33.115661 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc"] Jul 09 15:33:33.182203 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:33.182117 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:33:33.451298 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:33.451183 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" podUID="3175f218-ebab-4bdb-9c7a-3f5e90f9814c" containerName="istio-proxy" containerID="cri-o://3f490f4748d638c664bb35dfe2cc4e66ee6198095b510a6a6a20abe19e133c7a" gracePeriod=30 Jul 09 15:33:33.572969 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:33.572933 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:33:33.712928 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:33.712889 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:33:33.866970 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:33.866932 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:33:38.718215 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:38.718151 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:38.811638 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:38.811598 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-istiod-ca-cert\") pod \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\" (UID: \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\") " Jul 09 15:33:38.811843 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:38.811653 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9dsm\" (UniqueName: \"kubernetes.io/projected/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-kube-api-access-t9dsm\") pod \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\" (UID: \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\") " Jul 09 15:33:38.811843 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:38.811690 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-credential-socket\") pod \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\" (UID: \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\") " Jul 09 15:33:38.811843 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:38.811738 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-workload-socket\") pod \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\" (UID: \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\") " Jul 09 15:33:38.811843 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:38.811787 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-istio-envoy\") pod \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\" (UID: \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\") " Jul 09 15:33:38.812077 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:38.811846 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-istio-token\") pod \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\" (UID: \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\") " Jul 09 15:33:38.812077 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:38.811901 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-istio-data\") pod \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\" (UID: \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\") " Jul 09 15:33:38.812077 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:38.811937 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-workload-certs\") pod \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\" (UID: \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\") " Jul 09 15:33:38.812077 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:38.811968 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-istio-podinfo\") pod \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\" (UID: \"3175f218-ebab-4bdb-9c7a-3f5e90f9814c\") " Jul 09 15:33:38.812077 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:38.812041 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3175f218-ebab-4bdb-9c7a-3f5e90f9814c" (UID: "3175f218-ebab-4bdb-9c7a-3f5e90f9814c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:33:38.812314 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:38.812118 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3175f218-ebab-4bdb-9c7a-3f5e90f9814c" (UID: "3175f218-ebab-4bdb-9c7a-3f5e90f9814c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:33:38.812438 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:38.812420 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-istiod-ca-cert\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:33:38.812497 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:38.812443 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-credential-socket\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:33:38.812761 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:38.812672 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3175f218-ebab-4bdb-9c7a-3f5e90f9814c" (UID: "3175f218-ebab-4bdb-9c7a-3f5e90f9814c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:33:38.813015 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:38.812978 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3175f218-ebab-4bdb-9c7a-3f5e90f9814c" (UID: "3175f218-ebab-4bdb-9c7a-3f5e90f9814c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:33:38.813186 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:38.813059 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-istio-data" (OuterVolumeSpecName: "istio-data") pod "3175f218-ebab-4bdb-9c7a-3f5e90f9814c" (UID: "3175f218-ebab-4bdb-9c7a-3f5e90f9814c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:33:38.814360 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:38.814242 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-istio-token" (OuterVolumeSpecName: "istio-token") pod "3175f218-ebab-4bdb-9c7a-3f5e90f9814c" (UID: "3175f218-ebab-4bdb-9c7a-3f5e90f9814c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:33:38.814360 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:38.814321 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3175f218-ebab-4bdb-9c7a-3f5e90f9814c" (UID: "3175f218-ebab-4bdb-9c7a-3f5e90f9814c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:33:38.814527 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:38.814357 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3175f218-ebab-4bdb-9c7a-3f5e90f9814c" (UID: "3175f218-ebab-4bdb-9c7a-3f5e90f9814c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 15:33:38.814639 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:38.814618 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-kube-api-access-t9dsm" (OuterVolumeSpecName: "kube-api-access-t9dsm") pod "3175f218-ebab-4bdb-9c7a-3f5e90f9814c" (UID: "3175f218-ebab-4bdb-9c7a-3f5e90f9814c"). InnerVolumeSpecName "kube-api-access-t9dsm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:33:38.913626 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:38.913595 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-workload-certs\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:33:38.913626 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:38.913621 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-istio-podinfo\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:33:38.913626 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:38.913630 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9dsm\" (UniqueName: \"kubernetes.io/projected/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-kube-api-access-t9dsm\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:33:38.913927 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:38.913644 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-workload-socket\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:33:38.913927 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:38.913657 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-istio-envoy\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:33:38.913927 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:38.913666 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-istio-token\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:33:38.913927 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:38.913673 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3175f218-ebab-4bdb-9c7a-3f5e90f9814c-istio-data\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:33:39.484013 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:39.483982 2567 generic.go:356] "Generic (PLEG): container finished" podID="3175f218-ebab-4bdb-9c7a-3f5e90f9814c" containerID="3f490f4748d638c664bb35dfe2cc4e66ee6198095b510a6a6a20abe19e133c7a" exitCode=0 Jul 09 15:33:39.484200 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:39.484035 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" Jul 09 15:33:39.484200 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:39.484058 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" event={"ID":"3175f218-ebab-4bdb-9c7a-3f5e90f9814c","Type":"ContainerDied","Data":"3f490f4748d638c664bb35dfe2cc4e66ee6198095b510a6a6a20abe19e133c7a"} Jul 09 15:33:39.484200 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:39.484103 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc" event={"ID":"3175f218-ebab-4bdb-9c7a-3f5e90f9814c","Type":"ContainerDied","Data":"dbb0f7096b4d3e51be0a72f9bf620412cd060723bfaf4f4b67cd50f19b83dfc7"} Jul 09 15:33:39.484200 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:39.484125 2567 scope.go:117] "RemoveContainer" containerID="3f490f4748d638c664bb35dfe2cc4e66ee6198095b510a6a6a20abe19e133c7a" Jul 09 15:33:39.493557 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:39.493535 2567 scope.go:117] "RemoveContainer" containerID="628b53963c8aeabd5ff8bd12b034a1274c3b476e7643d2eff73062bcdc3545bb" Jul 09 15:33:39.504007 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:39.503983 2567 scope.go:117] "RemoveContainer" containerID="3f490f4748d638c664bb35dfe2cc4e66ee6198095b510a6a6a20abe19e133c7a" Jul 09 15:33:39.504550 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:33:39.504524 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f490f4748d638c664bb35dfe2cc4e66ee6198095b510a6a6a20abe19e133c7a\": container with ID starting with 3f490f4748d638c664bb35dfe2cc4e66ee6198095b510a6a6a20abe19e133c7a not found: ID does not exist" containerID="3f490f4748d638c664bb35dfe2cc4e66ee6198095b510a6a6a20abe19e133c7a" Jul 09 15:33:39.504611 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:39.504561 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f490f4748d638c664bb35dfe2cc4e66ee6198095b510a6a6a20abe19e133c7a"} err="failed to get container status \"3f490f4748d638c664bb35dfe2cc4e66ee6198095b510a6a6a20abe19e133c7a\": rpc error: code = NotFound desc = could not find container \"3f490f4748d638c664bb35dfe2cc4e66ee6198095b510a6a6a20abe19e133c7a\": container with ID starting with 3f490f4748d638c664bb35dfe2cc4e66ee6198095b510a6a6a20abe19e133c7a not found: ID does not exist" Jul 09 15:33:39.504611 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:39.504579 2567 scope.go:117] "RemoveContainer" containerID="628b53963c8aeabd5ff8bd12b034a1274c3b476e7643d2eff73062bcdc3545bb" Jul 09 15:33:39.504872 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:33:39.504848 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"628b53963c8aeabd5ff8bd12b034a1274c3b476e7643d2eff73062bcdc3545bb\": container with ID starting with 628b53963c8aeabd5ff8bd12b034a1274c3b476e7643d2eff73062bcdc3545bb not found: ID does not exist" containerID="628b53963c8aeabd5ff8bd12b034a1274c3b476e7643d2eff73062bcdc3545bb" Jul 09 15:33:39.504952 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:39.504876 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"628b53963c8aeabd5ff8bd12b034a1274c3b476e7643d2eff73062bcdc3545bb"} err="failed to get container status \"628b53963c8aeabd5ff8bd12b034a1274c3b476e7643d2eff73062bcdc3545bb\": rpc error: code = NotFound desc = could not find container \"628b53963c8aeabd5ff8bd12b034a1274c3b476e7643d2eff73062bcdc3545bb\": container with ID starting with 628b53963c8aeabd5ff8bd12b034a1274c3b476e7643d2eff73062bcdc3545bb not found: ID does not exist" Jul 09 15:33:39.524405 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:39.524372 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc"] Jul 09 15:33:39.534347 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:39.534308 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-cd893609-openshift-default-79c978cc6c-bqspc"] Jul 09 15:33:39.829722 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:39.829636 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3175f218-ebab-4bdb-9c7a-3f5e90f9814c" path="/var/lib/kubelet/pods/3175f218-ebab-4bdb-9c7a-3f5e90f9814c/volumes" Jul 09 15:33:41.663366 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:41.663325 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:33:41.949546 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:41.949457 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:33:42.415967 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:42.415926 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:33:43.592224 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:43.592182 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:33:43.925272 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:43.925179 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:33:49.028343 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:49.028300 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:33:49.134206 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:49.134165 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:33:56.090763 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.090670 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:33:56.473045 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.473010 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z"] Jul 09 15:33:56.473587 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.473562 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3175f218-ebab-4bdb-9c7a-3f5e90f9814c" containerName="istio-proxy" Jul 09 15:33:56.473587 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.473584 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3175f218-ebab-4bdb-9c7a-3f5e90f9814c" containerName="istio-proxy" Jul 09 15:33:56.473765 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.473674 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3175f218-ebab-4bdb-9c7a-3f5e90f9814c" containerName="istio-proxy" Jul 09 15:33:56.478535 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.478515 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:33:56.482003 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.481982 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-7cc32d82-openshift-default-dockercfg-g9xd2\"" Jul 09 15:33:56.501790 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.501758 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z"] Jul 09 15:33:56.534727 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.534688 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:33:56.590355 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.590323 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1287044e-9971-42e4-9270-2a3f245075cd-istio-envoy\") pod \"e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z\" (UID: \"1287044e-9971-42e4-9270-2a3f245075cd\") " pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:33:56.590514 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.590372 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwmwk\" (UniqueName: \"kubernetes.io/projected/1287044e-9971-42e4-9270-2a3f245075cd-kube-api-access-rwmwk\") pod \"e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z\" (UID: \"1287044e-9971-42e4-9270-2a3f245075cd\") " pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:33:56.590514 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.590431 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1287044e-9971-42e4-9270-2a3f245075cd-workload-certs\") pod \"e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z\" (UID: \"1287044e-9971-42e4-9270-2a3f245075cd\") " pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:33:56.590514 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.590470 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1287044e-9971-42e4-9270-2a3f245075cd-istio-token\") pod \"e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z\" (UID: \"1287044e-9971-42e4-9270-2a3f245075cd\") " pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:33:56.590514 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.590494 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1287044e-9971-42e4-9270-2a3f245075cd-istio-data\") pod \"e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z\" (UID: \"1287044e-9971-42e4-9270-2a3f245075cd\") " pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:33:56.590675 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.590587 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1287044e-9971-42e4-9270-2a3f245075cd-istiod-ca-cert\") pod \"e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z\" (UID: \"1287044e-9971-42e4-9270-2a3f245075cd\") " pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:33:56.590675 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.590631 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1287044e-9971-42e4-9270-2a3f245075cd-credential-socket\") pod \"e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z\" (UID: \"1287044e-9971-42e4-9270-2a3f245075cd\") " pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:33:56.590741 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.590698 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1287044e-9971-42e4-9270-2a3f245075cd-istio-podinfo\") pod \"e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z\" (UID: \"1287044e-9971-42e4-9270-2a3f245075cd\") " pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:33:56.590776 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.590745 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1287044e-9971-42e4-9270-2a3f245075cd-workload-socket\") pod \"e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z\" (UID: \"1287044e-9971-42e4-9270-2a3f245075cd\") " pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:33:56.691913 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.691870 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1287044e-9971-42e4-9270-2a3f245075cd-credential-socket\") pod \"e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z\" (UID: \"1287044e-9971-42e4-9270-2a3f245075cd\") " pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:33:56.692098 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.691935 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1287044e-9971-42e4-9270-2a3f245075cd-istio-podinfo\") pod \"e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z\" (UID: \"1287044e-9971-42e4-9270-2a3f245075cd\") " pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:33:56.692098 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.691977 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1287044e-9971-42e4-9270-2a3f245075cd-workload-socket\") pod \"e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z\" (UID: \"1287044e-9971-42e4-9270-2a3f245075cd\") " pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:33:56.692098 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.692000 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1287044e-9971-42e4-9270-2a3f245075cd-istio-envoy\") pod \"e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z\" (UID: \"1287044e-9971-42e4-9270-2a3f245075cd\") " pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:33:56.692098 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.692043 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwmwk\" (UniqueName: \"kubernetes.io/projected/1287044e-9971-42e4-9270-2a3f245075cd-kube-api-access-rwmwk\") pod \"e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z\" (UID: \"1287044e-9971-42e4-9270-2a3f245075cd\") " pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:33:56.692098 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.692081 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1287044e-9971-42e4-9270-2a3f245075cd-workload-certs\") pod \"e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z\" (UID: \"1287044e-9971-42e4-9270-2a3f245075cd\") " pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:33:56.692321 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.692106 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1287044e-9971-42e4-9270-2a3f245075cd-istio-token\") pod \"e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z\" (UID: \"1287044e-9971-42e4-9270-2a3f245075cd\") " pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:33:56.692321 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.692129 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1287044e-9971-42e4-9270-2a3f245075cd-istio-data\") pod \"e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z\" (UID: \"1287044e-9971-42e4-9270-2a3f245075cd\") " pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:33:56.692321 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.692207 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1287044e-9971-42e4-9270-2a3f245075cd-istiod-ca-cert\") pod \"e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z\" (UID: \"1287044e-9971-42e4-9270-2a3f245075cd\") " pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:33:56.692461 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.692344 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1287044e-9971-42e4-9270-2a3f245075cd-credential-socket\") pod \"e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z\" (UID: \"1287044e-9971-42e4-9270-2a3f245075cd\") " pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:33:56.692461 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.692353 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1287044e-9971-42e4-9270-2a3f245075cd-workload-socket\") pod \"e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z\" (UID: \"1287044e-9971-42e4-9270-2a3f245075cd\") " pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:33:56.692649 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.692629 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1287044e-9971-42e4-9270-2a3f245075cd-istio-data\") pod \"e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z\" (UID: \"1287044e-9971-42e4-9270-2a3f245075cd\") " pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:33:56.692728 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.692629 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1287044e-9971-42e4-9270-2a3f245075cd-workload-certs\") pod \"e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z\" (UID: \"1287044e-9971-42e4-9270-2a3f245075cd\") " pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:33:56.692952 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.692931 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1287044e-9971-42e4-9270-2a3f245075cd-istiod-ca-cert\") pod \"e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z\" (UID: \"1287044e-9971-42e4-9270-2a3f245075cd\") " pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:33:56.694473 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.694451 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1287044e-9971-42e4-9270-2a3f245075cd-istio-envoy\") pod \"e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z\" (UID: \"1287044e-9971-42e4-9270-2a3f245075cd\") " pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:33:56.694602 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.694583 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1287044e-9971-42e4-9270-2a3f245075cd-istio-podinfo\") pod \"e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z\" (UID: \"1287044e-9971-42e4-9270-2a3f245075cd\") " pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:33:56.713871 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.713841 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1287044e-9971-42e4-9270-2a3f245075cd-istio-token\") pod \"e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z\" (UID: \"1287044e-9971-42e4-9270-2a3f245075cd\") " pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:33:56.714100 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.714079 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwmwk\" (UniqueName: \"kubernetes.io/projected/1287044e-9971-42e4-9270-2a3f245075cd-kube-api-access-rwmwk\") pod \"e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z\" (UID: \"1287044e-9971-42e4-9270-2a3f245075cd\") " pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:33:56.751061 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.750961 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:33:56.790078 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.790040 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:33:56.947911 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.947884 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z"] Jul 09 15:33:56.950209 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:33:56.950182 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1287044e_9971_42e4_9270_2a3f245075cd.slice/crio-c51ca79f1d4aaa215d67c884b7bc3787356691e042b792161eeb2a6fffac04f7 WatchSource:0}: Error finding container c51ca79f1d4aaa215d67c884b7bc3787356691e042b792161eeb2a6fffac04f7: Status 404 returned error can't find the container with id c51ca79f1d4aaa215d67c884b7bc3787356691e042b792161eeb2a6fffac04f7 Jul 09 15:33:56.952771 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.952737 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:33:56.952883 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.952805 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:33:56.952883 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:56.952855 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:33:57.587671 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:57.587630 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" event={"ID":"1287044e-9971-42e4-9270-2a3f245075cd","Type":"ContainerStarted","Data":"b0fce9f6844519af844242d0c6d4edb6dcd42ad17f54dd8f364cbec74f09315b"} Jul 09 15:33:57.588111 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:57.587700 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" event={"ID":"1287044e-9971-42e4-9270-2a3f245075cd","Type":"ContainerStarted","Data":"c51ca79f1d4aaa215d67c884b7bc3787356691e042b792161eeb2a6fffac04f7"} Jul 09 15:33:57.618089 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:57.618001 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" podStartSLOduration=1.617985473 podStartE2EDuration="1.617985473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:33:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:33:57.616532764 +0000 UTC m=+1884.347039256" watchObservedRunningTime="2026-07-09 15:33:57.617985473 +0000 UTC m=+1884.348491963" Jul 09 15:33:57.790794 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:57.790752 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:33:58.795170 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:58.795138 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:33:59.598079 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:59.598046 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:33:59.599181 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:59.599151 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:33:59.732254 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:59.732215 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:33:59.858356 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:33:59.858267 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:34:03.079410 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:34:03.079370 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:34:03.183837 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:34:03.183776 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:34:04.193741 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:34:04.193702 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:34:12.671520 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:34:12.671405 2567 generic.go:356] "Generic (PLEG): container finished" podID="1287044e-9971-42e4-9270-2a3f245075cd" containerID="b0fce9f6844519af844242d0c6d4edb6dcd42ad17f54dd8f364cbec74f09315b" exitCode=0 Jul 09 15:34:12.671520 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:34:12.671482 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" event={"ID":"1287044e-9971-42e4-9270-2a3f245075cd","Type":"ContainerDied","Data":"b0fce9f6844519af844242d0c6d4edb6dcd42ad17f54dd8f364cbec74f09315b"} Jul 09 15:34:12.672042 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:34:12.671974 2567 scope.go:117] "RemoveContainer" containerID="b0fce9f6844519af844242d0c6d4edb6dcd42ad17f54dd8f364cbec74f09315b" Jul 09 15:34:12.672980 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:34:12.672955 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:34:12.673047 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:34:12.673021 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:34:12.673091 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:34:12.673049 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:34:13.678883 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:34:13.678846 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" event={"ID":"1287044e-9971-42e4-9270-2a3f245075cd","Type":"ContainerStarted","Data":"9a8e04e220c60bd3cd00cfdb4dd19e036582868f4e69361776b87bad3cf2a893"} Jul 09 15:34:13.791218 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:34:13.791177 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:34:13.792293 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:34:13.792239 2567 patch_prober.go:28] interesting pod/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 09 15:34:13.792455 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:34:13.792334 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" podUID="1287044e-9971-42e4-9270-2a3f245075cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 09 15:34:14.598318 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:34:14.598279 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:34:14.791137 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:34:14.791102 2567 patch_prober.go:28] interesting pod/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 09 15:34:14.791722 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:34:14.791690 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" podUID="1287044e-9971-42e4-9270-2a3f245075cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 09 15:34:15.790999 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:34:15.790956 2567 patch_prober.go:28] interesting pod/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 09 15:34:15.791232 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:34:15.791030 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" podUID="1287044e-9971-42e4-9270-2a3f245075cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 09 15:34:16.790534 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:34:16.790499 2567 patch_prober.go:28] interesting pod/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 09 15:34:16.790706 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:34:16.790558 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" podUID="1287044e-9971-42e4-9270-2a3f245075cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 09 15:34:17.799771 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:34:17.799738 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:34:17.801039 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:34:17.801018 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:34:22.718027 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:34:22.717987 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:34:43.037748 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:34:43.037687 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:34:43.149127 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:34:43.149088 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:34:48.206464 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:34:48.206428 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:34:53.262056 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:34:53.262022 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:35:29.900311 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:29.900224 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:35:32.990494 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:32.990454 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:35:37.375289 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:37.375233 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:35:37.659540 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:37.659452 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z"] Jul 09 15:35:37.659724 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:37.659700 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" podUID="1287044e-9971-42e4-9270-2a3f245075cd" containerName="istio-proxy" containerID="cri-o://9a8e04e220c60bd3cd00cfdb4dd19e036582868f4e69361776b87bad3cf2a893" gracePeriod=30 Jul 09 15:35:40.389525 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:40.389461 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:35:41.842710 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:41.842658 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:35:42.930266 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:42.930195 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:35:43.025759 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.025719 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1287044e-9971-42e4-9270-2a3f245075cd-workload-socket\") pod \"1287044e-9971-42e4-9270-2a3f245075cd\" (UID: \"1287044e-9971-42e4-9270-2a3f245075cd\") " Jul 09 15:35:43.025970 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.025766 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1287044e-9971-42e4-9270-2a3f245075cd-istio-data\") pod \"1287044e-9971-42e4-9270-2a3f245075cd\" (UID: \"1287044e-9971-42e4-9270-2a3f245075cd\") " Jul 09 15:35:43.025970 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.025856 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1287044e-9971-42e4-9270-2a3f245075cd-istio-envoy\") pod \"1287044e-9971-42e4-9270-2a3f245075cd\" (UID: \"1287044e-9971-42e4-9270-2a3f245075cd\") " Jul 09 15:35:43.025970 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.025880 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1287044e-9971-42e4-9270-2a3f245075cd-istio-token\") pod \"1287044e-9971-42e4-9270-2a3f245075cd\" (UID: \"1287044e-9971-42e4-9270-2a3f245075cd\") " Jul 09 15:35:43.025970 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.025913 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rwmwk\" (UniqueName: \"kubernetes.io/projected/1287044e-9971-42e4-9270-2a3f245075cd-kube-api-access-rwmwk\") pod \"1287044e-9971-42e4-9270-2a3f245075cd\" (UID: \"1287044e-9971-42e4-9270-2a3f245075cd\") " Jul 09 15:35:43.025970 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.025968 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1287044e-9971-42e4-9270-2a3f245075cd-workload-certs\") pod \"1287044e-9971-42e4-9270-2a3f245075cd\" (UID: \"1287044e-9971-42e4-9270-2a3f245075cd\") " Jul 09 15:35:43.026239 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.025996 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1287044e-9971-42e4-9270-2a3f245075cd-istio-podinfo\") pod \"1287044e-9971-42e4-9270-2a3f245075cd\" (UID: \"1287044e-9971-42e4-9270-2a3f245075cd\") " Jul 09 15:35:43.026239 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.026115 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1287044e-9971-42e4-9270-2a3f245075cd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1287044e-9971-42e4-9270-2a3f245075cd" (UID: "1287044e-9971-42e4-9270-2a3f245075cd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:35:43.026239 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.026166 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1287044e-9971-42e4-9270-2a3f245075cd-istiod-ca-cert\") pod \"1287044e-9971-42e4-9270-2a3f245075cd\" (UID: \"1287044e-9971-42e4-9270-2a3f245075cd\") " Jul 09 15:35:43.026239 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.026224 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1287044e-9971-42e4-9270-2a3f245075cd-credential-socket\") pod \"1287044e-9971-42e4-9270-2a3f245075cd\" (UID: \"1287044e-9971-42e4-9270-2a3f245075cd\") " Jul 09 15:35:43.026461 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.026363 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1287044e-9971-42e4-9270-2a3f245075cd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1287044e-9971-42e4-9270-2a3f245075cd" (UID: "1287044e-9971-42e4-9270-2a3f245075cd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:35:43.026579 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.026558 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1287044e-9971-42e4-9270-2a3f245075cd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1287044e-9971-42e4-9270-2a3f245075cd" (UID: "1287044e-9971-42e4-9270-2a3f245075cd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:35:43.026648 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.026602 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1287044e-9971-42e4-9270-2a3f245075cd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1287044e-9971-42e4-9270-2a3f245075cd" (UID: "1287044e-9971-42e4-9270-2a3f245075cd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:35:43.026711 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.026640 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1287044e-9971-42e4-9270-2a3f245075cd-istio-data" (OuterVolumeSpecName: "istio-data") pod "1287044e-9971-42e4-9270-2a3f245075cd" (UID: "1287044e-9971-42e4-9270-2a3f245075cd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:35:43.026711 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.026701 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1287044e-9971-42e4-9270-2a3f245075cd-workload-certs\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:35:43.026845 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.026723 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1287044e-9971-42e4-9270-2a3f245075cd-credential-socket\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:35:43.026845 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.026739 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1287044e-9971-42e4-9270-2a3f245075cd-workload-socket\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:35:43.028219 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.028186 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1287044e-9971-42e4-9270-2a3f245075cd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1287044e-9971-42e4-9270-2a3f245075cd" (UID: "1287044e-9971-42e4-9270-2a3f245075cd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:35:43.028219 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.028204 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1287044e-9971-42e4-9270-2a3f245075cd-kube-api-access-rwmwk" (OuterVolumeSpecName: "kube-api-access-rwmwk") pod "1287044e-9971-42e4-9270-2a3f245075cd" (UID: "1287044e-9971-42e4-9270-2a3f245075cd"). InnerVolumeSpecName "kube-api-access-rwmwk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:35:43.028219 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.028187 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1287044e-9971-42e4-9270-2a3f245075cd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1287044e-9971-42e4-9270-2a3f245075cd" (UID: "1287044e-9971-42e4-9270-2a3f245075cd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 15:35:43.028405 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.028245 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1287044e-9971-42e4-9270-2a3f245075cd-istio-token" (OuterVolumeSpecName: "istio-token") pod "1287044e-9971-42e4-9270-2a3f245075cd" (UID: "1287044e-9971-42e4-9270-2a3f245075cd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:35:43.127598 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.127560 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1287044e-9971-42e4-9270-2a3f245075cd-istio-envoy\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:35:43.127598 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.127591 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1287044e-9971-42e4-9270-2a3f245075cd-istio-token\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:35:43.127598 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.127600 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rwmwk\" (UniqueName: \"kubernetes.io/projected/1287044e-9971-42e4-9270-2a3f245075cd-kube-api-access-rwmwk\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:35:43.127598 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.127610 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1287044e-9971-42e4-9270-2a3f245075cd-istio-podinfo\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:35:43.127924 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.127618 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1287044e-9971-42e4-9270-2a3f245075cd-istiod-ca-cert\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:35:43.127924 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.127626 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1287044e-9971-42e4-9270-2a3f245075cd-istio-data\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:35:43.158339 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.158291 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:35:43.158548 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.158377 2567 generic.go:356] "Generic (PLEG): container finished" podID="1287044e-9971-42e4-9270-2a3f245075cd" containerID="9a8e04e220c60bd3cd00cfdb4dd19e036582868f4e69361776b87bad3cf2a893" exitCode=0 Jul 09 15:35:43.158548 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.158406 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" event={"ID":"1287044e-9971-42e4-9270-2a3f245075cd","Type":"ContainerDied","Data":"9a8e04e220c60bd3cd00cfdb4dd19e036582868f4e69361776b87bad3cf2a893"} Jul 09 15:35:43.158548 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.158443 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" Jul 09 15:35:43.158548 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.158460 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z" event={"ID":"1287044e-9971-42e4-9270-2a3f245075cd","Type":"ContainerDied","Data":"c51ca79f1d4aaa215d67c884b7bc3787356691e042b792161eeb2a6fffac04f7"} Jul 09 15:35:43.158548 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.158484 2567 scope.go:117] "RemoveContainer" containerID="9a8e04e220c60bd3cd00cfdb4dd19e036582868f4e69361776b87bad3cf2a893" Jul 09 15:35:43.168906 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.168886 2567 scope.go:117] "RemoveContainer" containerID="b0fce9f6844519af844242d0c6d4edb6dcd42ad17f54dd8f364cbec74f09315b" Jul 09 15:35:43.177577 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.177553 2567 scope.go:117] "RemoveContainer" containerID="9a8e04e220c60bd3cd00cfdb4dd19e036582868f4e69361776b87bad3cf2a893" Jul 09 15:35:43.177959 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:35:43.177936 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a8e04e220c60bd3cd00cfdb4dd19e036582868f4e69361776b87bad3cf2a893\": container with ID starting with 9a8e04e220c60bd3cd00cfdb4dd19e036582868f4e69361776b87bad3cf2a893 not found: ID does not exist" containerID="9a8e04e220c60bd3cd00cfdb4dd19e036582868f4e69361776b87bad3cf2a893" Jul 09 15:35:43.178023 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.177970 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a8e04e220c60bd3cd00cfdb4dd19e036582868f4e69361776b87bad3cf2a893"} err="failed to get container status \"9a8e04e220c60bd3cd00cfdb4dd19e036582868f4e69361776b87bad3cf2a893\": rpc error: code = NotFound desc = could not find container \"9a8e04e220c60bd3cd00cfdb4dd19e036582868f4e69361776b87bad3cf2a893\": container with ID starting with 9a8e04e220c60bd3cd00cfdb4dd19e036582868f4e69361776b87bad3cf2a893 not found: ID does not exist" Jul 09 15:35:43.178023 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.177990 2567 scope.go:117] "RemoveContainer" containerID="b0fce9f6844519af844242d0c6d4edb6dcd42ad17f54dd8f364cbec74f09315b" Jul 09 15:35:43.178256 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:35:43.178235 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0fce9f6844519af844242d0c6d4edb6dcd42ad17f54dd8f364cbec74f09315b\": container with ID starting with b0fce9f6844519af844242d0c6d4edb6dcd42ad17f54dd8f364cbec74f09315b not found: ID does not exist" containerID="b0fce9f6844519af844242d0c6d4edb6dcd42ad17f54dd8f364cbec74f09315b" Jul 09 15:35:43.178318 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.178267 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0fce9f6844519af844242d0c6d4edb6dcd42ad17f54dd8f364cbec74f09315b"} err="failed to get container status \"b0fce9f6844519af844242d0c6d4edb6dcd42ad17f54dd8f364cbec74f09315b\": rpc error: code = NotFound desc = could not find container \"b0fce9f6844519af844242d0c6d4edb6dcd42ad17f54dd8f364cbec74f09315b\": container with ID starting with b0fce9f6844519af844242d0c6d4edb6dcd42ad17f54dd8f364cbec74f09315b not found: ID does not exist" Jul 09 15:35:43.187247 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.187162 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z"] Jul 09 15:35:43.193262 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.193231 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-7cc32d82-openshift-default-7d64db986c-nmc7z"] Jul 09 15:35:43.280636 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.280602 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:35:43.831901 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:43.831868 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1287044e-9971-42e4-9270-2a3f245075cd" path="/var/lib/kubelet/pods/1287044e-9971-42e4-9270-2a3f245075cd/volumes" Jul 09 15:35:48.060551 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:48.060505 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:35:48.173196 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:48.173158 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:35:48.771192 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:48.771146 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:35:52.399427 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:52.399385 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:35:55.172830 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:55.172777 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:35:55.402654 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:55.402615 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:35:55.492313 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:35:55.492271 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:36:05.960141 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:05.960108 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:36:06.344163 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:06.344064 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:36:27.336226 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:27.336188 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:36:33.714201 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:33.714148 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:36:33.808733 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:33.808679 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:36:34.046758 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.046726 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:36:34.368578 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.368479 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k"] Jul 09 15:36:34.369044 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.369026 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3175f218-ebab-4bdb-9c7a-3f5e90f9814c" containerName="istio-proxy" Jul 09 15:36:34.369093 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.369047 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3175f218-ebab-4bdb-9c7a-3f5e90f9814c" containerName="istio-proxy" Jul 09 15:36:34.369093 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.369069 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1287044e-9971-42e4-9270-2a3f245075cd" containerName="istio-proxy" Jul 09 15:36:34.369093 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.369076 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1287044e-9971-42e4-9270-2a3f245075cd" containerName="istio-proxy" Jul 09 15:36:34.369093 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.369084 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1287044e-9971-42e4-9270-2a3f245075cd" containerName="istio-proxy" Jul 09 15:36:34.369093 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.369090 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1287044e-9971-42e4-9270-2a3f245075cd" containerName="istio-proxy" Jul 09 15:36:34.369601 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.369165 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3175f218-ebab-4bdb-9c7a-3f5e90f9814c" containerName="istio-proxy" Jul 09 15:36:34.369601 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.369174 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="1287044e-9971-42e4-9270-2a3f245075cd" containerName="istio-proxy" Jul 09 15:36:34.369601 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.369183 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="1287044e-9971-42e4-9270-2a3f245075cd" containerName="istio-proxy" Jul 09 15:36:34.373737 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.373712 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:36:34.376973 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.376946 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-dc88a7-openshift-default-dockercfg-v7th9\"" Jul 09 15:36:34.390045 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.390010 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k"] Jul 09 15:36:34.400185 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.400140 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:36:34.459076 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.459038 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-istiod-ca-cert\") pod \"e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k\" (UID: \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\") " pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:36:34.459290 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.459079 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-istio-token\") pod \"e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k\" (UID: \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\") " pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:36:34.459290 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.459111 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-istio-data\") pod \"e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k\" (UID: \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\") " pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:36:34.459290 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.459159 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-workload-certs\") pod \"e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k\" (UID: \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\") " pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:36:34.459290 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.459185 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-istio-envoy\") pod \"e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k\" (UID: \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\") " pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:36:34.459290 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.459227 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-workload-socket\") pod \"e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k\" (UID: \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\") " pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:36:34.459290 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.459272 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pkq4\" (UniqueName: \"kubernetes.io/projected/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-kube-api-access-6pkq4\") pod \"e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k\" (UID: \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\") " pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:36:34.459637 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.459307 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-credential-socket\") pod \"e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k\" (UID: \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\") " pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:36:34.459637 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.459366 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-istio-podinfo\") pod \"e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k\" (UID: \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\") " pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:36:34.561067 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.561023 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-istio-podinfo\") pod \"e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k\" (UID: \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\") " pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:36:34.561254 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.561124 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-istiod-ca-cert\") pod \"e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k\" (UID: \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\") " pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:36:34.561254 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.561156 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-istio-token\") pod \"e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k\" (UID: \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\") " pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:36:34.561378 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.561296 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-istio-data\") pod \"e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k\" (UID: \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\") " pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:36:34.561378 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.561345 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-workload-certs\") pod \"e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k\" (UID: \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\") " pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:36:34.561481 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.561398 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-istio-envoy\") pod \"e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k\" (UID: \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\") " pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:36:34.561481 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.561454 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-workload-socket\") pod \"e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k\" (UID: \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\") " pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:36:34.561583 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.561513 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pkq4\" (UniqueName: \"kubernetes.io/projected/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-kube-api-access-6pkq4\") pod \"e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k\" (UID: \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\") " pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:36:34.561583 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.561564 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-credential-socket\") pod \"e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k\" (UID: \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\") " pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:36:34.561721 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.561694 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-istio-data\") pod \"e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k\" (UID: \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\") " pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:36:34.561789 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.561749 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-workload-certs\") pod \"e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k\" (UID: \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\") " pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:36:34.561908 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.561885 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-workload-socket\") pod \"e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k\" (UID: \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\") " pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:36:34.562002 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.561984 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-credential-socket\") pod \"e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k\" (UID: \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\") " pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:36:34.562063 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.561987 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-istiod-ca-cert\") pod \"e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k\" (UID: \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\") " pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:36:34.564004 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.563978 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-istio-envoy\") pod \"e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k\" (UID: \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\") " pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:36:34.564142 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.564125 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-istio-podinfo\") pod \"e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k\" (UID: \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\") " pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:36:34.570404 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.570368 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-istio-token\") pod \"e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k\" (UID: \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\") " pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:36:34.570519 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.570464 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pkq4\" (UniqueName: \"kubernetes.io/projected/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-kube-api-access-6pkq4\") pod \"e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k\" (UID: \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\") " pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:36:34.669425 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.669329 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:36:34.684380 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.684344 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:36:34.836792 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.836764 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k"] Jul 09 15:36:34.839453 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:36:34.839426 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb5f37a1_8f78_4bd3_ab60_f1538d6020bc.slice/crio-3a9de9009ba6cbf869538f499c878153271592b92990a4be21246d4e4720b118 WatchSource:0}: Error finding container 3a9de9009ba6cbf869538f499c878153271592b92990a4be21246d4e4720b118: Status 404 returned error can't find the container with id 3a9de9009ba6cbf869538f499c878153271592b92990a4be21246d4e4720b118 Jul 09 15:36:34.841938 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.841898 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:36:34.842035 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.841993 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:36:34.842079 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:34.842037 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:36:35.442389 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:35.442345 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" event={"ID":"db5f37a1-8f78-4bd3-ab60-f1538d6020bc","Type":"ContainerStarted","Data":"518706e26c93010298539e2a918459c5e48736f69e2faf4e86e07511d1605d89"} Jul 09 15:36:35.442578 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:35.442413 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" event={"ID":"db5f37a1-8f78-4bd3-ab60-f1538d6020bc","Type":"ContainerStarted","Data":"3a9de9009ba6cbf869538f499c878153271592b92990a4be21246d4e4720b118"} Jul 09 15:36:35.471858 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:35.471780 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" podStartSLOduration=1.471762163 podStartE2EDuration="1.471762163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:36:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:36:35.469637397 +0000 UTC m=+2042.200143913" watchObservedRunningTime="2026-07-09 15:36:35.471762163 +0000 UTC m=+2042.202268653" Jul 09 15:36:35.684862 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:35.684812 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:36:35.689774 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:35.689743 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:36:36.447980 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:36.447950 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:36:36.448942 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:36.448924 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:36:37.447510 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:37.447464 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:36:37.587122 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:37.587085 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:36:37.677352 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:37.677311 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:36:37.822576 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:37.822533 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:36:38.163654 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:38.163548 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:36:38.266020 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:38.265982 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:36:41.960994 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:41.960950 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:36:42.608291 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:42.608257 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb"] Jul 09 15:36:42.612001 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:42.611977 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:36:42.614741 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:42.614712 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-d62682-openshift-default-dockercfg-scpgq\"" Jul 09 15:36:42.633051 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:42.633008 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb"] Jul 09 15:36:42.743612 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:42.743569 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9567e15d-f443-47f0-ba63-d17896e08a0b-istio-data\") pod \"e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb\" (UID: \"9567e15d-f443-47f0-ba63-d17896e08a0b\") " pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:36:42.743880 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:42.743630 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9567e15d-f443-47f0-ba63-d17896e08a0b-istiod-ca-cert\") pod \"e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb\" (UID: \"9567e15d-f443-47f0-ba63-d17896e08a0b\") " pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:36:42.743880 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:42.743699 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9567e15d-f443-47f0-ba63-d17896e08a0b-credential-socket\") pod \"e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb\" (UID: \"9567e15d-f443-47f0-ba63-d17896e08a0b\") " pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:36:42.743880 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:42.743728 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sngm\" (UniqueName: \"kubernetes.io/projected/9567e15d-f443-47f0-ba63-d17896e08a0b-kube-api-access-4sngm\") pod \"e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb\" (UID: \"9567e15d-f443-47f0-ba63-d17896e08a0b\") " pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:36:42.743880 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:42.743801 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9567e15d-f443-47f0-ba63-d17896e08a0b-workload-socket\") pod \"e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb\" (UID: \"9567e15d-f443-47f0-ba63-d17896e08a0b\") " pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:36:42.743880 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:42.743867 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9567e15d-f443-47f0-ba63-d17896e08a0b-workload-certs\") pod \"e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb\" (UID: \"9567e15d-f443-47f0-ba63-d17896e08a0b\") " pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:36:42.744131 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:42.743895 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9567e15d-f443-47f0-ba63-d17896e08a0b-istio-envoy\") pod \"e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb\" (UID: \"9567e15d-f443-47f0-ba63-d17896e08a0b\") " pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:36:42.744131 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:42.743917 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9567e15d-f443-47f0-ba63-d17896e08a0b-istio-podinfo\") pod \"e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb\" (UID: \"9567e15d-f443-47f0-ba63-d17896e08a0b\") " pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:36:42.744131 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:42.743988 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9567e15d-f443-47f0-ba63-d17896e08a0b-istio-token\") pod \"e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb\" (UID: \"9567e15d-f443-47f0-ba63-d17896e08a0b\") " pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:36:42.845200 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:42.845153 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9567e15d-f443-47f0-ba63-d17896e08a0b-workload-socket\") pod \"e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb\" (UID: \"9567e15d-f443-47f0-ba63-d17896e08a0b\") " pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:36:42.845200 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:42.845206 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9567e15d-f443-47f0-ba63-d17896e08a0b-workload-certs\") pod \"e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb\" (UID: \"9567e15d-f443-47f0-ba63-d17896e08a0b\") " pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:36:42.845466 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:42.845235 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9567e15d-f443-47f0-ba63-d17896e08a0b-istio-envoy\") pod \"e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb\" (UID: \"9567e15d-f443-47f0-ba63-d17896e08a0b\") " pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:36:42.845466 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:42.845257 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9567e15d-f443-47f0-ba63-d17896e08a0b-istio-podinfo\") pod \"e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb\" (UID: \"9567e15d-f443-47f0-ba63-d17896e08a0b\") " pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:36:42.845466 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:42.845422 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9567e15d-f443-47f0-ba63-d17896e08a0b-istio-token\") pod \"e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb\" (UID: \"9567e15d-f443-47f0-ba63-d17896e08a0b\") " pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:36:42.845628 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:42.845513 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9567e15d-f443-47f0-ba63-d17896e08a0b-istio-data\") pod \"e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb\" (UID: \"9567e15d-f443-47f0-ba63-d17896e08a0b\") " pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:36:42.845628 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:42.845558 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9567e15d-f443-47f0-ba63-d17896e08a0b-istiod-ca-cert\") pod \"e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb\" (UID: \"9567e15d-f443-47f0-ba63-d17896e08a0b\") " pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:36:42.845628 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:42.845598 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9567e15d-f443-47f0-ba63-d17896e08a0b-workload-socket\") pod \"e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb\" (UID: \"9567e15d-f443-47f0-ba63-d17896e08a0b\") " pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:36:42.845628 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:42.845609 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9567e15d-f443-47f0-ba63-d17896e08a0b-credential-socket\") pod \"e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb\" (UID: \"9567e15d-f443-47f0-ba63-d17896e08a0b\") " pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:36:42.845805 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:42.845637 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9567e15d-f443-47f0-ba63-d17896e08a0b-workload-certs\") pod \"e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb\" (UID: \"9567e15d-f443-47f0-ba63-d17896e08a0b\") " pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:36:42.845805 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:42.845660 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sngm\" (UniqueName: \"kubernetes.io/projected/9567e15d-f443-47f0-ba63-d17896e08a0b-kube-api-access-4sngm\") pod \"e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb\" (UID: \"9567e15d-f443-47f0-ba63-d17896e08a0b\") " pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:36:42.845940 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:42.845915 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9567e15d-f443-47f0-ba63-d17896e08a0b-istio-data\") pod \"e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb\" (UID: \"9567e15d-f443-47f0-ba63-d17896e08a0b\") " pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:36:42.845992 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:42.845926 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9567e15d-f443-47f0-ba63-d17896e08a0b-credential-socket\") pod \"e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb\" (UID: \"9567e15d-f443-47f0-ba63-d17896e08a0b\") " pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:36:42.846347 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:42.846318 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9567e15d-f443-47f0-ba63-d17896e08a0b-istiod-ca-cert\") pod \"e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb\" (UID: \"9567e15d-f443-47f0-ba63-d17896e08a0b\") " pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:36:42.847724 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:42.847703 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9567e15d-f443-47f0-ba63-d17896e08a0b-istio-envoy\") pod \"e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb\" (UID: \"9567e15d-f443-47f0-ba63-d17896e08a0b\") " pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:36:42.847854 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:42.847799 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9567e15d-f443-47f0-ba63-d17896e08a0b-istio-podinfo\") pod \"e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb\" (UID: \"9567e15d-f443-47f0-ba63-d17896e08a0b\") " pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:36:42.856262 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:42.856234 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9567e15d-f443-47f0-ba63-d17896e08a0b-istio-token\") pod \"e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb\" (UID: \"9567e15d-f443-47f0-ba63-d17896e08a0b\") " pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:36:42.856516 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:42.856487 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sngm\" (UniqueName: \"kubernetes.io/projected/9567e15d-f443-47f0-ba63-d17896e08a0b-kube-api-access-4sngm\") pod \"e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb\" (UID: \"9567e15d-f443-47f0-ba63-d17896e08a0b\") " pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:36:42.923902 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:42.923783 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:36:43.080568 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:43.080540 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb"] Jul 09 15:36:43.082046 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:36:43.082018 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9567e15d_f443_47f0_ba63_d17896e08a0b.slice/crio-bf451a19c76d67a1905a88d71d0f25c5429452470bf00f5060f52e56619acfd1 WatchSource:0}: Error finding container bf451a19c76d67a1905a88d71d0f25c5429452470bf00f5060f52e56619acfd1: Status 404 returned error can't find the container with id bf451a19c76d67a1905a88d71d0f25c5429452470bf00f5060f52e56619acfd1 Jul 09 15:36:43.084532 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:43.084492 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:36:43.084617 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:43.084596 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:36:43.084696 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:43.084678 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:36:43.490635 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:43.490582 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" event={"ID":"9567e15d-f443-47f0-ba63-d17896e08a0b","Type":"ContainerStarted","Data":"e5b7277f8338bded3c16e7e7fba63bbed05f207c6a4934998716f9870d4e66c1"} Jul 09 15:36:43.490919 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:43.490655 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" event={"ID":"9567e15d-f443-47f0-ba63-d17896e08a0b","Type":"ContainerStarted","Data":"bf451a19c76d67a1905a88d71d0f25c5429452470bf00f5060f52e56619acfd1"} Jul 09 15:36:43.515189 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:43.515130 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" podStartSLOduration=1.515111917 podStartE2EDuration="1.515111917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:36:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:36:43.512964545 +0000 UTC m=+2050.243471048" watchObservedRunningTime="2026-07-09 15:36:43.515111917 +0000 UTC m=+2050.245618408" Jul 09 15:36:43.923938 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:43.923850 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:36:44.929399 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:44.929357 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:36:45.503523 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:45.503489 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:36:45.504439 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:45.504415 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:36:45.667350 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:45.667311 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:36:45.761381 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:45.761283 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:36:45.872643 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:45.872601 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:36:46.230067 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:46.230016 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:36:46.332989 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:46.332955 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:36:50.013660 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:50.013620 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:36:55.560966 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:55.560932 2567 generic.go:356] "Generic (PLEG): container finished" podID="db5f37a1-8f78-4bd3-ab60-f1538d6020bc" containerID="518706e26c93010298539e2a918459c5e48736f69e2faf4e86e07511d1605d89" exitCode=0 Jul 09 15:36:55.561396 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:55.561005 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" event={"ID":"db5f37a1-8f78-4bd3-ab60-f1538d6020bc","Type":"ContainerDied","Data":"518706e26c93010298539e2a918459c5e48736f69e2faf4e86e07511d1605d89"} Jul 09 15:36:55.561495 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:55.561478 2567 scope.go:117] "RemoveContainer" containerID="518706e26c93010298539e2a918459c5e48736f69e2faf4e86e07511d1605d89" Jul 09 15:36:55.562630 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:55.562604 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:36:55.562686 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:55.562673 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:36:55.562721 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:55.562709 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:36:56.570130 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:56.570096 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" event={"ID":"db5f37a1-8f78-4bd3-ab60-f1538d6020bc","Type":"ContainerStarted","Data":"a408a97669c3c7b78169cfd976069eecce85672766f2832cc081c0511fb1c5ac"} Jul 09 15:36:56.685572 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:56.685531 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:36:56.685930 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:56.685897 2567 patch_prober.go:28] interesting pod/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 09 15:36:56.686068 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:56.685979 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" podUID="db5f37a1-8f78-4bd3-ab60-f1538d6020bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 09 15:36:57.685023 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:57.684983 2567 patch_prober.go:28] interesting pod/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 09 15:36:57.685452 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:57.685039 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" podUID="db5f37a1-8f78-4bd3-ab60-f1538d6020bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 09 15:36:58.685190 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:58.685150 2567 patch_prober.go:28] interesting pod/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 09 15:36:58.685671 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:58.685227 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" podUID="db5f37a1-8f78-4bd3-ab60-f1538d6020bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 09 15:36:59.380629 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:59.380578 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:36:59.489768 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:59.489729 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:36:59.685103 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:59.685010 2567 patch_prober.go:28] interesting pod/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 09 15:36:59.685103 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:36:59.685065 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" podUID="db5f37a1-8f78-4bd3-ab60-f1538d6020bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 09 15:37:00.641217 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:00.641177 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:37:00.688555 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:00.688521 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:37:00.688990 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:00.688852 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:37:00.689595 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:00.689574 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:37:01.546093 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:01.546033 2567 patch_prober.go:28] interesting pod/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 09 15:37:01.546312 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:01.546130 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" podUID="9567e15d-f443-47f0-ba63-d17896e08a0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 09 15:37:03.613152 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:03.613122 2567 generic.go:356] "Generic (PLEG): container finished" podID="9567e15d-f443-47f0-ba63-d17896e08a0b" containerID="e5b7277f8338bded3c16e7e7fba63bbed05f207c6a4934998716f9870d4e66c1" exitCode=0 Jul 09 15:37:03.613589 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:03.613195 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" event={"ID":"9567e15d-f443-47f0-ba63-d17896e08a0b","Type":"ContainerDied","Data":"e5b7277f8338bded3c16e7e7fba63bbed05f207c6a4934998716f9870d4e66c1"} Jul 09 15:37:03.613635 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:03.613623 2567 scope.go:117] "RemoveContainer" containerID="e5b7277f8338bded3c16e7e7fba63bbed05f207c6a4934998716f9870d4e66c1" Jul 09 15:37:03.614664 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:03.614639 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:37:03.614718 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:03.614710 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:37:03.614756 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:03.614743 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:37:04.624025 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:04.623989 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" event={"ID":"9567e15d-f443-47f0-ba63-d17896e08a0b","Type":"ContainerStarted","Data":"30f8a4a7e422ea7ae9834320b028406959d0838ef33f7fbd0bd3cd58fa1af7f8"} Jul 09 15:37:04.924695 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:04.924580 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:37:04.924911 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:04.924881 2567 patch_prober.go:28] interesting pod/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 09 15:37:04.925009 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:04.924958 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" podUID="9567e15d-f443-47f0-ba63-d17896e08a0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 09 15:37:05.924773 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:05.924731 2567 patch_prober.go:28] interesting pod/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 09 15:37:05.925217 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:05.924792 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" podUID="9567e15d-f443-47f0-ba63-d17896e08a0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 09 15:37:06.924286 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:06.924241 2567 patch_prober.go:28] interesting pod/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 09 15:37:06.924526 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:06.924300 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" podUID="9567e15d-f443-47f0-ba63-d17896e08a0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 09 15:37:07.041579 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:07.041540 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:37:07.141942 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:07.141901 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:37:07.925257 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:07.925219 2567 patch_prober.go:28] interesting pod/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 09 15:37:07.925490 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:07.925285 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" podUID="9567e15d-f443-47f0-ba63-d17896e08a0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 09 15:37:08.927983 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:08.927949 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:37:08.928468 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:08.928232 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:37:08.929045 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:08.929026 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:37:13.175831 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:13.175791 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:37:27.307009 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:27.306958 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:37:33.870349 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:33.870313 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hml7m_84a10578-4913-456b-9fb7-e6440914b677/ovn-acl-logging/0.log" Jul 09 15:37:33.880248 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:33.880222 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hml7m_84a10578-4913-456b-9fb7-e6440914b677/ovn-acl-logging/0.log" Jul 09 15:37:48.794657 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:48.794610 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:37:49.668385 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:49.668341 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:37:49.843298 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:49.843259 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:37:50.239373 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:50.239336 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:37:50.439742 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:50.439704 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:37:50.538532 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:50.538434 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:37:50.716250 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:50.716214 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:37:53.146602 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:53.146566 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:37:55.306056 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:37:55.306014 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:38:30.722884 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:30.722787 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:38:30.971365 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:30.971327 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k"] Jul 09 15:38:30.971681 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:30.971650 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" podUID="db5f37a1-8f78-4bd3-ab60-f1538d6020bc" containerName="istio-proxy" containerID="cri-o://a408a97669c3c7b78169cfd976069eecce85672766f2832cc081c0511fb1c5ac" gracePeriod=30 Jul 09 15:38:31.009614 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:31.009518 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:38:31.460057 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:31.460019 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:38:31.867087 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:31.866991 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:38:31.995165 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:31.995125 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:38:32.098751 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:32.098709 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:38:32.335357 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:32.335307 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:38:36.134203 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:36.134172 2567 generic.go:356] "Generic (PLEG): container finished" podID="db5f37a1-8f78-4bd3-ab60-f1538d6020bc" containerID="a408a97669c3c7b78169cfd976069eecce85672766f2832cc081c0511fb1c5ac" exitCode=0 Jul 09 15:38:36.134688 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:36.134240 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" event={"ID":"db5f37a1-8f78-4bd3-ab60-f1538d6020bc","Type":"ContainerDied","Data":"a408a97669c3c7b78169cfd976069eecce85672766f2832cc081c0511fb1c5ac"} Jul 09 15:38:36.134688 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:36.134301 2567 scope.go:117] "RemoveContainer" containerID="518706e26c93010298539e2a918459c5e48736f69e2faf4e86e07511d1605d89" Jul 09 15:38:36.245405 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:36.245342 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:38:36.382800 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:36.382759 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-istio-data\") pod \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\" (UID: \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\") " Jul 09 15:38:36.382990 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:36.382864 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-workload-certs\") pod \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\" (UID: \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\") " Jul 09 15:38:36.382990 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:36.382914 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-credential-socket\") pod \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\" (UID: \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\") " Jul 09 15:38:36.382990 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:36.382976 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-istio-envoy\") pod \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\" (UID: \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\") " Jul 09 15:38:36.383150 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:36.383014 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6pkq4\" (UniqueName: \"kubernetes.io/projected/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-kube-api-access-6pkq4\") pod \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\" (UID: \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\") " Jul 09 15:38:36.383150 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:36.383041 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-workload-socket\") pod \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\" (UID: \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\") " Jul 09 15:38:36.383150 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:36.383082 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-istio-token\") pod \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\" (UID: \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\") " Jul 09 15:38:36.383150 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:36.383114 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-istio-podinfo\") pod \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\" (UID: \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\") " Jul 09 15:38:36.383354 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:36.383155 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-istiod-ca-cert\") pod \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\" (UID: \"db5f37a1-8f78-4bd3-ab60-f1538d6020bc\") " Jul 09 15:38:36.383354 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:36.383163 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "db5f37a1-8f78-4bd3-ab60-f1538d6020bc" (UID: "db5f37a1-8f78-4bd3-ab60-f1538d6020bc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:38:36.383354 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:36.383282 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "db5f37a1-8f78-4bd3-ab60-f1538d6020bc" (UID: "db5f37a1-8f78-4bd3-ab60-f1538d6020bc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:38:36.383495 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:36.383352 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "db5f37a1-8f78-4bd3-ab60-f1538d6020bc" (UID: "db5f37a1-8f78-4bd3-ab60-f1538d6020bc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:38:36.383615 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:36.383593 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "db5f37a1-8f78-4bd3-ab60-f1538d6020bc" (UID: "db5f37a1-8f78-4bd3-ab60-f1538d6020bc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:38:36.383682 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:36.383661 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-workload-socket\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:38:36.383739 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:36.383687 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-workload-certs\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:38:36.383739 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:36.383693 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-istio-data" (OuterVolumeSpecName: "istio-data") pod "db5f37a1-8f78-4bd3-ab60-f1538d6020bc" (UID: "db5f37a1-8f78-4bd3-ab60-f1538d6020bc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:38:36.383739 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:36.383703 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-credential-socket\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:38:36.385347 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:36.385321 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "db5f37a1-8f78-4bd3-ab60-f1538d6020bc" (UID: "db5f37a1-8f78-4bd3-ab60-f1538d6020bc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 15:38:36.385480 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:36.385419 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-istio-token" (OuterVolumeSpecName: "istio-token") pod "db5f37a1-8f78-4bd3-ab60-f1538d6020bc" (UID: "db5f37a1-8f78-4bd3-ab60-f1538d6020bc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:38:36.385651 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:36.385618 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-kube-api-access-6pkq4" (OuterVolumeSpecName: "kube-api-access-6pkq4") pod "db5f37a1-8f78-4bd3-ab60-f1538d6020bc" (UID: "db5f37a1-8f78-4bd3-ab60-f1538d6020bc"). InnerVolumeSpecName "kube-api-access-6pkq4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:38:36.385651 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:36.385631 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "db5f37a1-8f78-4bd3-ab60-f1538d6020bc" (UID: "db5f37a1-8f78-4bd3-ab60-f1538d6020bc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:38:36.485259 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:36.485208 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-istio-envoy\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:38:36.485259 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:36.485253 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6pkq4\" (UniqueName: \"kubernetes.io/projected/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-kube-api-access-6pkq4\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:38:36.485259 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:36.485263 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-istio-token\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:38:36.485259 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:36.485272 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-istio-podinfo\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:38:36.485259 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:36.485282 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-istiod-ca-cert\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:38:36.485575 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:36.485290 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db5f37a1-8f78-4bd3-ab60-f1538d6020bc-istio-data\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:38:37.141065 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:37.141033 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" Jul 09 15:38:37.141495 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:37.141037 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k" event={"ID":"db5f37a1-8f78-4bd3-ab60-f1538d6020bc","Type":"ContainerDied","Data":"3a9de9009ba6cbf869538f499c878153271592b92990a4be21246d4e4720b118"} Jul 09 15:38:37.141495 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:37.141165 2567 scope.go:117] "RemoveContainer" containerID="a408a97669c3c7b78169cfd976069eecce85672766f2832cc081c0511fb1c5ac" Jul 09 15:38:37.164548 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:37.164511 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k"] Jul 09 15:38:37.171500 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:37.171471 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-dc88a7-openshift-default-7b74868695-rmr7k"] Jul 09 15:38:37.829199 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:37.829167 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db5f37a1-8f78-4bd3-ab60-f1538d6020bc" path="/var/lib/kubelet/pods/db5f37a1-8f78-4bd3-ab60-f1538d6020bc/volumes" Jul 09 15:38:38.285201 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:38.285168 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:38:38.391925 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:38.391890 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:38:38.558326 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:38.558227 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb"] Jul 09 15:38:38.558527 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:38.558503 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" podUID="9567e15d-f443-47f0-ba63-d17896e08a0b" containerName="istio-proxy" containerID="cri-o://30f8a4a7e422ea7ae9834320b028406959d0838ef33f7fbd0bd3cd58fa1af7f8" gracePeriod=30 Jul 09 15:38:38.596449 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:38.596402 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:38:38.931281 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:38.931190 2567 patch_prober.go:28] interesting pod/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 09 15:38:38.931430 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:38.931278 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" podUID="9567e15d-f443-47f0-ba63-d17896e08a0b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 15:38:39.153496 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.153456 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:38:39.508128 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.508094 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4"] Jul 09 15:38:39.508623 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.508608 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db5f37a1-8f78-4bd3-ab60-f1538d6020bc" containerName="istio-proxy" Jul 09 15:38:39.508671 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.508624 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="db5f37a1-8f78-4bd3-ab60-f1538d6020bc" containerName="istio-proxy" Jul 09 15:38:39.508671 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.508643 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db5f37a1-8f78-4bd3-ab60-f1538d6020bc" containerName="istio-proxy" Jul 09 15:38:39.508671 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.508649 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="db5f37a1-8f78-4bd3-ab60-f1538d6020bc" containerName="istio-proxy" Jul 09 15:38:39.508781 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.508708 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="db5f37a1-8f78-4bd3-ab60-f1538d6020bc" containerName="istio-proxy" Jul 09 15:38:39.508781 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.508715 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="db5f37a1-8f78-4bd3-ab60-f1538d6020bc" containerName="istio-proxy" Jul 09 15:38:39.511053 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.511036 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:38:39.517890 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.517860 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-0164f1-openshift-default-dockercfg-pj9bm\"" Jul 09 15:38:39.531880 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.531811 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4"] Jul 09 15:38:39.548404 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.548374 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:38:39.620705 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.620658 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-istio-data\") pod \"e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4\" (UID: \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\") " pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:38:39.620705 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.620709 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59mjj\" (UniqueName: \"kubernetes.io/projected/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-kube-api-access-59mjj\") pod \"e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4\" (UID: \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\") " pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:38:39.620950 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.620796 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-workload-certs\") pod \"e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4\" (UID: \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\") " pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:38:39.620950 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.620882 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-istio-podinfo\") pod \"e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4\" (UID: \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\") " pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:38:39.620950 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.620914 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-istio-token\") pod \"e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4\" (UID: \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\") " pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:38:39.621062 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.620982 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-istio-envoy\") pod \"e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4\" (UID: \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\") " pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:38:39.621062 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.621025 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-istiod-ca-cert\") pod \"e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4\" (UID: \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\") " pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:38:39.621062 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.621056 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-credential-socket\") pod \"e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4\" (UID: \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\") " pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:38:39.621152 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.621080 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-workload-socket\") pod \"e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4\" (UID: \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\") " pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:38:39.721708 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.721674 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-workload-certs\") pod \"e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4\" (UID: \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\") " pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:38:39.721940 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.721760 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-istio-podinfo\") pod \"e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4\" (UID: \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\") " pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:38:39.721940 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.721798 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-istio-token\") pod \"e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4\" (UID: \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\") " pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:38:39.721940 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.721853 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-istio-envoy\") pod \"e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4\" (UID: \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\") " pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:38:39.721940 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.721890 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-istiod-ca-cert\") pod \"e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4\" (UID: \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\") " pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:38:39.721940 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.721923 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-credential-socket\") pod \"e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4\" (UID: \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\") " pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:38:39.722170 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.721974 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-workload-socket\") pod \"e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4\" (UID: \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\") " pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:38:39.722170 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.722035 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-istio-data\") pod \"e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4\" (UID: \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\") " pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:38:39.722170 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.722072 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59mjj\" (UniqueName: \"kubernetes.io/projected/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-kube-api-access-59mjj\") pod \"e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4\" (UID: \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\") " pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:38:39.722170 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.722141 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-workload-certs\") pod \"e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4\" (UID: \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\") " pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:38:39.722428 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.722403 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-credential-socket\") pod \"e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4\" (UID: \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\") " pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:38:39.722498 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.722406 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-istio-data\") pod \"e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4\" (UID: \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\") " pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:38:39.722676 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.722654 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-istiod-ca-cert\") pod \"e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4\" (UID: \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\") " pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:38:39.722736 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.722677 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-workload-socket\") pod \"e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4\" (UID: \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\") " pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:38:39.724679 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.724652 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-istio-envoy\") pod \"e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4\" (UID: \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\") " pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:38:39.725019 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.724999 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-istio-podinfo\") pod \"e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4\" (UID: \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\") " pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:38:39.731066 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.731041 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-istio-token\") pod \"e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4\" (UID: \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\") " pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:38:39.731544 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.731527 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59mjj\" (UniqueName: \"kubernetes.io/projected/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-kube-api-access-59mjj\") pod \"e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4\" (UID: \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\") " pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:38:39.822183 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.822087 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:38:39.873503 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.873457 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:38:39.977142 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.977116 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4"] Jul 09 15:38:39.979411 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:38:39.979380 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cb8185d_8f9d_46c6_aa1f_e01c9b0101af.slice/crio-5802e205849ff8fbcb6b4ecbbf30adb3644daf220391ab2d8c3d0b4627fc708f WatchSource:0}: Error finding container 5802e205849ff8fbcb6b4ecbbf30adb3644daf220391ab2d8c3d0b4627fc708f: Status 404 returned error can't find the container with id 5802e205849ff8fbcb6b4ecbbf30adb3644daf220391ab2d8c3d0b4627fc708f Jul 09 15:38:39.981590 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.981557 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:38:39.981710 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.981627 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:38:39.981710 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:39.981654 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:38:40.161972 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:40.161884 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" event={"ID":"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af","Type":"ContainerStarted","Data":"0bf506cc88e5bfb3d3efe65a9f68f25d9fbc74467527912e00abf271576b5c7b"} Jul 09 15:38:40.161972 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:40.161939 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" event={"ID":"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af","Type":"ContainerStarted","Data":"5802e205849ff8fbcb6b4ecbbf30adb3644daf220391ab2d8c3d0b4627fc708f"} Jul 09 15:38:40.189161 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:40.189100 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" podStartSLOduration=1.189083545 podStartE2EDuration="1.189083545s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:38:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:38:40.186067708 +0000 UTC m=+2166.916574212" watchObservedRunningTime="2026-07-09 15:38:40.189083545 +0000 UTC m=+2166.919590035" Jul 09 15:38:40.822806 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:40.822771 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:38:41.830573 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:41.830542 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:38:42.173077 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:42.172986 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:38:42.173943 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:42.173919 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:38:42.227117 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:42.227048 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:38:42.349555 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:42.349519 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:38:42.686366 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:42.686328 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:38:42.814711 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:42.814666 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:38:43.287351 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:43.287314 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:38:43.402608 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:43.402569 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:38:43.819764 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:43.819740 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:38:43.866498 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:43.866399 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9567e15d-f443-47f0-ba63-d17896e08a0b-istio-data\") pod \"9567e15d-f443-47f0-ba63-d17896e08a0b\" (UID: \"9567e15d-f443-47f0-ba63-d17896e08a0b\") " Jul 09 15:38:43.866498 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:43.866443 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9567e15d-f443-47f0-ba63-d17896e08a0b-credential-socket\") pod \"9567e15d-f443-47f0-ba63-d17896e08a0b\" (UID: \"9567e15d-f443-47f0-ba63-d17896e08a0b\") " Jul 09 15:38:43.866498 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:43.866469 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9567e15d-f443-47f0-ba63-d17896e08a0b-istio-token\") pod \"9567e15d-f443-47f0-ba63-d17896e08a0b\" (UID: \"9567e15d-f443-47f0-ba63-d17896e08a0b\") " Jul 09 15:38:43.866498 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:43.866493 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9567e15d-f443-47f0-ba63-d17896e08a0b-istio-podinfo\") pod \"9567e15d-f443-47f0-ba63-d17896e08a0b\" (UID: \"9567e15d-f443-47f0-ba63-d17896e08a0b\") " Jul 09 15:38:43.866856 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:43.866528 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4sngm\" (UniqueName: \"kubernetes.io/projected/9567e15d-f443-47f0-ba63-d17896e08a0b-kube-api-access-4sngm\") pod \"9567e15d-f443-47f0-ba63-d17896e08a0b\" (UID: \"9567e15d-f443-47f0-ba63-d17896e08a0b\") " Jul 09 15:38:43.866856 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:43.866568 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9567e15d-f443-47f0-ba63-d17896e08a0b-istiod-ca-cert\") pod \"9567e15d-f443-47f0-ba63-d17896e08a0b\" (UID: \"9567e15d-f443-47f0-ba63-d17896e08a0b\") " Jul 09 15:38:43.866856 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:43.866656 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9567e15d-f443-47f0-ba63-d17896e08a0b-workload-certs\") pod \"9567e15d-f443-47f0-ba63-d17896e08a0b\" (UID: \"9567e15d-f443-47f0-ba63-d17896e08a0b\") " Jul 09 15:38:43.866856 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:43.866719 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9567e15d-f443-47f0-ba63-d17896e08a0b-workload-socket\") pod \"9567e15d-f443-47f0-ba63-d17896e08a0b\" (UID: \"9567e15d-f443-47f0-ba63-d17896e08a0b\") " Jul 09 15:38:43.866856 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:43.866750 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9567e15d-f443-47f0-ba63-d17896e08a0b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9567e15d-f443-47f0-ba63-d17896e08a0b" (UID: "9567e15d-f443-47f0-ba63-d17896e08a0b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:38:43.866856 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:43.866786 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9567e15d-f443-47f0-ba63-d17896e08a0b-istio-envoy\") pod \"9567e15d-f443-47f0-ba63-d17896e08a0b\" (UID: \"9567e15d-f443-47f0-ba63-d17896e08a0b\") " Jul 09 15:38:43.867621 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:43.866986 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9567e15d-f443-47f0-ba63-d17896e08a0b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9567e15d-f443-47f0-ba63-d17896e08a0b" (UID: "9567e15d-f443-47f0-ba63-d17896e08a0b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:38:43.867621 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:43.867073 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9567e15d-f443-47f0-ba63-d17896e08a0b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9567e15d-f443-47f0-ba63-d17896e08a0b" (UID: "9567e15d-f443-47f0-ba63-d17896e08a0b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:38:43.867621 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:43.867263 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9567e15d-f443-47f0-ba63-d17896e08a0b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9567e15d-f443-47f0-ba63-d17896e08a0b" (UID: "9567e15d-f443-47f0-ba63-d17896e08a0b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:38:43.867621 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:43.867332 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9567e15d-f443-47f0-ba63-d17896e08a0b-istiod-ca-cert\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:38:43.867621 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:43.867341 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9567e15d-f443-47f0-ba63-d17896e08a0b-istio-data" (OuterVolumeSpecName: "istio-data") pod "9567e15d-f443-47f0-ba63-d17896e08a0b" (UID: "9567e15d-f443-47f0-ba63-d17896e08a0b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:38:43.867621 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:43.867356 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9567e15d-f443-47f0-ba63-d17896e08a0b-workload-certs\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:38:43.867621 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:43.867371 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9567e15d-f443-47f0-ba63-d17896e08a0b-credential-socket\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:38:43.869152 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:43.869126 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9567e15d-f443-47f0-ba63-d17896e08a0b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9567e15d-f443-47f0-ba63-d17896e08a0b" (UID: "9567e15d-f443-47f0-ba63-d17896e08a0b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:38:43.869283 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:43.869261 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9567e15d-f443-47f0-ba63-d17896e08a0b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9567e15d-f443-47f0-ba63-d17896e08a0b" (UID: "9567e15d-f443-47f0-ba63-d17896e08a0b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 15:38:43.869358 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:43.869266 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9567e15d-f443-47f0-ba63-d17896e08a0b-kube-api-access-4sngm" (OuterVolumeSpecName: "kube-api-access-4sngm") pod "9567e15d-f443-47f0-ba63-d17896e08a0b" (UID: "9567e15d-f443-47f0-ba63-d17896e08a0b"). InnerVolumeSpecName "kube-api-access-4sngm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:38:43.869677 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:43.869660 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9567e15d-f443-47f0-ba63-d17896e08a0b-istio-token" (OuterVolumeSpecName: "istio-token") pod "9567e15d-f443-47f0-ba63-d17896e08a0b" (UID: "9567e15d-f443-47f0-ba63-d17896e08a0b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:38:43.968846 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:43.968781 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9567e15d-f443-47f0-ba63-d17896e08a0b-workload-socket\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:38:43.968846 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:43.968845 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9567e15d-f443-47f0-ba63-d17896e08a0b-istio-envoy\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:38:43.968846 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:43.968857 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9567e15d-f443-47f0-ba63-d17896e08a0b-istio-data\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:38:43.969083 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:43.968865 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9567e15d-f443-47f0-ba63-d17896e08a0b-istio-token\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:38:43.969083 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:43.968875 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9567e15d-f443-47f0-ba63-d17896e08a0b-istio-podinfo\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:38:43.969083 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:43.968884 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4sngm\" (UniqueName: \"kubernetes.io/projected/9567e15d-f443-47f0-ba63-d17896e08a0b-kube-api-access-4sngm\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:38:44.186871 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:44.186753 2567 generic.go:356] "Generic (PLEG): container finished" podID="9567e15d-f443-47f0-ba63-d17896e08a0b" containerID="30f8a4a7e422ea7ae9834320b028406959d0838ef33f7fbd0bd3cd58fa1af7f8" exitCode=0 Jul 09 15:38:44.186871 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:44.186805 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" Jul 09 15:38:44.186871 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:44.186854 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" event={"ID":"9567e15d-f443-47f0-ba63-d17896e08a0b","Type":"ContainerDied","Data":"30f8a4a7e422ea7ae9834320b028406959d0838ef33f7fbd0bd3cd58fa1af7f8"} Jul 09 15:38:44.187176 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:44.186924 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb" event={"ID":"9567e15d-f443-47f0-ba63-d17896e08a0b","Type":"ContainerDied","Data":"bf451a19c76d67a1905a88d71d0f25c5429452470bf00f5060f52e56619acfd1"} Jul 09 15:38:44.187176 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:44.186953 2567 scope.go:117] "RemoveContainer" containerID="30f8a4a7e422ea7ae9834320b028406959d0838ef33f7fbd0bd3cd58fa1af7f8" Jul 09 15:38:44.199134 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:44.199113 2567 scope.go:117] "RemoveContainer" containerID="e5b7277f8338bded3c16e7e7fba63bbed05f207c6a4934998716f9870d4e66c1" Jul 09 15:38:44.209948 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:44.209799 2567 scope.go:117] "RemoveContainer" containerID="30f8a4a7e422ea7ae9834320b028406959d0838ef33f7fbd0bd3cd58fa1af7f8" Jul 09 15:38:44.210263 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:38:44.210235 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30f8a4a7e422ea7ae9834320b028406959d0838ef33f7fbd0bd3cd58fa1af7f8\": container with ID starting with 30f8a4a7e422ea7ae9834320b028406959d0838ef33f7fbd0bd3cd58fa1af7f8 not found: ID does not exist" containerID="30f8a4a7e422ea7ae9834320b028406959d0838ef33f7fbd0bd3cd58fa1af7f8" Jul 09 15:38:44.210343 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:44.210283 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30f8a4a7e422ea7ae9834320b028406959d0838ef33f7fbd0bd3cd58fa1af7f8"} err="failed to get container status \"30f8a4a7e422ea7ae9834320b028406959d0838ef33f7fbd0bd3cd58fa1af7f8\": rpc error: code = NotFound desc = could not find container \"30f8a4a7e422ea7ae9834320b028406959d0838ef33f7fbd0bd3cd58fa1af7f8\": container with ID starting with 30f8a4a7e422ea7ae9834320b028406959d0838ef33f7fbd0bd3cd58fa1af7f8 not found: ID does not exist" Jul 09 15:38:44.210415 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:44.210347 2567 scope.go:117] "RemoveContainer" containerID="e5b7277f8338bded3c16e7e7fba63bbed05f207c6a4934998716f9870d4e66c1" Jul 09 15:38:44.210732 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:38:44.210709 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5b7277f8338bded3c16e7e7fba63bbed05f207c6a4934998716f9870d4e66c1\": container with ID starting with e5b7277f8338bded3c16e7e7fba63bbed05f207c6a4934998716f9870d4e66c1 not found: ID does not exist" containerID="e5b7277f8338bded3c16e7e7fba63bbed05f207c6a4934998716f9870d4e66c1" Jul 09 15:38:44.210841 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:44.210736 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5b7277f8338bded3c16e7e7fba63bbed05f207c6a4934998716f9870d4e66c1"} err="failed to get container status \"e5b7277f8338bded3c16e7e7fba63bbed05f207c6a4934998716f9870d4e66c1\": rpc error: code = NotFound desc = could not find container \"e5b7277f8338bded3c16e7e7fba63bbed05f207c6a4934998716f9870d4e66c1\": container with ID starting with e5b7277f8338bded3c16e7e7fba63bbed05f207c6a4934998716f9870d4e66c1 not found: ID does not exist" Jul 09 15:38:44.214330 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:44.214304 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb"] Jul 09 15:38:44.216655 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:44.216626 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-d62682-openshift-default-797687b87f-4qbtb"] Jul 09 15:38:44.751007 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:44.750973 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:38:44.850118 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:44.850077 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:38:45.830262 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:45.830227 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9567e15d-f443-47f0-ba63-d17896e08a0b" path="/var/lib/kubelet/pods/9567e15d-f443-47f0-ba63-d17896e08a0b/volumes" Jul 09 15:38:46.692354 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:46.692311 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:38:47.200914 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.200835 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:38:47.623402 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.623354 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj"] Jul 09 15:38:47.623934 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.623913 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9567e15d-f443-47f0-ba63-d17896e08a0b" containerName="istio-proxy" Jul 09 15:38:47.623934 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.623935 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9567e15d-f443-47f0-ba63-d17896e08a0b" containerName="istio-proxy" Jul 09 15:38:47.624107 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.623961 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9567e15d-f443-47f0-ba63-d17896e08a0b" containerName="istio-proxy" Jul 09 15:38:47.624107 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.623970 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9567e15d-f443-47f0-ba63-d17896e08a0b" containerName="istio-proxy" Jul 09 15:38:47.624107 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.624072 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9567e15d-f443-47f0-ba63-d17896e08a0b" containerName="istio-proxy" Jul 09 15:38:47.624107 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.624089 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9567e15d-f443-47f0-ba63-d17896e08a0b" containerName="istio-proxy" Jul 09 15:38:47.627438 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.627415 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:38:47.652629 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.652597 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-a65149-openshift-default-dockercfg-s6dfc\"" Jul 09 15:38:47.671759 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.671721 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj"] Jul 09 15:38:47.711570 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.711533 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3831594-36d5-4089-94b2-ea8c7aacc3dc-istio-data\") pod \"e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj\" (UID: \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\") " pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:38:47.711855 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.711575 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3831594-36d5-4089-94b2-ea8c7aacc3dc-workload-socket\") pod \"e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj\" (UID: \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\") " pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:38:47.711855 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.711604 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3831594-36d5-4089-94b2-ea8c7aacc3dc-istio-token\") pod \"e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj\" (UID: \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\") " pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:38:47.711855 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.711681 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3831594-36d5-4089-94b2-ea8c7aacc3dc-workload-certs\") pod \"e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj\" (UID: \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\") " pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:38:47.711855 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.711752 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3831594-36d5-4089-94b2-ea8c7aacc3dc-credential-socket\") pod \"e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj\" (UID: \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\") " pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:38:47.712106 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.711902 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3831594-36d5-4089-94b2-ea8c7aacc3dc-istio-podinfo\") pod \"e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj\" (UID: \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\") " pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:38:47.712106 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.711938 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3831594-36d5-4089-94b2-ea8c7aacc3dc-istio-envoy\") pod \"e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj\" (UID: \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\") " pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:38:47.712106 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.711965 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3831594-36d5-4089-94b2-ea8c7aacc3dc-istiod-ca-cert\") pod \"e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj\" (UID: \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\") " pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:38:47.712106 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.712000 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glpns\" (UniqueName: \"kubernetes.io/projected/e3831594-36d5-4089-94b2-ea8c7aacc3dc-kube-api-access-glpns\") pod \"e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj\" (UID: \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\") " pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:38:47.779074 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.779038 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:38:47.813328 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.813285 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3831594-36d5-4089-94b2-ea8c7aacc3dc-istio-podinfo\") pod \"e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj\" (UID: \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\") " pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:38:47.813328 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.813334 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3831594-36d5-4089-94b2-ea8c7aacc3dc-istio-envoy\") pod \"e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj\" (UID: \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\") " pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:38:47.813328 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.813351 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3831594-36d5-4089-94b2-ea8c7aacc3dc-istiod-ca-cert\") pod \"e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj\" (UID: \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\") " pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:38:47.813652 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.813374 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glpns\" (UniqueName: \"kubernetes.io/projected/e3831594-36d5-4089-94b2-ea8c7aacc3dc-kube-api-access-glpns\") pod \"e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj\" (UID: \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\") " pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:38:47.813652 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.813435 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3831594-36d5-4089-94b2-ea8c7aacc3dc-istio-data\") pod \"e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj\" (UID: \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\") " pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:38:47.813652 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.813461 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3831594-36d5-4089-94b2-ea8c7aacc3dc-workload-socket\") pod \"e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj\" (UID: \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\") " pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:38:47.813652 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.813597 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3831594-36d5-4089-94b2-ea8c7aacc3dc-istio-token\") pod \"e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj\" (UID: \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\") " pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:38:47.813652 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.813644 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3831594-36d5-4089-94b2-ea8c7aacc3dc-workload-certs\") pod \"e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj\" (UID: \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\") " pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:38:47.813938 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.813712 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3831594-36d5-4089-94b2-ea8c7aacc3dc-credential-socket\") pod \"e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj\" (UID: \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\") " pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:38:47.813938 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.813864 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3831594-36d5-4089-94b2-ea8c7aacc3dc-workload-socket\") pod \"e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj\" (UID: \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\") " pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:38:47.814138 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.814113 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3831594-36d5-4089-94b2-ea8c7aacc3dc-workload-certs\") pod \"e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj\" (UID: \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\") " pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:38:47.814194 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.814151 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3831594-36d5-4089-94b2-ea8c7aacc3dc-credential-socket\") pod \"e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj\" (UID: \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\") " pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:38:47.814194 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.814183 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3831594-36d5-4089-94b2-ea8c7aacc3dc-istiod-ca-cert\") pod \"e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj\" (UID: \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\") " pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:38:47.814353 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.814336 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3831594-36d5-4089-94b2-ea8c7aacc3dc-istio-data\") pod \"e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj\" (UID: \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\") " pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:38:47.815886 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.815862 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3831594-36d5-4089-94b2-ea8c7aacc3dc-istio-envoy\") pod \"e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj\" (UID: \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\") " pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:38:47.816416 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.816397 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3831594-36d5-4089-94b2-ea8c7aacc3dc-istio-podinfo\") pod \"e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj\" (UID: \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\") " pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:38:47.826866 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.826800 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3831594-36d5-4089-94b2-ea8c7aacc3dc-istio-token\") pod \"e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj\" (UID: \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\") " pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:38:47.827107 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.826962 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glpns\" (UniqueName: \"kubernetes.io/projected/e3831594-36d5-4089-94b2-ea8c7aacc3dc-kube-api-access-glpns\") pod \"e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj\" (UID: \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\") " pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:38:47.891975 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.891859 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:38:47.938883 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:47.938839 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:38:48.101878 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:48.101847 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj"] Jul 09 15:38:48.104205 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:38:48.104180 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3831594_36d5_4089_94b2_ea8c7aacc3dc.slice/crio-d973924ddd08e57302d73247e8838ceae3072d20f9decb67bcb37eca35afa809 WatchSource:0}: Error finding container d973924ddd08e57302d73247e8838ceae3072d20f9decb67bcb37eca35afa809: Status 404 returned error can't find the container with id d973924ddd08e57302d73247e8838ceae3072d20f9decb67bcb37eca35afa809 Jul 09 15:38:48.106907 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:48.106863 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:38:48.107005 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:48.106968 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:38:48.107048 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:48.107017 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:38:48.214072 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:48.214031 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" event={"ID":"e3831594-36d5-4089-94b2-ea8c7aacc3dc","Type":"ContainerStarted","Data":"34067c9a4564f6bcada47623d1da2f50c09515b34718e59f96dadb1bf553a402"} Jul 09 15:38:48.214507 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:48.214096 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" event={"ID":"e3831594-36d5-4089-94b2-ea8c7aacc3dc","Type":"ContainerStarted","Data":"d973924ddd08e57302d73247e8838ceae3072d20f9decb67bcb37eca35afa809"} Jul 09 15:38:48.239282 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:48.239214 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" podStartSLOduration=1.239191493 podStartE2EDuration="1.239191493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:38:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:38:48.238102038 +0000 UTC m=+2174.968608529" watchObservedRunningTime="2026-07-09 15:38:48.239191493 +0000 UTC m=+2174.969697985" Jul 09 15:38:48.939336 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:48.939296 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:38:49.944580 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:49.944549 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:38:50.225447 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:50.225407 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:38:50.226354 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:50.226336 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:38:50.264761 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:50.264724 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:38:50.504801 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:50.504703 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:38:50.600044 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:50.600002 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:38:51.251001 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:51.250953 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:38:51.361141 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:51.361098 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:38:51.494854 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:51.494796 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:38:54.712916 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:54.712875 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:38:58.214004 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:58.213955 2567 patch_prober.go:28] interesting pod/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 09 15:38:58.214439 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:38:58.214036 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" podUID="6cb8185d-8f9d-46c6-aa1f-e01c9b0101af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 09 15:39:00.282490 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:00.282457 2567 generic.go:356] "Generic (PLEG): container finished" podID="6cb8185d-8f9d-46c6-aa1f-e01c9b0101af" containerID="0bf506cc88e5bfb3d3efe65a9f68f25d9fbc74467527912e00abf271576b5c7b" exitCode=0 Jul 09 15:39:00.282943 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:00.282526 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" event={"ID":"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af","Type":"ContainerDied","Data":"0bf506cc88e5bfb3d3efe65a9f68f25d9fbc74467527912e00abf271576b5c7b"} Jul 09 15:39:00.283004 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:00.282973 2567 scope.go:117] "RemoveContainer" containerID="0bf506cc88e5bfb3d3efe65a9f68f25d9fbc74467527912e00abf271576b5c7b" Jul 09 15:39:00.284187 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:00.284159 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:39:00.284256 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:00.284241 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:39:00.284293 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:00.284279 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:39:01.290179 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:01.290147 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" event={"ID":"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af","Type":"ContainerStarted","Data":"0e21956285e5ca4beeea265664203826e746d4db9ed575da4ae7dc8d98a43db7"} Jul 09 15:39:01.822708 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:01.822665 2567 patch_prober.go:28] interesting pod/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 09 15:39:01.822920 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:01.822743 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" podUID="6cb8185d-8f9d-46c6-aa1f-e01c9b0101af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 09 15:39:01.828987 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:01.828957 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:39:02.823374 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:02.823332 2567 patch_prober.go:28] interesting pod/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 09 15:39:02.824026 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:02.823990 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" podUID="6cb8185d-8f9d-46c6-aa1f-e01c9b0101af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 09 15:39:03.097905 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:03.095954 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:03.571887 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:03.571842 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:03.823349 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:03.823239 2567 patch_prober.go:28] interesting pod/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 09 15:39:03.823662 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:03.823608 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" podUID="6cb8185d-8f9d-46c6-aa1f-e01c9b0101af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 09 15:39:03.956996 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:03.956958 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:04.132132 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:04.132028 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:04.351524 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:04.351486 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:04.823315 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:04.823269 2567 patch_prober.go:28] interesting pod/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 09 15:39:04.824882 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:04.824377 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" podUID="6cb8185d-8f9d-46c6-aa1f-e01c9b0101af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 09 15:39:05.829262 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:05.829235 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:39:05.829667 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:05.829488 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:39:05.830387 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:05.830369 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:39:10.358985 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:10.358951 2567 generic.go:356] "Generic (PLEG): container finished" podID="e3831594-36d5-4089-94b2-ea8c7aacc3dc" containerID="34067c9a4564f6bcada47623d1da2f50c09515b34718e59f96dadb1bf553a402" exitCode=0 Jul 09 15:39:10.359492 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:10.359028 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" event={"ID":"e3831594-36d5-4089-94b2-ea8c7aacc3dc","Type":"ContainerDied","Data":"34067c9a4564f6bcada47623d1da2f50c09515b34718e59f96dadb1bf553a402"} Jul 09 15:39:10.359492 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:10.359454 2567 scope.go:117] "RemoveContainer" containerID="34067c9a4564f6bcada47623d1da2f50c09515b34718e59f96dadb1bf553a402" Jul 09 15:39:10.360508 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:10.360484 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:39:10.360565 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:10.360557 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:39:10.360606 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:10.360589 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:39:10.741607 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:10.741555 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:10.840208 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:10.840164 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:11.036462 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:11.036373 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4"] Jul 09 15:39:11.036635 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:11.036596 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" podUID="6cb8185d-8f9d-46c6-aa1f-e01c9b0101af" containerName="istio-proxy" containerID="cri-o://0e21956285e5ca4beeea265664203826e746d4db9ed575da4ae7dc8d98a43db7" gracePeriod=30 Jul 09 15:39:11.074860 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:11.074791 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:11.366325 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:11.366227 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" event={"ID":"e3831594-36d5-4089-94b2-ea8c7aacc3dc","Type":"ContainerStarted","Data":"1ed28797c6fe987c3eb806f08110ea024a5ddb692a6129c746b18c946c8c0f10"} Jul 09 15:39:11.590388 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:11.590315 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:11.939988 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:11.939952 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:39:11.940272 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:11.940246 2567 patch_prober.go:28] interesting pod/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 09 15:39:11.940325 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:11.940302 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" podUID="e3831594-36d5-4089-94b2-ea8c7aacc3dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 09 15:39:12.031505 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:12.031464 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:12.144842 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:12.144782 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:12.250145 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:12.250102 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:12.548014 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:12.547923 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:12.939954 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:12.939862 2567 patch_prober.go:28] interesting pod/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 09 15:39:12.939954 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:12.939922 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" podUID="e3831594-36d5-4089-94b2-ea8c7aacc3dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 09 15:39:13.939240 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:13.939193 2567 patch_prober.go:28] interesting pod/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 09 15:39:13.939685 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:13.939257 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" podUID="e3831594-36d5-4089-94b2-ea8c7aacc3dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 09 15:39:14.940174 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:14.940140 2567 patch_prober.go:28] interesting pod/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 09 15:39:14.940579 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:14.940200 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" podUID="e3831594-36d5-4089-94b2-ea8c7aacc3dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 09 15:39:15.943708 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:15.943672 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:39:15.944165 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:15.944065 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:39:15.944913 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:15.944893 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:39:16.297261 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.297237 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:39:16.395408 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.395359 2567 generic.go:356] "Generic (PLEG): container finished" podID="6cb8185d-8f9d-46c6-aa1f-e01c9b0101af" containerID="0e21956285e5ca4beeea265664203826e746d4db9ed575da4ae7dc8d98a43db7" exitCode=0 Jul 09 15:39:16.395408 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.395409 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" Jul 09 15:39:16.395626 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.395433 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" event={"ID":"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af","Type":"ContainerDied","Data":"0e21956285e5ca4beeea265664203826e746d4db9ed575da4ae7dc8d98a43db7"} Jul 09 15:39:16.395626 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.395497 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4" event={"ID":"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af","Type":"ContainerDied","Data":"5802e205849ff8fbcb6b4ecbbf30adb3644daf220391ab2d8c3d0b4627fc708f"} Jul 09 15:39:16.395626 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.395531 2567 scope.go:117] "RemoveContainer" containerID="0e21956285e5ca4beeea265664203826e746d4db9ed575da4ae7dc8d98a43db7" Jul 09 15:39:16.409678 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.409642 2567 scope.go:117] "RemoveContainer" containerID="0bf506cc88e5bfb3d3efe65a9f68f25d9fbc74467527912e00abf271576b5c7b" Jul 09 15:39:16.420138 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.420109 2567 scope.go:117] "RemoveContainer" containerID="0e21956285e5ca4beeea265664203826e746d4db9ed575da4ae7dc8d98a43db7" Jul 09 15:39:16.420500 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:39:16.420476 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e21956285e5ca4beeea265664203826e746d4db9ed575da4ae7dc8d98a43db7\": container with ID starting with 0e21956285e5ca4beeea265664203826e746d4db9ed575da4ae7dc8d98a43db7 not found: ID does not exist" containerID="0e21956285e5ca4beeea265664203826e746d4db9ed575da4ae7dc8d98a43db7" Jul 09 15:39:16.420611 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.420509 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e21956285e5ca4beeea265664203826e746d4db9ed575da4ae7dc8d98a43db7"} err="failed to get container status \"0e21956285e5ca4beeea265664203826e746d4db9ed575da4ae7dc8d98a43db7\": rpc error: code = NotFound desc = could not find container \"0e21956285e5ca4beeea265664203826e746d4db9ed575da4ae7dc8d98a43db7\": container with ID starting with 0e21956285e5ca4beeea265664203826e746d4db9ed575da4ae7dc8d98a43db7 not found: ID does not exist" Jul 09 15:39:16.420611 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.420528 2567 scope.go:117] "RemoveContainer" containerID="0bf506cc88e5bfb3d3efe65a9f68f25d9fbc74467527912e00abf271576b5c7b" Jul 09 15:39:16.420863 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:39:16.420810 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bf506cc88e5bfb3d3efe65a9f68f25d9fbc74467527912e00abf271576b5c7b\": container with ID starting with 0bf506cc88e5bfb3d3efe65a9f68f25d9fbc74467527912e00abf271576b5c7b not found: ID does not exist" containerID="0bf506cc88e5bfb3d3efe65a9f68f25d9fbc74467527912e00abf271576b5c7b" Jul 09 15:39:16.420951 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.420875 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bf506cc88e5bfb3d3efe65a9f68f25d9fbc74467527912e00abf271576b5c7b"} err="failed to get container status \"0bf506cc88e5bfb3d3efe65a9f68f25d9fbc74467527912e00abf271576b5c7b\": rpc error: code = NotFound desc = could not find container \"0bf506cc88e5bfb3d3efe65a9f68f25d9fbc74467527912e00abf271576b5c7b\": container with ID starting with 0bf506cc88e5bfb3d3efe65a9f68f25d9fbc74467527912e00abf271576b5c7b not found: ID does not exist" Jul 09 15:39:16.445271 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.445235 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-istio-token\") pod \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\" (UID: \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\") " Jul 09 15:39:16.445271 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.445277 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-59mjj\" (UniqueName: \"kubernetes.io/projected/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-kube-api-access-59mjj\") pod \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\" (UID: \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\") " Jul 09 15:39:16.445539 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.445320 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-istio-podinfo\") pod \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\" (UID: \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\") " Jul 09 15:39:16.445539 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.445337 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-istiod-ca-cert\") pod \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\" (UID: \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\") " Jul 09 15:39:16.445539 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.445368 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-workload-certs\") pod \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\" (UID: \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\") " Jul 09 15:39:16.445539 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.445412 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-credential-socket\") pod \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\" (UID: \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\") " Jul 09 15:39:16.445539 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.445468 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-istio-envoy\") pod \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\" (UID: \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\") " Jul 09 15:39:16.445539 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.445496 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-istio-data\") pod \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\" (UID: \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\") " Jul 09 15:39:16.445882 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.445545 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-workload-socket\") pod \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\" (UID: \"6cb8185d-8f9d-46c6-aa1f-e01c9b0101af\") " Jul 09 15:39:16.445882 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.445785 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6cb8185d-8f9d-46c6-aa1f-e01c9b0101af" (UID: "6cb8185d-8f9d-46c6-aa1f-e01c9b0101af"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:39:16.445882 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.445834 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6cb8185d-8f9d-46c6-aa1f-e01c9b0101af" (UID: "6cb8185d-8f9d-46c6-aa1f-e01c9b0101af"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:39:16.446041 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.446024 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-workload-certs\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:39:16.446041 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.446030 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6cb8185d-8f9d-46c6-aa1f-e01c9b0101af" (UID: "6cb8185d-8f9d-46c6-aa1f-e01c9b0101af"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:39:16.446159 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.446048 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-credential-socket\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:39:16.446287 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.446259 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6cb8185d-8f9d-46c6-aa1f-e01c9b0101af" (UID: "6cb8185d-8f9d-46c6-aa1f-e01c9b0101af"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:39:16.446545 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.446512 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-istio-data" (OuterVolumeSpecName: "istio-data") pod "6cb8185d-8f9d-46c6-aa1f-e01c9b0101af" (UID: "6cb8185d-8f9d-46c6-aa1f-e01c9b0101af"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:39:16.447656 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.447636 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-kube-api-access-59mjj" (OuterVolumeSpecName: "kube-api-access-59mjj") pod "6cb8185d-8f9d-46c6-aa1f-e01c9b0101af" (UID: "6cb8185d-8f9d-46c6-aa1f-e01c9b0101af"). InnerVolumeSpecName "kube-api-access-59mjj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:39:16.447862 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.447843 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-istio-token" (OuterVolumeSpecName: "istio-token") pod "6cb8185d-8f9d-46c6-aa1f-e01c9b0101af" (UID: "6cb8185d-8f9d-46c6-aa1f-e01c9b0101af"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:39:16.447981 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.447967 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6cb8185d-8f9d-46c6-aa1f-e01c9b0101af" (UID: "6cb8185d-8f9d-46c6-aa1f-e01c9b0101af"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 15:39:16.448071 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.448052 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6cb8185d-8f9d-46c6-aa1f-e01c9b0101af" (UID: "6cb8185d-8f9d-46c6-aa1f-e01c9b0101af"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:39:16.546793 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.546674 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-istio-envoy\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:39:16.546793 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.546709 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-istio-data\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:39:16.546793 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.546718 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-workload-socket\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:39:16.546793 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.546729 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-istio-token\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:39:16.546793 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.546742 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59mjj\" (UniqueName: \"kubernetes.io/projected/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-kube-api-access-59mjj\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:39:16.546793 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.546754 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-istio-podinfo\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:39:16.546793 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.546767 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af-istiod-ca-cert\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:39:16.737111 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.737077 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4"] Jul 09 15:39:16.743095 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:16.743066 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-0164f1-openshift-default-845dcfdf8c-2wxv4"] Jul 09 15:39:17.829947 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:17.829913 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6cb8185d-8f9d-46c6-aa1f-e01c9b0101af" path="/var/lib/kubelet/pods/6cb8185d-8f9d-46c6-aa1f-e01c9b0101af/volumes" Jul 09 15:39:18.598953 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:18.598909 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:18.690295 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:18.690248 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:18.881197 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:18.881098 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj"] Jul 09 15:39:18.881620 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:18.881314 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" podUID="e3831594-36d5-4089-94b2-ea8c7aacc3dc" containerName="istio-proxy" containerID="cri-o://1ed28797c6fe987c3eb806f08110ea024a5ddb692a6129c746b18c946c8c0f10" gracePeriod=30 Jul 09 15:39:18.906559 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:18.906526 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:19.476563 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:19.476521 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:19.784545 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:19.784466 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k"] Jul 09 15:39:19.784914 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:19.784900 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cb8185d-8f9d-46c6-aa1f-e01c9b0101af" containerName="istio-proxy" Jul 09 15:39:19.784966 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:19.784914 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cb8185d-8f9d-46c6-aa1f-e01c9b0101af" containerName="istio-proxy" Jul 09 15:39:19.785018 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:19.785009 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cb8185d-8f9d-46c6-aa1f-e01c9b0101af" containerName="istio-proxy" Jul 09 15:39:19.785055 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:19.785019 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cb8185d-8f9d-46c6-aa1f-e01c9b0101af" containerName="istio-proxy" Jul 09 15:39:19.788180 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:19.788160 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:39:19.791307 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:19.791277 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-dcc4b5-openshift-default-dockercfg-gsnl2\"" Jul 09 15:39:19.808949 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:19.808917 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k"] Jul 09 15:39:19.827470 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:19.827433 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:19.982388 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:19.982334 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf5b03dd-989b-440e-aea3-5bce8bc0598d-workload-certs\") pod \"e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k\" (UID: \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\") " pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:39:19.982388 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:19.982388 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf5b03dd-989b-440e-aea3-5bce8bc0598d-workload-socket\") pod \"e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k\" (UID: \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\") " pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:39:19.983129 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:19.982509 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf5b03dd-989b-440e-aea3-5bce8bc0598d-istio-data\") pod \"e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k\" (UID: \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\") " pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:39:19.983129 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:19.982558 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf5b03dd-989b-440e-aea3-5bce8bc0598d-credential-socket\") pod \"e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k\" (UID: \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\") " pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:39:19.983129 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:19.982658 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf5b03dd-989b-440e-aea3-5bce8bc0598d-istio-podinfo\") pod \"e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k\" (UID: \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\") " pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:39:19.983129 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:19.982779 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf5b03dd-989b-440e-aea3-5bce8bc0598d-istio-envoy\") pod \"e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k\" (UID: \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\") " pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:39:19.983129 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:19.982927 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf5b03dd-989b-440e-aea3-5bce8bc0598d-istio-token\") pod \"e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k\" (UID: \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\") " pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:39:19.983129 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:19.982973 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf5b03dd-989b-440e-aea3-5bce8bc0598d-istiod-ca-cert\") pod \"e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k\" (UID: \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\") " pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:39:19.983129 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:19.982993 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4f2s8\" (UniqueName: \"kubernetes.io/projected/cf5b03dd-989b-440e-aea3-5bce8bc0598d-kube-api-access-4f2s8\") pod \"e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k\" (UID: \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\") " pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:39:20.083860 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:20.083733 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf5b03dd-989b-440e-aea3-5bce8bc0598d-credential-socket\") pod \"e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k\" (UID: \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\") " pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:39:20.083860 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:20.083854 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf5b03dd-989b-440e-aea3-5bce8bc0598d-istio-podinfo\") pod \"e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k\" (UID: \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\") " pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:39:20.084107 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:20.083904 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf5b03dd-989b-440e-aea3-5bce8bc0598d-istio-envoy\") pod \"e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k\" (UID: \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\") " pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:39:20.084107 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:20.083976 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf5b03dd-989b-440e-aea3-5bce8bc0598d-istio-token\") pod \"e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k\" (UID: \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\") " pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:39:20.084107 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:20.084020 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf5b03dd-989b-440e-aea3-5bce8bc0598d-istiod-ca-cert\") pod \"e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k\" (UID: \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\") " pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:39:20.084107 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:20.084047 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4f2s8\" (UniqueName: \"kubernetes.io/projected/cf5b03dd-989b-440e-aea3-5bce8bc0598d-kube-api-access-4f2s8\") pod \"e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k\" (UID: \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\") " pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:39:20.084340 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:20.084115 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf5b03dd-989b-440e-aea3-5bce8bc0598d-workload-certs\") pod \"e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k\" (UID: \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\") " pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:39:20.084340 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:20.084140 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf5b03dd-989b-440e-aea3-5bce8bc0598d-workload-socket\") pod \"e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k\" (UID: \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\") " pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:39:20.084340 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:20.084185 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf5b03dd-989b-440e-aea3-5bce8bc0598d-istio-data\") pod \"e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k\" (UID: \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\") " pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:39:20.084499 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:20.084476 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf5b03dd-989b-440e-aea3-5bce8bc0598d-workload-certs\") pod \"e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k\" (UID: \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\") " pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:39:20.084556 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:20.084531 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf5b03dd-989b-440e-aea3-5bce8bc0598d-istio-data\") pod \"e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k\" (UID: \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\") " pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:39:20.084614 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:20.084587 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf5b03dd-989b-440e-aea3-5bce8bc0598d-workload-socket\") pod \"e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k\" (UID: \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\") " pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:39:20.084614 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:20.084596 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf5b03dd-989b-440e-aea3-5bce8bc0598d-credential-socket\") pod \"e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k\" (UID: \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\") " pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:39:20.084876 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:20.084854 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf5b03dd-989b-440e-aea3-5bce8bc0598d-istiod-ca-cert\") pod \"e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k\" (UID: \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\") " pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:39:20.086272 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:20.086245 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf5b03dd-989b-440e-aea3-5bce8bc0598d-istio-envoy\") pod \"e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k\" (UID: \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\") " pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:39:20.086376 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:20.086363 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf5b03dd-989b-440e-aea3-5bce8bc0598d-istio-podinfo\") pod \"e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k\" (UID: \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\") " pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:39:20.094346 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:20.094309 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf5b03dd-989b-440e-aea3-5bce8bc0598d-istio-token\") pod \"e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k\" (UID: \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\") " pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:39:20.094592 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:20.094570 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4f2s8\" (UniqueName: \"kubernetes.io/projected/cf5b03dd-989b-440e-aea3-5bce8bc0598d-kube-api-access-4f2s8\") pod \"e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k\" (UID: \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\") " pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:39:20.100316 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:20.100291 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:39:20.248203 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:20.248171 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k"] Jul 09 15:39:20.249950 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:39:20.249925 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf5b03dd_989b_440e_aea3_5bce8bc0598d.slice/crio-259759156aae85e7ca9a1924eae7b909348b380b8803fe46cf0575ab857da7d9 WatchSource:0}: Error finding container 259759156aae85e7ca9a1924eae7b909348b380b8803fe46cf0575ab857da7d9: Status 404 returned error can't find the container with id 259759156aae85e7ca9a1924eae7b909348b380b8803fe46cf0575ab857da7d9 Jul 09 15:39:20.252172 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:20.252136 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:39:20.252257 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:20.252230 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:39:20.252296 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:20.252280 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:39:20.370569 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:20.370472 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:20.424269 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:20.424229 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" event={"ID":"cf5b03dd-989b-440e-aea3-5bce8bc0598d","Type":"ContainerStarted","Data":"bec4cbba80cc33ff866f5be3580c47018ecf71137f8c69f86ebea68075e90c50"} Jul 09 15:39:20.424475 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:20.424296 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" event={"ID":"cf5b03dd-989b-440e-aea3-5bce8bc0598d","Type":"ContainerStarted","Data":"259759156aae85e7ca9a1924eae7b909348b380b8803fe46cf0575ab857da7d9"} Jul 09 15:39:20.447118 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:20.447053 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" podStartSLOduration=1.44703012 podStartE2EDuration="1.44703012s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:39:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:39:20.44666285 +0000 UTC m=+2207.177169342" watchObservedRunningTime="2026-07-09 15:39:20.44703012 +0000 UTC m=+2207.177536611" Jul 09 15:39:21.101086 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:21.101046 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:39:21.106728 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:21.106695 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:39:21.429128 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:21.429043 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:39:21.429987 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:21.429961 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:39:22.455366 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:22.455328 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:22.589331 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:22.589293 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:22.694557 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:22.694525 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:22.957651 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:22.957619 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:23.414925 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:23.414798 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:23.535278 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:23.535248 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:24.150115 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.150047 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:39:24.231367 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.231329 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3831594-36d5-4089-94b2-ea8c7aacc3dc-istio-token\") pod \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\" (UID: \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\") " Jul 09 15:39:24.231573 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.231394 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3831594-36d5-4089-94b2-ea8c7aacc3dc-workload-certs\") pod \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\" (UID: \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\") " Jul 09 15:39:24.231573 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.231414 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3831594-36d5-4089-94b2-ea8c7aacc3dc-workload-socket\") pod \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\" (UID: \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\") " Jul 09 15:39:24.231573 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.231436 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-glpns\" (UniqueName: \"kubernetes.io/projected/e3831594-36d5-4089-94b2-ea8c7aacc3dc-kube-api-access-glpns\") pod \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\" (UID: \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\") " Jul 09 15:39:24.231573 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.231494 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3831594-36d5-4089-94b2-ea8c7aacc3dc-credential-socket\") pod \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\" (UID: \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\") " Jul 09 15:39:24.231573 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.231511 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3831594-36d5-4089-94b2-ea8c7aacc3dc-istiod-ca-cert\") pod \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\" (UID: \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\") " Jul 09 15:39:24.231875 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.231542 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3831594-36d5-4089-94b2-ea8c7aacc3dc-istio-data\") pod \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\" (UID: \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\") " Jul 09 15:39:24.231875 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.231646 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3831594-36d5-4089-94b2-ea8c7aacc3dc-istio-envoy\") pod \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\" (UID: \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\") " Jul 09 15:39:24.231875 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.231700 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3831594-36d5-4089-94b2-ea8c7aacc3dc-istio-podinfo\") pod \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\" (UID: \"e3831594-36d5-4089-94b2-ea8c7aacc3dc\") " Jul 09 15:39:24.231875 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.231692 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3831594-36d5-4089-94b2-ea8c7aacc3dc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e3831594-36d5-4089-94b2-ea8c7aacc3dc" (UID: "e3831594-36d5-4089-94b2-ea8c7aacc3dc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:39:24.231875 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.231772 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3831594-36d5-4089-94b2-ea8c7aacc3dc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e3831594-36d5-4089-94b2-ea8c7aacc3dc" (UID: "e3831594-36d5-4089-94b2-ea8c7aacc3dc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:39:24.232222 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.231874 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3831594-36d5-4089-94b2-ea8c7aacc3dc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e3831594-36d5-4089-94b2-ea8c7aacc3dc" (UID: "e3831594-36d5-4089-94b2-ea8c7aacc3dc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:39:24.232222 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.232001 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3831594-36d5-4089-94b2-ea8c7aacc3dc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e3831594-36d5-4089-94b2-ea8c7aacc3dc" (UID: "e3831594-36d5-4089-94b2-ea8c7aacc3dc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:39:24.232222 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.232175 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3831594-36d5-4089-94b2-ea8c7aacc3dc-workload-certs\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:39:24.232222 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.232192 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3831594-36d5-4089-94b2-ea8c7aacc3dc-workload-socket\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:39:24.232222 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.232202 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3831594-36d5-4089-94b2-ea8c7aacc3dc-credential-socket\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:39:24.232222 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.232211 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3831594-36d5-4089-94b2-ea8c7aacc3dc-istiod-ca-cert\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:39:24.232676 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.232649 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3831594-36d5-4089-94b2-ea8c7aacc3dc-istio-data" (OuterVolumeSpecName: "istio-data") pod "e3831594-36d5-4089-94b2-ea8c7aacc3dc" (UID: "e3831594-36d5-4089-94b2-ea8c7aacc3dc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:39:24.233850 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.233810 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3831594-36d5-4089-94b2-ea8c7aacc3dc-istio-token" (OuterVolumeSpecName: "istio-token") pod "e3831594-36d5-4089-94b2-ea8c7aacc3dc" (UID: "e3831594-36d5-4089-94b2-ea8c7aacc3dc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:39:24.234111 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.234091 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3831594-36d5-4089-94b2-ea8c7aacc3dc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e3831594-36d5-4089-94b2-ea8c7aacc3dc" (UID: "e3831594-36d5-4089-94b2-ea8c7aacc3dc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:39:24.234308 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.234281 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e3831594-36d5-4089-94b2-ea8c7aacc3dc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e3831594-36d5-4089-94b2-ea8c7aacc3dc" (UID: "e3831594-36d5-4089-94b2-ea8c7aacc3dc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 15:39:24.234406 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.234314 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3831594-36d5-4089-94b2-ea8c7aacc3dc-kube-api-access-glpns" (OuterVolumeSpecName: "kube-api-access-glpns") pod "e3831594-36d5-4089-94b2-ea8c7aacc3dc" (UID: "e3831594-36d5-4089-94b2-ea8c7aacc3dc"). InnerVolumeSpecName "kube-api-access-glpns". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:39:24.333176 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.333138 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3831594-36d5-4089-94b2-ea8c7aacc3dc-istio-data\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:39:24.333176 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.333167 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3831594-36d5-4089-94b2-ea8c7aacc3dc-istio-envoy\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:39:24.333176 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.333176 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3831594-36d5-4089-94b2-ea8c7aacc3dc-istio-podinfo\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:39:24.333176 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.333184 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3831594-36d5-4089-94b2-ea8c7aacc3dc-istio-token\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:39:24.333448 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.333194 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glpns\" (UniqueName: \"kubernetes.io/projected/e3831594-36d5-4089-94b2-ea8c7aacc3dc-kube-api-access-glpns\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:39:24.447111 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.447018 2567 generic.go:356] "Generic (PLEG): container finished" podID="e3831594-36d5-4089-94b2-ea8c7aacc3dc" containerID="1ed28797c6fe987c3eb806f08110ea024a5ddb692a6129c746b18c946c8c0f10" exitCode=0 Jul 09 15:39:24.447111 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.447069 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" event={"ID":"e3831594-36d5-4089-94b2-ea8c7aacc3dc","Type":"ContainerDied","Data":"1ed28797c6fe987c3eb806f08110ea024a5ddb692a6129c746b18c946c8c0f10"} Jul 09 15:39:24.447309 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.447113 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" event={"ID":"e3831594-36d5-4089-94b2-ea8c7aacc3dc","Type":"ContainerDied","Data":"d973924ddd08e57302d73247e8838ceae3072d20f9decb67bcb37eca35afa809"} Jul 09 15:39:24.447309 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.447136 2567 scope.go:117] "RemoveContainer" containerID="1ed28797c6fe987c3eb806f08110ea024a5ddb692a6129c746b18c946c8c0f10" Jul 09 15:39:24.447309 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.447070 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj" Jul 09 15:39:24.457530 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.457504 2567 scope.go:117] "RemoveContainer" containerID="34067c9a4564f6bcada47623d1da2f50c09515b34718e59f96dadb1bf553a402" Jul 09 15:39:24.467443 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.467414 2567 scope.go:117] "RemoveContainer" containerID="1ed28797c6fe987c3eb806f08110ea024a5ddb692a6129c746b18c946c8c0f10" Jul 09 15:39:24.467763 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:39:24.467739 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ed28797c6fe987c3eb806f08110ea024a5ddb692a6129c746b18c946c8c0f10\": container with ID starting with 1ed28797c6fe987c3eb806f08110ea024a5ddb692a6129c746b18c946c8c0f10 not found: ID does not exist" containerID="1ed28797c6fe987c3eb806f08110ea024a5ddb692a6129c746b18c946c8c0f10" Jul 09 15:39:24.467865 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.467773 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ed28797c6fe987c3eb806f08110ea024a5ddb692a6129c746b18c946c8c0f10"} err="failed to get container status \"1ed28797c6fe987c3eb806f08110ea024a5ddb692a6129c746b18c946c8c0f10\": rpc error: code = NotFound desc = could not find container \"1ed28797c6fe987c3eb806f08110ea024a5ddb692a6129c746b18c946c8c0f10\": container with ID starting with 1ed28797c6fe987c3eb806f08110ea024a5ddb692a6129c746b18c946c8c0f10 not found: ID does not exist" Jul 09 15:39:24.467865 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.467792 2567 scope.go:117] "RemoveContainer" containerID="34067c9a4564f6bcada47623d1da2f50c09515b34718e59f96dadb1bf553a402" Jul 09 15:39:24.468066 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:39:24.468049 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34067c9a4564f6bcada47623d1da2f50c09515b34718e59f96dadb1bf553a402\": container with ID starting with 34067c9a4564f6bcada47623d1da2f50c09515b34718e59f96dadb1bf553a402 not found: ID does not exist" containerID="34067c9a4564f6bcada47623d1da2f50c09515b34718e59f96dadb1bf553a402" Jul 09 15:39:24.468110 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.468073 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34067c9a4564f6bcada47623d1da2f50c09515b34718e59f96dadb1bf553a402"} err="failed to get container status \"34067c9a4564f6bcada47623d1da2f50c09515b34718e59f96dadb1bf553a402\": rpc error: code = NotFound desc = could not find container \"34067c9a4564f6bcada47623d1da2f50c09515b34718e59f96dadb1bf553a402\": container with ID starting with 34067c9a4564f6bcada47623d1da2f50c09515b34718e59f96dadb1bf553a402 not found: ID does not exist" Jul 09 15:39:24.474031 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.474003 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj"] Jul 09 15:39:24.479468 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:24.479434 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-a65149-openshift-default-548cf5c945-9k5pj"] Jul 09 15:39:25.829968 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:25.829934 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3831594-36d5-4089-94b2-ea8c7aacc3dc" path="/var/lib/kubelet/pods/e3831594-36d5-4089-94b2-ea8c7aacc3dc/volumes" Jul 09 15:39:27.060975 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.060918 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:27.354021 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.353917 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:27.684477 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.684395 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7"] Jul 09 15:39:27.684860 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.684846 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cb8185d-8f9d-46c6-aa1f-e01c9b0101af" containerName="istio-proxy" Jul 09 15:39:27.684860 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.684860 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cb8185d-8f9d-46c6-aa1f-e01c9b0101af" containerName="istio-proxy" Jul 09 15:39:27.684953 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.684871 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3831594-36d5-4089-94b2-ea8c7aacc3dc" containerName="istio-proxy" Jul 09 15:39:27.684953 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.684876 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3831594-36d5-4089-94b2-ea8c7aacc3dc" containerName="istio-proxy" Jul 09 15:39:27.684953 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.684953 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3831594-36d5-4089-94b2-ea8c7aacc3dc" containerName="istio-proxy" Jul 09 15:39:27.685048 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.684963 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3831594-36d5-4089-94b2-ea8c7aacc3dc" containerName="istio-proxy" Jul 09 15:39:27.688082 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.688065 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:39:27.695843 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.695273 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-863e67-openshift-default-dockercfg-q9j9p\"" Jul 09 15:39:27.708957 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.708919 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7"] Jul 09 15:39:27.767036 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.767000 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-istio-data\") pod \"e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7\" (UID: \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\") " pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:39:27.767230 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.767045 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-workload-socket\") pod \"e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7\" (UID: \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\") " pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:39:27.767230 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.767087 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fnr2\" (UniqueName: \"kubernetes.io/projected/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-kube-api-access-5fnr2\") pod \"e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7\" (UID: \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\") " pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:39:27.767230 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.767108 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-istio-token\") pod \"e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7\" (UID: \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\") " pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:39:27.767230 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.767151 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-workload-certs\") pod \"e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7\" (UID: \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\") " pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:39:27.767230 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.767210 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-istio-envoy\") pod \"e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7\" (UID: \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\") " pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:39:27.767431 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.767232 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-istiod-ca-cert\") pod \"e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7\" (UID: \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\") " pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:39:27.767431 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.767260 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-credential-socket\") pod \"e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7\" (UID: \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\") " pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:39:27.767431 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.767305 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-istio-podinfo\") pod \"e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7\" (UID: \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\") " pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:39:27.869009 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.868949 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-istio-data\") pod \"e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7\" (UID: \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\") " pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:39:27.869424 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.869393 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-istio-data\") pod \"e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7\" (UID: \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\") " pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:39:27.869982 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.869951 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-workload-socket\") pod \"e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7\" (UID: \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\") " pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:39:27.870110 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.870072 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fnr2\" (UniqueName: \"kubernetes.io/projected/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-kube-api-access-5fnr2\") pod \"e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7\" (UID: \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\") " pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:39:27.870110 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.870096 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-istio-token\") pod \"e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7\" (UID: \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\") " pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:39:27.870215 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.870118 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-workload-certs\") pod \"e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7\" (UID: \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\") " pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:39:27.870215 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.870135 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-workload-socket\") pod \"e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7\" (UID: \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\") " pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:39:27.870215 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.870156 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-istio-envoy\") pod \"e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7\" (UID: \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\") " pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:39:27.870215 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.870183 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-istiod-ca-cert\") pod \"e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7\" (UID: \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\") " pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:39:27.870440 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.870224 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-credential-socket\") pod \"e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7\" (UID: \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\") " pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:39:27.870440 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.870314 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-istio-podinfo\") pod \"e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7\" (UID: \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\") " pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:39:27.870720 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.870697 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-workload-certs\") pod \"e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7\" (UID: \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\") " pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:39:27.870845 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.870777 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-credential-socket\") pod \"e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7\" (UID: \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\") " pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:39:27.871429 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.871407 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-istiod-ca-cert\") pod \"e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7\" (UID: \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\") " pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:39:27.872733 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.872702 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-istio-podinfo\") pod \"e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7\" (UID: \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\") " pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:39:27.872907 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.872883 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-istio-envoy\") pod \"e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7\" (UID: \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\") " pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:39:27.888911 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.888884 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-istio-token\") pod \"e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7\" (UID: \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\") " pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:39:27.889035 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.889021 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fnr2\" (UniqueName: \"kubernetes.io/projected/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-kube-api-access-5fnr2\") pod \"e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7\" (UID: \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\") " pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:39:27.957265 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:27.957233 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:28.009380 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:28.009060 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:39:28.128439 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:28.128341 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:28.210562 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:28.209728 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7"] Jul 09 15:39:28.220516 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:39:28.220486 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfa19dfd_2f5c_47ab_bdd7_1e1e0c33a59b.slice/crio-029812627b2a27c1d6fe150d3b9e3255ba8cd49ca55db8db489a1157d9c781d4 WatchSource:0}: Error finding container 029812627b2a27c1d6fe150d3b9e3255ba8cd49ca55db8db489a1157d9c781d4: Status 404 returned error can't find the container with id 029812627b2a27c1d6fe150d3b9e3255ba8cd49ca55db8db489a1157d9c781d4 Jul 09 15:39:28.223043 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:28.223013 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:39:28.223136 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:28.223088 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:39:28.223136 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:28.223118 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:39:28.474267 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:28.474163 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" event={"ID":"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b","Type":"ContainerStarted","Data":"8fc99c690cf393137a3e62ffb1fee19633bb3480e61212f9dd891cce49cb6014"} Jul 09 15:39:28.474426 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:28.474274 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" event={"ID":"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b","Type":"ContainerStarted","Data":"029812627b2a27c1d6fe150d3b9e3255ba8cd49ca55db8db489a1157d9c781d4"} Jul 09 15:39:28.503328 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:28.503256 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" podStartSLOduration=1.5032345569999999 podStartE2EDuration="1.503234557s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:39:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:39:28.502773545 +0000 UTC m=+2215.233280037" watchObservedRunningTime="2026-07-09 15:39:28.503234557 +0000 UTC m=+2215.233741049" Jul 09 15:39:29.012168 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:29.011655 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:39:29.012168 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:29.011797 2567 patch_prober.go:28] interesting pod/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 09 15:39:29.012168 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:29.011870 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" podUID="cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 09 15:39:30.010435 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:30.010389 2567 patch_prober.go:28] interesting pod/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 09 15:39:30.010972 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:30.010461 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" podUID="cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 09 15:39:30.564211 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:30.564169 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:30.678235 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:30.678189 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:30.796534 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:30.796488 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:30.945546 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:30.945447 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:31.018401 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:31.018142 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:39:31.448515 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:31.448478 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:31.491273 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:31.491232 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:39:31.492448 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:31.492420 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:39:31.678790 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:31.678740 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:34.511981 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:34.511886 2567 generic.go:356] "Generic (PLEG): container finished" podID="cf5b03dd-989b-440e-aea3-5bce8bc0598d" containerID="bec4cbba80cc33ff866f5be3580c47018ecf71137f8c69f86ebea68075e90c50" exitCode=0 Jul 09 15:39:34.511981 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:34.511966 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" event={"ID":"cf5b03dd-989b-440e-aea3-5bce8bc0598d","Type":"ContainerDied","Data":"bec4cbba80cc33ff866f5be3580c47018ecf71137f8c69f86ebea68075e90c50"} Jul 09 15:39:34.512441 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:34.512426 2567 scope.go:117] "RemoveContainer" containerID="bec4cbba80cc33ff866f5be3580c47018ecf71137f8c69f86ebea68075e90c50" Jul 09 15:39:34.513547 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:34.513517 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:39:34.513637 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:34.513621 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:39:34.513679 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:34.513665 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:39:35.030444 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:35.030408 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:35.522173 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:35.522138 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" event={"ID":"cf5b03dd-989b-440e-aea3-5bce8bc0598d","Type":"ContainerStarted","Data":"b0e7150fd2853f18ac80b97d864de105f9bf03001f75612c6cbe2202613d69ef"} Jul 09 15:39:36.101095 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:36.101037 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:39:36.102521 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:36.102491 2567 patch_prober.go:28] interesting pod/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 09 15:39:36.102756 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:36.102719 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" podUID="cf5b03dd-989b-440e-aea3-5bce8bc0598d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 09 15:39:36.430059 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:36.429957 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:39:37.101190 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:37.101137 2567 patch_prober.go:28] interesting pod/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 09 15:39:37.101713 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:37.101217 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" podUID="cf5b03dd-989b-440e-aea3-5bce8bc0598d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 09 15:39:38.100792 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:38.100748 2567 patch_prober.go:28] interesting pod/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 09 15:39:38.101112 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:38.101080 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" podUID="cf5b03dd-989b-440e-aea3-5bce8bc0598d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 09 15:39:39.101348 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:39.101295 2567 patch_prober.go:28] interesting pod/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 09 15:39:39.101915 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:39.101375 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" podUID="cf5b03dd-989b-440e-aea3-5bce8bc0598d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 09 15:39:40.109749 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:40.109701 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:39:40.114334 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:40.114219 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:39:44.579158 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:44.579122 2567 generic.go:356] "Generic (PLEG): container finished" podID="cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b" containerID="8fc99c690cf393137a3e62ffb1fee19633bb3480e61212f9dd891cce49cb6014" exitCode=0 Jul 09 15:39:44.579749 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:44.579201 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" event={"ID":"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b","Type":"ContainerDied","Data":"8fc99c690cf393137a3e62ffb1fee19633bb3480e61212f9dd891cce49cb6014"} Jul 09 15:39:44.579749 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:44.579623 2567 scope.go:117] "RemoveContainer" containerID="8fc99c690cf393137a3e62ffb1fee19633bb3480e61212f9dd891cce49cb6014" Jul 09 15:39:44.580669 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:44.580641 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:39:44.580781 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:44.580726 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:39:44.580781 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:44.580768 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:39:45.546741 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:45.546701 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:45.587382 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:45.587346 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" event={"ID":"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b","Type":"ContainerStarted","Data":"8aa1896c851cdc2ba32b9860d3d2cbec9498c443e98cbbce043e40cf5c359c17"} Jul 09 15:39:45.646224 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:45.646184 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:46.010209 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:46.010170 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:39:46.012983 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:46.012945 2567 patch_prober.go:28] interesting pod/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 09 15:39:46.013154 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:46.013008 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" podUID="cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 09 15:39:46.491962 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:46.491921 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:39:47.010645 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:47.010599 2567 patch_prober.go:28] interesting pod/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 09 15:39:47.011211 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:47.010669 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" podUID="cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 09 15:39:48.010654 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:48.010612 2567 patch_prober.go:28] interesting pod/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 09 15:39:48.011316 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:48.011283 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" podUID="cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 09 15:39:49.010560 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:49.010510 2567 patch_prober.go:28] interesting pod/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 09 15:39:49.010891 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:49.010854 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" podUID="cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 09 15:39:50.014515 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:50.014483 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:39:50.016045 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:50.015995 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:39:52.432372 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:52.432334 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:39:52.532526 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:39:52.532481 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:41:14.265469 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:41:14.265428 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:42:28.935094 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:28.935054 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:42:33.905284 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:33.905252 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hml7m_84a10578-4913-456b-9fb7-e6440914b677/ovn-acl-logging/0.log" Jul 09 15:42:33.917084 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:33.917052 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hml7m_84a10578-4913-456b-9fb7-e6440914b677/ovn-acl-logging/0.log" Jul 09 15:42:40.172044 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:40.171979 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:42:40.605924 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:40.605858 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:42:41.076222 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:41.076185 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:42:41.339311 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:41.339216 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:42:41.502527 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:41.502486 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:42:41.791570 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:41.791534 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:42:48.182098 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:48.182062 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:42:48.287507 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:48.287478 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:42:48.522255 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:48.522216 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k"] Jul 09 15:42:48.522497 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:48.522473 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" podUID="cf5b03dd-989b-440e-aea3-5bce8bc0598d" containerName="istio-proxy" containerID="cri-o://b0e7150fd2853f18ac80b97d864de105f9bf03001f75612c6cbe2202613d69ef" gracePeriod=30 Jul 09 15:42:48.586827 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:48.586784 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:42:49.207808 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:49.207765 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:42:49.624581 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:49.624489 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:42:49.912174 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:49.912084 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:42:50.034518 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:50.034480 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:42:50.349191 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:50.349156 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:42:53.785060 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:53.784986 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:42:53.888453 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:53.888413 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf5b03dd-989b-440e-aea3-5bce8bc0598d-istio-data\") pod \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\" (UID: \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\") " Jul 09 15:42:53.888453 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:53.888453 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf5b03dd-989b-440e-aea3-5bce8bc0598d-istio-token\") pod \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\" (UID: \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\") " Jul 09 15:42:53.888712 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:53.888472 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf5b03dd-989b-440e-aea3-5bce8bc0598d-istiod-ca-cert\") pod \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\" (UID: \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\") " Jul 09 15:42:53.888712 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:53.888498 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf5b03dd-989b-440e-aea3-5bce8bc0598d-istio-podinfo\") pod \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\" (UID: \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\") " Jul 09 15:42:53.888712 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:53.888616 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf5b03dd-989b-440e-aea3-5bce8bc0598d-workload-certs\") pod \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\" (UID: \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\") " Jul 09 15:42:53.888712 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:53.888645 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf5b03dd-989b-440e-aea3-5bce8bc0598d-workload-socket\") pod \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\" (UID: \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\") " Jul 09 15:42:53.888712 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:53.888686 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4f2s8\" (UniqueName: \"kubernetes.io/projected/cf5b03dd-989b-440e-aea3-5bce8bc0598d-kube-api-access-4f2s8\") pod \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\" (UID: \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\") " Jul 09 15:42:53.889016 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:53.888754 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf5b03dd-989b-440e-aea3-5bce8bc0598d-istio-envoy\") pod \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\" (UID: \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\") " Jul 09 15:42:53.889016 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:53.888872 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf5b03dd-989b-440e-aea3-5bce8bc0598d-credential-socket\") pod \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\" (UID: \"cf5b03dd-989b-440e-aea3-5bce8bc0598d\") " Jul 09 15:42:53.889016 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:53.888961 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf5b03dd-989b-440e-aea3-5bce8bc0598d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cf5b03dd-989b-440e-aea3-5bce8bc0598d" (UID: "cf5b03dd-989b-440e-aea3-5bce8bc0598d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:42:53.889016 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:53.888971 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf5b03dd-989b-440e-aea3-5bce8bc0598d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cf5b03dd-989b-440e-aea3-5bce8bc0598d" (UID: "cf5b03dd-989b-440e-aea3-5bce8bc0598d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:42:53.889016 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:53.888983 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf5b03dd-989b-440e-aea3-5bce8bc0598d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cf5b03dd-989b-440e-aea3-5bce8bc0598d" (UID: "cf5b03dd-989b-440e-aea3-5bce8bc0598d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:42:53.889285 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:53.889172 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf5b03dd-989b-440e-aea3-5bce8bc0598d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cf5b03dd-989b-440e-aea3-5bce8bc0598d" (UID: "cf5b03dd-989b-440e-aea3-5bce8bc0598d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:42:53.889340 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:53.889328 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf5b03dd-989b-440e-aea3-5bce8bc0598d-istiod-ca-cert\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:42:53.889405 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:53.889346 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf5b03dd-989b-440e-aea3-5bce8bc0598d-workload-certs\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:42:53.889405 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:53.889359 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf5b03dd-989b-440e-aea3-5bce8bc0598d-workload-socket\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:42:53.889405 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:53.889371 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf5b03dd-989b-440e-aea3-5bce8bc0598d-credential-socket\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:42:53.889556 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:53.889435 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf5b03dd-989b-440e-aea3-5bce8bc0598d-istio-data" (OuterVolumeSpecName: "istio-data") pod "cf5b03dd-989b-440e-aea3-5bce8bc0598d" (UID: "cf5b03dd-989b-440e-aea3-5bce8bc0598d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:42:53.891082 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:53.891038 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf5b03dd-989b-440e-aea3-5bce8bc0598d-istio-token" (OuterVolumeSpecName: "istio-token") pod "cf5b03dd-989b-440e-aea3-5bce8bc0598d" (UID: "cf5b03dd-989b-440e-aea3-5bce8bc0598d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:42:53.891227 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:53.891084 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf5b03dd-989b-440e-aea3-5bce8bc0598d-kube-api-access-4f2s8" (OuterVolumeSpecName: "kube-api-access-4f2s8") pod "cf5b03dd-989b-440e-aea3-5bce8bc0598d" (UID: "cf5b03dd-989b-440e-aea3-5bce8bc0598d"). InnerVolumeSpecName "kube-api-access-4f2s8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:42:53.891418 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:53.891397 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf5b03dd-989b-440e-aea3-5bce8bc0598d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cf5b03dd-989b-440e-aea3-5bce8bc0598d" (UID: "cf5b03dd-989b-440e-aea3-5bce8bc0598d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:42:53.891485 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:53.891401 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cf5b03dd-989b-440e-aea3-5bce8bc0598d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cf5b03dd-989b-440e-aea3-5bce8bc0598d" (UID: "cf5b03dd-989b-440e-aea3-5bce8bc0598d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 15:42:53.990406 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:53.990355 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4f2s8\" (UniqueName: \"kubernetes.io/projected/cf5b03dd-989b-440e-aea3-5bce8bc0598d-kube-api-access-4f2s8\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:42:53.990406 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:53.990405 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf5b03dd-989b-440e-aea3-5bce8bc0598d-istio-envoy\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:42:53.990406 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:53.990415 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf5b03dd-989b-440e-aea3-5bce8bc0598d-istio-data\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:42:53.990406 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:53.990423 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf5b03dd-989b-440e-aea3-5bce8bc0598d-istio-token\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:42:53.990406 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:53.990431 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf5b03dd-989b-440e-aea3-5bce8bc0598d-istio-podinfo\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:42:54.634077 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:54.633980 2567 generic.go:356] "Generic (PLEG): container finished" podID="cf5b03dd-989b-440e-aea3-5bce8bc0598d" containerID="b0e7150fd2853f18ac80b97d864de105f9bf03001f75612c6cbe2202613d69ef" exitCode=0 Jul 09 15:42:54.634077 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:54.634048 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" Jul 09 15:42:54.634077 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:54.634061 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" event={"ID":"cf5b03dd-989b-440e-aea3-5bce8bc0598d","Type":"ContainerDied","Data":"b0e7150fd2853f18ac80b97d864de105f9bf03001f75612c6cbe2202613d69ef"} Jul 09 15:42:54.634394 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:54.634155 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k" event={"ID":"cf5b03dd-989b-440e-aea3-5bce8bc0598d","Type":"ContainerDied","Data":"259759156aae85e7ca9a1924eae7b909348b380b8803fe46cf0575ab857da7d9"} Jul 09 15:42:54.634394 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:54.634198 2567 scope.go:117] "RemoveContainer" containerID="b0e7150fd2853f18ac80b97d864de105f9bf03001f75612c6cbe2202613d69ef" Jul 09 15:42:54.645171 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:54.645151 2567 scope.go:117] "RemoveContainer" containerID="bec4cbba80cc33ff866f5be3580c47018ecf71137f8c69f86ebea68075e90c50" Jul 09 15:42:54.654090 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:54.654068 2567 scope.go:117] "RemoveContainer" containerID="b0e7150fd2853f18ac80b97d864de105f9bf03001f75612c6cbe2202613d69ef" Jul 09 15:42:54.654407 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:42:54.654386 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0e7150fd2853f18ac80b97d864de105f9bf03001f75612c6cbe2202613d69ef\": container with ID starting with b0e7150fd2853f18ac80b97d864de105f9bf03001f75612c6cbe2202613d69ef not found: ID does not exist" containerID="b0e7150fd2853f18ac80b97d864de105f9bf03001f75612c6cbe2202613d69ef" Jul 09 15:42:54.654455 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:54.654418 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0e7150fd2853f18ac80b97d864de105f9bf03001f75612c6cbe2202613d69ef"} err="failed to get container status \"b0e7150fd2853f18ac80b97d864de105f9bf03001f75612c6cbe2202613d69ef\": rpc error: code = NotFound desc = could not find container \"b0e7150fd2853f18ac80b97d864de105f9bf03001f75612c6cbe2202613d69ef\": container with ID starting with b0e7150fd2853f18ac80b97d864de105f9bf03001f75612c6cbe2202613d69ef not found: ID does not exist" Jul 09 15:42:54.654455 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:54.654436 2567 scope.go:117] "RemoveContainer" containerID="bec4cbba80cc33ff866f5be3580c47018ecf71137f8c69f86ebea68075e90c50" Jul 09 15:42:54.654672 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:42:54.654650 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bec4cbba80cc33ff866f5be3580c47018ecf71137f8c69f86ebea68075e90c50\": container with ID starting with bec4cbba80cc33ff866f5be3580c47018ecf71137f8c69f86ebea68075e90c50 not found: ID does not exist" containerID="bec4cbba80cc33ff866f5be3580c47018ecf71137f8c69f86ebea68075e90c50" Jul 09 15:42:54.654734 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:54.654681 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bec4cbba80cc33ff866f5be3580c47018ecf71137f8c69f86ebea68075e90c50"} err="failed to get container status \"bec4cbba80cc33ff866f5be3580c47018ecf71137f8c69f86ebea68075e90c50\": rpc error: code = NotFound desc = could not find container \"bec4cbba80cc33ff866f5be3580c47018ecf71137f8c69f86ebea68075e90c50\": container with ID starting with bec4cbba80cc33ff866f5be3580c47018ecf71137f8c69f86ebea68075e90c50 not found: ID does not exist" Jul 09 15:42:54.664585 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:54.664553 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k"] Jul 09 15:42:54.675934 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:54.675902 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-dcc4b5-openshift-default-6f5b544ccf-kn75k"] Jul 09 15:42:55.829003 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:55.828970 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf5b03dd-989b-440e-aea3-5bce8bc0598d" path="/var/lib/kubelet/pods/cf5b03dd-989b-440e-aea3-5bce8bc0598d/volumes" Jul 09 15:42:56.437519 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:56.437483 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:42:56.545704 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:56.545671 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:42:56.761216 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:56.761179 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7"] Jul 09 15:42:56.761420 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:56.761384 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" podUID="cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b" containerName="istio-proxy" containerID="cri-o://8aa1896c851cdc2ba32b9860d3d2cbec9498c443e98cbbce043e40cf5c359c17" gracePeriod=30 Jul 09 15:42:56.810390 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:42:56.810352 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:43:02.029279 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.029207 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:43:02.072740 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.072708 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-istio-token\") pod \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\" (UID: \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\") " Jul 09 15:43:02.072960 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.072758 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-workload-certs\") pod \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\" (UID: \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\") " Jul 09 15:43:02.072960 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.072786 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fnr2\" (UniqueName: \"kubernetes.io/projected/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-kube-api-access-5fnr2\") pod \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\" (UID: \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\") " Jul 09 15:43:02.072960 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.072830 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-workload-socket\") pod \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\" (UID: \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\") " Jul 09 15:43:02.072960 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.072853 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-istiod-ca-cert\") pod \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\" (UID: \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\") " Jul 09 15:43:02.072960 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.072909 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-istio-data\") pod \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\" (UID: \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\") " Jul 09 15:43:02.072960 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.072942 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-istio-envoy\") pod \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\" (UID: \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\") " Jul 09 15:43:02.073303 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.073021 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-istio-podinfo\") pod \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\" (UID: \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\") " Jul 09 15:43:02.073303 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.073048 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-credential-socket\") pod \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\" (UID: \"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b\") " Jul 09 15:43:02.073303 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.073101 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b" (UID: "cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:43:02.073460 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.073330 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b" (UID: "cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:43:02.073460 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.073365 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b" (UID: "cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:43:02.073460 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.073439 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b" (UID: "cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:43:02.073677 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.073657 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-credential-socket\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:43:02.073736 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.073684 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-workload-certs\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:43:02.073736 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.073700 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-workload-socket\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:43:02.073736 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.073714 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-istiod-ca-cert\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:43:02.074019 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.073995 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-istio-data" (OuterVolumeSpecName: "istio-data") pod "cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b" (UID: "cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:43:02.075465 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.075437 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b" (UID: "cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:43:02.075841 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.075800 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-istio-token" (OuterVolumeSpecName: "istio-token") pod "cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b" (UID: "cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:43:02.075952 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.075858 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-kube-api-access-5fnr2" (OuterVolumeSpecName: "kube-api-access-5fnr2") pod "cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b" (UID: "cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b"). InnerVolumeSpecName "kube-api-access-5fnr2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:43:02.076069 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.076048 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b" (UID: "cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 15:43:02.174796 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.174757 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-istio-token\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:43:02.174796 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.174791 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fnr2\" (UniqueName: \"kubernetes.io/projected/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-kube-api-access-5fnr2\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:43:02.174796 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.174801 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-istio-data\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:43:02.175055 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.174810 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-istio-envoy\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:43:02.175055 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.174843 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b-istio-podinfo\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 09 15:43:02.678466 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.678434 2567 generic.go:356] "Generic (PLEG): container finished" podID="cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b" containerID="8aa1896c851cdc2ba32b9860d3d2cbec9498c443e98cbbce043e40cf5c359c17" exitCode=0 Jul 09 15:43:02.678677 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.678490 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" Jul 09 15:43:02.678677 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.678526 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" event={"ID":"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b","Type":"ContainerDied","Data":"8aa1896c851cdc2ba32b9860d3d2cbec9498c443e98cbbce043e40cf5c359c17"} Jul 09 15:43:02.678677 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.678587 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7" event={"ID":"cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b","Type":"ContainerDied","Data":"029812627b2a27c1d6fe150d3b9e3255ba8cd49ca55db8db489a1157d9c781d4"} Jul 09 15:43:02.678677 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.678617 2567 scope.go:117] "RemoveContainer" containerID="8aa1896c851cdc2ba32b9860d3d2cbec9498c443e98cbbce043e40cf5c359c17" Jul 09 15:43:02.688665 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.688644 2567 scope.go:117] "RemoveContainer" containerID="8fc99c690cf393137a3e62ffb1fee19633bb3480e61212f9dd891cce49cb6014" Jul 09 15:43:02.700042 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.700013 2567 scope.go:117] "RemoveContainer" containerID="8aa1896c851cdc2ba32b9860d3d2cbec9498c443e98cbbce043e40cf5c359c17" Jul 09 15:43:02.700409 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:43:02.700387 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8aa1896c851cdc2ba32b9860d3d2cbec9498c443e98cbbce043e40cf5c359c17\": container with ID starting with 8aa1896c851cdc2ba32b9860d3d2cbec9498c443e98cbbce043e40cf5c359c17 not found: ID does not exist" containerID="8aa1896c851cdc2ba32b9860d3d2cbec9498c443e98cbbce043e40cf5c359c17" Jul 09 15:43:02.700514 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.700426 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8aa1896c851cdc2ba32b9860d3d2cbec9498c443e98cbbce043e40cf5c359c17"} err="failed to get container status \"8aa1896c851cdc2ba32b9860d3d2cbec9498c443e98cbbce043e40cf5c359c17\": rpc error: code = NotFound desc = could not find container \"8aa1896c851cdc2ba32b9860d3d2cbec9498c443e98cbbce043e40cf5c359c17\": container with ID starting with 8aa1896c851cdc2ba32b9860d3d2cbec9498c443e98cbbce043e40cf5c359c17 not found: ID does not exist" Jul 09 15:43:02.700514 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.700457 2567 scope.go:117] "RemoveContainer" containerID="8fc99c690cf393137a3e62ffb1fee19633bb3480e61212f9dd891cce49cb6014" Jul 09 15:43:02.700768 ip-10-0-130-114 kubenswrapper[2567]: E0709 15:43:02.700747 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fc99c690cf393137a3e62ffb1fee19633bb3480e61212f9dd891cce49cb6014\": container with ID starting with 8fc99c690cf393137a3e62ffb1fee19633bb3480e61212f9dd891cce49cb6014 not found: ID does not exist" containerID="8fc99c690cf393137a3e62ffb1fee19633bb3480e61212f9dd891cce49cb6014" Jul 09 15:43:02.700829 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.700778 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fc99c690cf393137a3e62ffb1fee19633bb3480e61212f9dd891cce49cb6014"} err="failed to get container status \"8fc99c690cf393137a3e62ffb1fee19633bb3480e61212f9dd891cce49cb6014\": rpc error: code = NotFound desc = could not find container \"8fc99c690cf393137a3e62ffb1fee19633bb3480e61212f9dd891cce49cb6014\": container with ID starting with 8fc99c690cf393137a3e62ffb1fee19633bb3480e61212f9dd891cce49cb6014 not found: ID does not exist" Jul 09 15:43:02.708573 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.708542 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7"] Jul 09 15:43:02.716253 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.716224 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-863e67-openshift-default-786dc8cdcf-q8ng7"] Jul 09 15:43:02.827148 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:02.827118 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-p44qb_026a46f5-a5f0-4cfc-95d6-a4e920ac37ff/manager/0.log" Jul 09 15:43:03.108740 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:03.108692 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-58d8c4744b-nrkr4_1c767830-cb4b-44a3-9b9f-3bb5f0e55ecc/manager/0.log" Jul 09 15:43:03.531283 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:03.531252 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66b8fbd475-29fz2_0461189c-aa86-4d9e-b865-ab27bf0830ef/copy-manifests/0.log" Jul 09 15:43:03.539389 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:03.539360 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66b8fbd475-29fz2_0461189c-aa86-4d9e-b865-ab27bf0830ef/manager/0.log" Jul 09 15:43:03.829741 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:03.829657 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b" path="/var/lib/kubelet/pods/cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b/volumes" Jul 09 15:43:05.981690 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:05.981655 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-5wr6x_9ec5bec4-baca-4251-9fdc-0085ccbc5776/manager/0.log" Jul 09 15:43:06.651644 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:06.651611 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4j6xdb_8e7db3e0-a93f-456c-b4a3-395682060224/istio-proxy/0.log" Jul 09 15:43:07.209073 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:07.209038 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-b4hrg_62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978/istio-proxy/1.log" Jul 09 15:43:07.476349 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:07.476248 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-9zt6p_4007bf49-82d0-4310-b0f4-21bf752c912e/payload-processing/0.log" Jul 09 15:43:07.991061 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:07.991030 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-qdbw9_b5e394ed-ab9b-4e85-a315-2dfbeb449ec7/main/0.log" Jul 09 15:43:08.249135 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:08.249036 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-bg222_23488e80-cefa-4039-86e5-a1fdb1d73838/main/0.log" Jul 09 15:43:08.517337 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:08.517249 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-8nhwg_710b727a-4d18-4f43-9273-4b11afd32afd/main/0.log" Jul 09 15:43:10.004910 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:10.004874 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:43:13.082895 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:13.082864 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:43:13.242737 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:13.242703 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" limit=1024 Jul 09 15:43:18.046922 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:18.046888 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-p8vsh_fce18369-7aaf-4358-b5f5-46b0218924fd/global-pull-secret-syncer/0.log" Jul 09 15:43:18.163285 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:18.163251 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9647m_daad1095-91b7-47c6-bddf-dd0b6f01f8a9/konnectivity-agent/0.log" Jul 09 15:43:18.630163 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:18.630129 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-114.ec2.internal_b4114c320ef9f8839a18178df34d18c8/haproxy/0.log" Jul 09 15:43:23.739754 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:23.739716 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-5wr6x_9ec5bec4-baca-4251-9fdc-0085ccbc5776/manager/0.log" Jul 09 15:43:25.681532 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:25.681493 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f708c85c-e7da-4a8d-ba05-7a4c442fb6f5/alertmanager/0.log" Jul 09 15:43:25.712700 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:25.712665 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f708c85c-e7da-4a8d-ba05-7a4c442fb6f5/config-reloader/0.log" Jul 09 15:43:25.739465 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:25.739432 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f708c85c-e7da-4a8d-ba05-7a4c442fb6f5/kube-rbac-proxy-web/0.log" Jul 09 15:43:25.766539 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:25.766507 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f708c85c-e7da-4a8d-ba05-7a4c442fb6f5/kube-rbac-proxy/0.log" Jul 09 15:43:25.794792 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:25.794755 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f708c85c-e7da-4a8d-ba05-7a4c442fb6f5/kube-rbac-proxy-metric/0.log" Jul 09 15:43:25.823847 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:25.823801 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f708c85c-e7da-4a8d-ba05-7a4c442fb6f5/prom-label-proxy/0.log" Jul 09 15:43:25.856888 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:25.856849 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f708c85c-e7da-4a8d-ba05-7a4c442fb6f5/init-config-reloader/0.log" Jul 09 15:43:25.946353 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:25.946264 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-nzgcs_38241def-a91a-478f-b3cc-cf4f2eea029d/kube-state-metrics/0.log" Jul 09 15:43:25.966781 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:25.966748 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-nzgcs_38241def-a91a-478f-b3cc-cf4f2eea029d/kube-rbac-proxy-main/0.log" Jul 09 15:43:25.995453 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:25.995424 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-nzgcs_38241def-a91a-478f-b3cc-cf4f2eea029d/kube-rbac-proxy-self/0.log" Jul 09 15:43:26.031840 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:26.031766 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-9bbc5d8d4-gxffh_48425dff-9ae5-4922-a795-503e8ea60a7c/metrics-server/0.log" Jul 09 15:43:26.070431 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:26.070392 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-w44lv_2e6e9898-83c1-42bf-862a-ee35c5629109/monitoring-plugin/0.log" Jul 09 15:43:26.109224 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:26.109177 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6wps2_8c194403-1e4f-4469-8b40-ead103f9edef/node-exporter/0.log" Jul 09 15:43:26.135022 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:26.134994 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6wps2_8c194403-1e4f-4469-8b40-ead103f9edef/kube-rbac-proxy/0.log" Jul 09 15:43:26.161537 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:26.161510 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6wps2_8c194403-1e4f-4469-8b40-ead103f9edef/init-textfile/0.log" Jul 09 15:43:26.392065 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:26.392002 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-kc99w_af9403ab-61af-488f-b826-9cf5be0bcd3a/kube-rbac-proxy-main/0.log" Jul 09 15:43:26.415745 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:26.415708 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-kc99w_af9403ab-61af-488f-b826-9cf5be0bcd3a/kube-rbac-proxy-self/0.log" Jul 09 15:43:26.441724 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:26.441682 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-kc99w_af9403ab-61af-488f-b826-9cf5be0bcd3a/openshift-state-metrics/0.log" Jul 09 15:43:26.530463 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:26.530368 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f412c893-a405-4dbf-b710-678a4f373ac7/prometheus/0.log" Jul 09 15:43:26.609494 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:26.609455 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f412c893-a405-4dbf-b710-678a4f373ac7/config-reloader/0.log" Jul 09 15:43:26.659102 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:26.659066 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f412c893-a405-4dbf-b710-678a4f373ac7/thanos-sidecar/0.log" Jul 09 15:43:26.697021 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:26.696995 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f412c893-a405-4dbf-b710-678a4f373ac7/kube-rbac-proxy-web/0.log" Jul 09 15:43:26.726300 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:26.726264 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f412c893-a405-4dbf-b710-678a4f373ac7/kube-rbac-proxy/0.log" Jul 09 15:43:26.767954 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:26.767926 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f412c893-a405-4dbf-b710-678a4f373ac7/kube-rbac-proxy-thanos/0.log" Jul 09 15:43:26.798469 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:26.798379 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f412c893-a405-4dbf-b710-678a4f373ac7/init-config-reloader/0.log" Jul 09 15:43:26.851623 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:26.851589 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-wc22m_3f1edfb6-5fd6-4635-adf3-e2662f222fb0/prometheus-operator/0.log" Jul 09 15:43:26.880955 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:26.880927 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-wc22m_3f1edfb6-5fd6-4635-adf3-e2662f222fb0/kube-rbac-proxy/0.log" Jul 09 15:43:26.910583 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:26.910547 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-4cpbv_b988013e-d1a6-416c-885c-b200d6686a8d/prometheus-operator-admission-webhook/0.log" Jul 09 15:43:26.951316 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:26.951285 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-c6854b798-q486f_49820873-b1af-45a8-aed3-24c2d227f933/telemeter-client/0.log" Jul 09 15:43:26.993547 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:26.993515 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-c6854b798-q486f_49820873-b1af-45a8-aed3-24c2d227f933/reload/0.log" Jul 09 15:43:27.034269 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:27.034236 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-c6854b798-q486f_49820873-b1af-45a8-aed3-24c2d227f933/kube-rbac-proxy/0.log" Jul 09 15:43:27.076395 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:27.076308 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78898947d8-cj86l_e0367d01-739b-4d39-9457-e5702482e786/thanos-query/0.log" Jul 09 15:43:27.104769 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:27.104714 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78898947d8-cj86l_e0367d01-739b-4d39-9457-e5702482e786/kube-rbac-proxy-web/0.log" Jul 09 15:43:27.151354 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:27.151322 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78898947d8-cj86l_e0367d01-739b-4d39-9457-e5702482e786/kube-rbac-proxy/0.log" Jul 09 15:43:27.211870 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:27.211803 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78898947d8-cj86l_e0367d01-739b-4d39-9457-e5702482e786/prom-label-proxy/0.log" Jul 09 15:43:27.252112 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:27.252076 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78898947d8-cj86l_e0367d01-739b-4d39-9457-e5702482e786/kube-rbac-proxy-rules/0.log" Jul 09 15:43:27.288737 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:27.288703 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78898947d8-cj86l_e0367d01-739b-4d39-9457-e5702482e786/kube-rbac-proxy-metrics/0.log" Jul 09 15:43:27.942757 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:27.942724 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-v4z7t/perf-node-gather-daemonset-7xncj"] Jul 09 15:43:27.943188 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:27.943174 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b" containerName="istio-proxy" Jul 09 15:43:27.943235 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:27.943189 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b" containerName="istio-proxy" Jul 09 15:43:27.943235 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:27.943200 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b" containerName="istio-proxy" Jul 09 15:43:27.943235 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:27.943205 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b" containerName="istio-proxy" Jul 09 15:43:27.943235 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:27.943216 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf5b03dd-989b-440e-aea3-5bce8bc0598d" containerName="istio-proxy" Jul 09 15:43:27.943235 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:27.943222 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf5b03dd-989b-440e-aea3-5bce8bc0598d" containerName="istio-proxy" Jul 09 15:43:27.943235 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:27.943230 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf5b03dd-989b-440e-aea3-5bce8bc0598d" containerName="istio-proxy" Jul 09 15:43:27.943235 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:27.943236 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf5b03dd-989b-440e-aea3-5bce8bc0598d" containerName="istio-proxy" Jul 09 15:43:27.943480 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:27.943249 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3831594-36d5-4089-94b2-ea8c7aacc3dc" containerName="istio-proxy" Jul 09 15:43:27.943480 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:27.943255 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3831594-36d5-4089-94b2-ea8c7aacc3dc" containerName="istio-proxy" Jul 09 15:43:27.943480 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:27.943318 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf5b03dd-989b-440e-aea3-5bce8bc0598d" containerName="istio-proxy" Jul 09 15:43:27.943480 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:27.943326 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b" containerName="istio-proxy" Jul 09 15:43:27.943480 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:27.943333 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfa19dfd-2f5c-47ab-bdd7-1e1e0c33a59b" containerName="istio-proxy" Jul 09 15:43:27.946537 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:27.946515 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-7xncj" Jul 09 15:43:27.950172 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:27.950148 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-v4z7t\"/\"kube-root-ca.crt\"" Jul 09 15:43:27.951720 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:27.951698 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-v4z7t\"/\"openshift-service-ca.crt\"" Jul 09 15:43:27.951882 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:27.951706 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-v4z7t\"/\"default-dockercfg-8kg4f\"" Jul 09 15:43:27.959661 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:27.959633 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-v4z7t/perf-node-gather-daemonset-7xncj"] Jul 09 15:43:28.032241 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:28.032202 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7aa8882f-e45a-4e16-b742-0d07cd204bda-proc\") pod \"perf-node-gather-daemonset-7xncj\" (UID: \"7aa8882f-e45a-4e16-b742-0d07cd204bda\") " pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-7xncj" Jul 09 15:43:28.032443 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:28.032346 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7aa8882f-e45a-4e16-b742-0d07cd204bda-sys\") pod \"perf-node-gather-daemonset-7xncj\" (UID: \"7aa8882f-e45a-4e16-b742-0d07cd204bda\") " pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-7xncj" Jul 09 15:43:28.032443 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:28.032417 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7aa8882f-e45a-4e16-b742-0d07cd204bda-podres\") pod \"perf-node-gather-daemonset-7xncj\" (UID: \"7aa8882f-e45a-4e16-b742-0d07cd204bda\") " pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-7xncj" Jul 09 15:43:28.032443 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:28.032436 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7aa8882f-e45a-4e16-b742-0d07cd204bda-lib-modules\") pod \"perf-node-gather-daemonset-7xncj\" (UID: \"7aa8882f-e45a-4e16-b742-0d07cd204bda\") " pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-7xncj" Jul 09 15:43:28.032575 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:28.032461 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5trg\" (UniqueName: \"kubernetes.io/projected/7aa8882f-e45a-4e16-b742-0d07cd204bda-kube-api-access-w5trg\") pod \"perf-node-gather-daemonset-7xncj\" (UID: \"7aa8882f-e45a-4e16-b742-0d07cd204bda\") " pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-7xncj" Jul 09 15:43:28.133253 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:28.133214 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7aa8882f-e45a-4e16-b742-0d07cd204bda-sys\") pod \"perf-node-gather-daemonset-7xncj\" (UID: \"7aa8882f-e45a-4e16-b742-0d07cd204bda\") " pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-7xncj" Jul 09 15:43:28.133463 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:28.133275 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7aa8882f-e45a-4e16-b742-0d07cd204bda-podres\") pod \"perf-node-gather-daemonset-7xncj\" (UID: \"7aa8882f-e45a-4e16-b742-0d07cd204bda\") " pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-7xncj" Jul 09 15:43:28.133463 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:28.133296 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7aa8882f-e45a-4e16-b742-0d07cd204bda-lib-modules\") pod \"perf-node-gather-daemonset-7xncj\" (UID: \"7aa8882f-e45a-4e16-b742-0d07cd204bda\") " pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-7xncj" Jul 09 15:43:28.133463 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:28.133377 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7aa8882f-e45a-4e16-b742-0d07cd204bda-sys\") pod \"perf-node-gather-daemonset-7xncj\" (UID: \"7aa8882f-e45a-4e16-b742-0d07cd204bda\") " pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-7xncj" Jul 09 15:43:28.133463 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:28.133403 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7aa8882f-e45a-4e16-b742-0d07cd204bda-lib-modules\") pod \"perf-node-gather-daemonset-7xncj\" (UID: \"7aa8882f-e45a-4e16-b742-0d07cd204bda\") " pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-7xncj" Jul 09 15:43:28.133463 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:28.133410 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7aa8882f-e45a-4e16-b742-0d07cd204bda-podres\") pod \"perf-node-gather-daemonset-7xncj\" (UID: \"7aa8882f-e45a-4e16-b742-0d07cd204bda\") " pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-7xncj" Jul 09 15:43:28.133463 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:28.133415 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5trg\" (UniqueName: \"kubernetes.io/projected/7aa8882f-e45a-4e16-b742-0d07cd204bda-kube-api-access-w5trg\") pod \"perf-node-gather-daemonset-7xncj\" (UID: \"7aa8882f-e45a-4e16-b742-0d07cd204bda\") " pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-7xncj" Jul 09 15:43:28.133729 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:28.133504 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7aa8882f-e45a-4e16-b742-0d07cd204bda-proc\") pod \"perf-node-gather-daemonset-7xncj\" (UID: \"7aa8882f-e45a-4e16-b742-0d07cd204bda\") " pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-7xncj" Jul 09 15:43:28.133729 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:28.133615 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7aa8882f-e45a-4e16-b742-0d07cd204bda-proc\") pod \"perf-node-gather-daemonset-7xncj\" (UID: \"7aa8882f-e45a-4e16-b742-0d07cd204bda\") " pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-7xncj" Jul 09 15:43:28.143811 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:28.143778 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5trg\" (UniqueName: \"kubernetes.io/projected/7aa8882f-e45a-4e16-b742-0d07cd204bda-kube-api-access-w5trg\") pod \"perf-node-gather-daemonset-7xncj\" (UID: \"7aa8882f-e45a-4e16-b742-0d07cd204bda\") " pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-7xncj" Jul 09 15:43:28.257650 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:28.257604 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-7xncj" Jul 09 15:43:28.406559 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:28.406531 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-v4z7t/perf-node-gather-daemonset-7xncj"] Jul 09 15:43:28.408073 ip-10-0-130-114 kubenswrapper[2567]: W0709 15:43:28.408042 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7aa8882f_e45a_4e16_b742_0d07cd204bda.slice/crio-34837b5b446c0b96249cdd9a6c4590f6ebf29e6de724f9799f84e3112a3c74cf WatchSource:0}: Error finding container 34837b5b446c0b96249cdd9a6c4590f6ebf29e6de724f9799f84e3112a3c74cf: Status 404 returned error can't find the container with id 34837b5b446c0b96249cdd9a6c4590f6ebf29e6de724f9799f84e3112a3c74cf Jul 09 15:43:28.823389 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:28.823349 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-7xncj" event={"ID":"7aa8882f-e45a-4e16-b742-0d07cd204bda","Type":"ContainerStarted","Data":"66fd9944f39006c6e3c9d3bf98e37d4e132b9bacce6bc0d6f3f0c3a776033e1d"} Jul 09 15:43:28.823603 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:28.823492 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-7xncj" Jul 09 15:43:28.823603 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:28.823512 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-7xncj" event={"ID":"7aa8882f-e45a-4e16-b742-0d07cd204bda","Type":"ContainerStarted","Data":"34837b5b446c0b96249cdd9a6c4590f6ebf29e6de724f9799f84e3112a3c74cf"} Jul 09 15:43:28.844709 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:28.844648 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-7xncj" podStartSLOduration=1.844632119 podStartE2EDuration="1.844632119s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:43:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:43:28.842573584 +0000 UTC m=+2455.573080112" watchObservedRunningTime="2026-07-09 15:43:28.844632119 +0000 UTC m=+2455.575138609" Jul 09 15:43:29.657653 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:29.657620 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-56b7f84594-t25x8_9445f493-727e-40b8-9fc6-d8ece0562e11/console/0.log" Jul 09 15:43:29.701295 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:29.701261 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-9llhr_0eb4fb70-53ef-4394-9acc-63bf16dba693/download-server/0.log" Jul 09 15:43:31.602779 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:31.602747 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sg7zr_ae72603b-84db-40aa-8fef-f79bfb5459ae/dns/0.log" Jul 09 15:43:31.632357 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:31.632316 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sg7zr_ae72603b-84db-40aa-8fef-f79bfb5459ae/kube-rbac-proxy/0.log" Jul 09 15:43:31.747487 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:31.747461 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-k9ppv_67924f4e-f69d-48b8-ba23-e4c110bb24e5/dns-node-resolver/0.log" Jul 09 15:43:32.548861 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:32.548808 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-w9rr7_ffeceed9-e851-4ef7-8ad8-603243ee5157/node-ca/0.log" Jul 09 15:43:33.688616 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:33.688585 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4j6xdb_8e7db3e0-a93f-456c-b4a3-395682060224/istio-proxy/0.log" Jul 09 15:43:33.969639 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:33.969609 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-b4hrg_62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978/istio-proxy/0.log" Jul 09 15:43:34.023765 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:34.023733 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-b4hrg_62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978/istio-proxy/1.log" Jul 09 15:43:34.177057 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:34.177022 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-9zt6p_4007bf49-82d0-4310-b0f4-21bf752c912e/payload-processing/0.log" Jul 09 15:43:34.839519 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:34.839490 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-7xncj" Jul 09 15:43:35.041351 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:35.041322 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-j4w6r_bc92e63d-1f89-4cf1-b209-a53b3f7687ff/serve-healthcheck-canary/0.log" Jul 09 15:43:35.726516 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:35.726484 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5g8vz_b3c37120-9403-437d-bb77-51a322b71fd6/kube-rbac-proxy/0.log" Jul 09 15:43:35.756911 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:35.756884 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5g8vz_b3c37120-9403-437d-bb77-51a322b71fd6/exporter/0.log" Jul 09 15:43:35.789751 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:35.789716 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5g8vz_b3c37120-9403-437d-bb77-51a322b71fd6/extractor/0.log" Jul 09 15:43:38.653136 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:38.653100 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-p44qb_026a46f5-a5f0-4cfc-95d6-a4e920ac37ff/manager/0.log" Jul 09 15:43:38.774151 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:38.774121 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4j6xdb_8e7db3e0-a93f-456c-b4a3-395682060224/istio-proxy/0.log" Jul 09 15:43:39.011951 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:39.011896 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-58d8c4744b-nrkr4_1c767830-cb4b-44a3-9b9f-3bb5f0e55ecc/manager/0.log" Jul 09 15:43:39.065593 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:39.065544 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-b4hrg_62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978/istio-proxy/0.log" Jul 09 15:43:39.117328 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:39.117290 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-b4hrg_62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978/istio-proxy/1.log" Jul 09 15:43:39.234901 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:39.234870 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66b8fbd475-29fz2_0461189c-aa86-4d9e-b865-ab27bf0830ef/manager/0.log" Jul 09 15:43:39.260126 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:39.260089 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-9zt6p_4007bf49-82d0-4310-b0f4-21bf752c912e/payload-processing/0.log" Jul 09 15:43:39.283499 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:39.283416 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66b8fbd475-29fz2_0461189c-aa86-4d9e-b865-ab27bf0830ef/copy-manifests/0.log" Jul 09 15:43:40.537723 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:40.537630 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4j6xdb_8e7db3e0-a93f-456c-b4a3-395682060224/istio-proxy/0.log" Jul 09 15:43:40.810680 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:40.810589 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7dcd8df84f-6dwp5_706796e9-a59f-4675-88d4-37545a6d864b/manager/0.log" Jul 09 15:43:40.811973 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:40.811946 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-b4hrg_62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978/istio-proxy/0.log" Jul 09 15:43:40.866353 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:40.866318 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-b4hrg_62cc6ea3-b3e5-40c5-ad1f-b2dd27e84978/istio-proxy/1.log" Jul 09 15:43:40.987781 ip-10-0-130-114 kubenswrapper[2567]: I0709 15:43:40.987749 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-9zt6p_4007bf49-82d0-4310-b0f4-21bf752c912e/payload-processing/0.log"