Jul 14 02:01:37.748975 ip-10-0-131-25 systemd[1]: Starting Kubernetes Kubelet... Jul 14 02:01:38.129883 ip-10-0-131-25 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 02:01:38.129883 ip-10-0-131-25 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 14 02:01:38.129883 ip-10-0-131-25 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 02:01:38.129883 ip-10-0-131-25 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 14 02:01:38.129883 ip-10-0-131-25 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 02:01:38.129883 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.061500 2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 14 02:01:38.129883 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068470 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 02:01:38.129883 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068491 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 02:01:38.129883 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068497 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 02:01:38.129883 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068501 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 02:01:38.129883 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068507 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 02:01:38.129883 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068514 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 02:01:38.129883 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068520 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 02:01:38.129883 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068525 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 02:01:38.129873 ip-10-0-131-25 systemd[1]: Started Kubernetes Kubelet. Jul 14 02:01:38.131209 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068530 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 02:01:38.131209 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068534 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 02:01:38.131209 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068538 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 02:01:38.131209 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068542 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 02:01:38.131209 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068546 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 02:01:38.131209 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068550 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 02:01:38.131209 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068554 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 02:01:38.131209 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068557 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 02:01:38.131209 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068561 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 02:01:38.131209 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068565 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 02:01:38.131209 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068569 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 02:01:38.131209 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068573 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 14 02:01:38.131209 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068577 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 02:01:38.131209 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068580 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 02:01:38.131209 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068585 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 02:01:38.131209 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068588 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 02:01:38.131209 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068592 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 02:01:38.131209 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068596 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 02:01:38.131209 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068599 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 02:01:38.131209 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068604 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 02:01:38.133111 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068608 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 02:01:38.133111 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068612 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 02:01:38.133111 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068617 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 02:01:38.133111 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068621 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 02:01:38.133111 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068633 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 02:01:38.133111 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068638 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 02:01:38.133111 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068642 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 02:01:38.133111 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068645 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 02:01:38.133111 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068650 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 02:01:38.133111 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068654 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 02:01:38.133111 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068658 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 02:01:38.133111 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068661 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 02:01:38.133111 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068665 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 02:01:38.133111 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068670 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 02:01:38.133111 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068673 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 02:01:38.133111 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068677 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 02:01:38.133111 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068681 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 02:01:38.133111 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068684 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 02:01:38.133111 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068688 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 02:01:38.133111 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068692 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 02:01:38.134139 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068696 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 02:01:38.134139 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068700 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 02:01:38.134139 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068704 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 02:01:38.134139 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068708 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 02:01:38.134139 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068712 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 02:01:38.134139 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068716 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 02:01:38.134139 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068728 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 02:01:38.134139 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068732 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 02:01:38.134139 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068735 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 02:01:38.134139 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068739 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 02:01:38.134139 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068745 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 02:01:38.134139 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068748 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 02:01:38.134139 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068755 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 02:01:38.134139 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068759 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 02:01:38.134139 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068764 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 02:01:38.134139 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068768 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 02:01:38.134139 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068772 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 02:01:38.134139 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068776 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 02:01:38.134139 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068781 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 02:01:38.134139 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068785 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 02:01:38.135183 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068789 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 02:01:38.135183 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068793 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 02:01:38.135183 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068797 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 02:01:38.135183 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068800 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 02:01:38.135183 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068804 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 02:01:38.135183 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068808 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 02:01:38.135183 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068813 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 02:01:38.135183 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068817 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 02:01:38.135183 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068820 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 02:01:38.135183 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068824 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 02:01:38.135183 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068828 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 02:01:38.135183 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068832 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 02:01:38.135183 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068836 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 02:01:38.135183 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068840 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 02:01:38.135183 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068844 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 02:01:38.135183 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068848 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 02:01:38.135183 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068852 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 02:01:38.135183 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068856 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 02:01:38.135183 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068860 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 02:01:38.136059 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068864 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 02:01:38.136059 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068870 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 02:01:38.136059 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068874 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 02:01:38.136059 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068878 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 02:01:38.136059 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068882 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 02:01:38.136059 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068887 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 02:01:38.136059 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068890 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 02:01:38.136059 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068894 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 02:01:38.136059 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068898 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 02:01:38.136059 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068902 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 02:01:38.136059 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068905 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 02:01:38.136059 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.068909 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 02:01:38.136059 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.069921 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 02:01:38.136059 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.069931 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 02:01:38.136059 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.069935 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 02:01:38.136059 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.069942 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 02:01:38.136059 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.069948 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 02:01:38.136059 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.069952 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 02:01:38.136059 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.069957 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 02:01:38.137036 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.069961 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 02:01:38.137036 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.069966 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 02:01:38.137036 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.069970 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 02:01:38.137036 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.069974 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 02:01:38.137036 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.069978 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 02:01:38.137036 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.069983 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 02:01:38.137036 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.069987 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 02:01:38.137036 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.069992 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 02:01:38.137036 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.069996 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 02:01:38.137036 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070000 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 02:01:38.137036 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070005 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 02:01:38.137036 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070009 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 02:01:38.137036 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070013 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 02:01:38.137036 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070017 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 02:01:38.137036 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070020 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 02:01:38.137036 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070024 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 02:01:38.137036 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070029 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 02:01:38.137036 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070033 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 02:01:38.137036 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070039 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 02:01:38.137847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070043 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 02:01:38.137847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070047 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 02:01:38.137847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070051 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 02:01:38.137847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070055 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 02:01:38.137847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070059 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 02:01:38.137847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070062 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 02:01:38.137847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070066 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 02:01:38.137847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070070 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 02:01:38.137847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070074 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 02:01:38.137847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070078 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 02:01:38.137847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070082 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 02:01:38.137847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070086 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 02:01:38.137847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070089 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 02:01:38.137847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070093 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 02:01:38.137847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070096 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 02:01:38.137847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070101 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 02:01:38.137847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070104 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 02:01:38.137847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070108 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 02:01:38.137847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070111 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 02:01:38.137847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070115 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 02:01:38.138458 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070119 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 02:01:38.138458 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070123 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 02:01:38.138458 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070127 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 02:01:38.138458 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070130 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 02:01:38.138458 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070135 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 02:01:38.138458 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070141 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 02:01:38.138458 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070145 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 02:01:38.138458 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070149 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 02:01:38.138458 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070152 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 02:01:38.138458 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070156 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 02:01:38.138458 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070161 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 02:01:38.138458 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070165 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 02:01:38.138458 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070169 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 02:01:38.138458 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070173 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 14 02:01:38.138458 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070176 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 02:01:38.138458 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070180 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 02:01:38.138458 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070184 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 02:01:38.138458 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070187 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 02:01:38.138458 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070191 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 02:01:38.138458 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070195 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 02:01:38.138458 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070199 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 02:01:38.139381 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070204 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 02:01:38.139381 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070209 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 02:01:38.139381 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070213 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 02:01:38.139381 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070219 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 02:01:38.139381 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070224 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 02:01:38.139381 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070228 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 02:01:38.139381 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070232 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 02:01:38.139381 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070235 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 02:01:38.139381 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070239 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 02:01:38.139381 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070242 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 02:01:38.139381 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070246 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 02:01:38.139381 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070249 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 02:01:38.139381 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070253 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 02:01:38.139381 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070258 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 02:01:38.139381 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070262 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 02:01:38.139381 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070283 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 02:01:38.139381 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070288 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 02:01:38.139381 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070292 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 02:01:38.139381 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070296 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 02:01:38.140307 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070299 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 02:01:38.140307 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070303 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 02:01:38.140307 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070307 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 02:01:38.140307 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070311 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 02:01:38.140307 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070315 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 02:01:38.140307 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070318 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 02:01:38.140307 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070323 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 02:01:38.140307 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070326 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 02:01:38.140307 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070330 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 02:01:38.140307 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070334 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 02:01:38.140307 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070338 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 02:01:38.140307 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070341 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 02:01:38.140307 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070345 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 02:01:38.140307 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070882 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 02:01:38.140307 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070888 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 02:01:38.140307 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070893 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 02:01:38.140307 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070897 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 02:01:38.140307 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070902 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 02:01:38.140307 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070906 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 02:01:38.140307 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070911 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 02:01:38.141157 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070915 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 14 02:01:38.141157 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070918 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 02:01:38.141157 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070922 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 02:01:38.141157 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070926 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 02:01:38.141157 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.070997 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 02:01:38.141157 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071004 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 02:01:38.141157 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071009 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 02:01:38.141157 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071014 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 02:01:38.141157 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071019 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 02:01:38.141157 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071024 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 02:01:38.141157 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071029 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 02:01:38.141157 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071033 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 02:01:38.141157 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071037 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 02:01:38.141157 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071041 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 02:01:38.141157 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071045 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 02:01:38.141157 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071049 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 02:01:38.141157 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071053 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 02:01:38.141157 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071057 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 02:01:38.141157 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071060 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 02:01:38.141898 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071064 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 02:01:38.141898 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071067 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 02:01:38.141898 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071071 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 02:01:38.141898 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071075 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 02:01:38.141898 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071078 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 02:01:38.141898 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071082 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 02:01:38.141898 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071086 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 02:01:38.141898 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071090 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 02:01:38.141898 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071094 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 02:01:38.141898 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071097 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 02:01:38.141898 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071101 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 02:01:38.141898 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071109 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 02:01:38.141898 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071115 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 02:01:38.141898 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071118 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 02:01:38.141898 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071124 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 02:01:38.141898 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071129 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 02:01:38.141898 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071133 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 02:01:38.141898 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071138 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 02:01:38.141898 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071142 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 02:01:38.142691 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071146 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 02:01:38.142691 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071150 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 02:01:38.142691 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071154 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 02:01:38.142691 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071158 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 02:01:38.142691 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071164 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 02:01:38.142691 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071169 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 02:01:38.142691 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071173 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 02:01:38.142691 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071177 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 02:01:38.142691 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071181 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 02:01:38.142691 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071184 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 02:01:38.142691 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071188 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 02:01:38.142691 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071191 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 02:01:38.142691 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071196 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 02:01:38.142691 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071199 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 02:01:38.142691 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071205 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 02:01:38.142691 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071210 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 02:01:38.142691 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071214 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 02:01:38.142691 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071221 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 02:01:38.142691 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071225 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 02:01:38.142691 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071229 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 02:01:38.143644 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071233 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 02:01:38.143644 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071237 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 02:01:38.143644 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071241 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 02:01:38.143644 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071245 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 02:01:38.143644 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071251 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 02:01:38.143644 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071259 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 02:01:38.143644 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071265 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 02:01:38.143644 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071286 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 02:01:38.143644 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071290 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 02:01:38.143644 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071295 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 02:01:38.143644 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071299 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 02:01:38.143644 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071303 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 02:01:38.143644 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071307 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 02:01:38.143644 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071310 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 02:01:38.143644 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071314 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 02:01:38.143644 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071318 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 02:01:38.143644 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071322 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 02:01:38.143644 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071326 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 02:01:38.143644 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071330 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 02:01:38.143644 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071333 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 02:01:38.144518 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071337 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 02:01:38.144518 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071340 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 02:01:38.144518 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071344 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 02:01:38.144518 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071348 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 02:01:38.144518 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071352 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 02:01:38.144518 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071356 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 02:01:38.144518 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071359 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 02:01:38.144518 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071362 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 02:01:38.144518 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071366 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 02:01:38.144518 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071370 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 02:01:38.144518 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071374 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 02:01:38.144518 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071377 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 02:01:38.144518 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071381 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 02:01:38.144518 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.071385 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 02:01:38.144518 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071470 2577 flags.go:64] FLAG: --address="0.0.0.0" Jul 14 02:01:38.144518 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071480 2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 14 02:01:38.144518 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071499 2577 flags.go:64] FLAG: --anonymous-auth="true" Jul 14 02:01:38.144518 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071507 2577 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 14 02:01:38.144518 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071513 2577 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 14 02:01:38.144518 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071518 2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 14 02:01:38.144518 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071524 2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 14 02:01:38.145850 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071530 2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 14 02:01:38.145850 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071535 2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 14 02:01:38.145850 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071540 2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 14 02:01:38.145850 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071545 2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 14 02:01:38.145850 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071550 2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 14 02:01:38.145850 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071554 2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 14 02:01:38.145850 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071560 2577 flags.go:64] FLAG: --cgroup-root="" Jul 14 02:01:38.145850 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071564 2577 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 14 02:01:38.145850 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071569 2577 flags.go:64] FLAG: --client-ca-file="" Jul 14 02:01:38.145850 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071576 2577 flags.go:64] FLAG: --cloud-provider="external" Jul 14 02:01:38.145850 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071581 2577 flags.go:64] FLAG: --cluster-dns="[]" Jul 14 02:01:38.145850 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071587 2577 flags.go:64] FLAG: --cluster-domain="" Jul 14 02:01:38.145850 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071591 2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 14 02:01:38.145850 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071596 2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 14 02:01:38.145850 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071600 2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 14 02:01:38.145850 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071605 2577 flags.go:64] FLAG: --container-log-max-files="5" Jul 14 02:01:38.145850 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071610 2577 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 14 02:01:38.145850 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071615 2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 14 02:01:38.145850 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071619 2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 14 02:01:38.145850 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071624 2577 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 14 02:01:38.145850 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071628 2577 flags.go:64] FLAG: --contention-profiling="false" Jul 14 02:01:38.145850 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071633 2577 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 14 02:01:38.145850 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071637 2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 14 02:01:38.145850 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071642 2577 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 14 02:01:38.146678 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071647 2577 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 14 02:01:38.146678 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071652 2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 14 02:01:38.146678 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071656 2577 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 14 02:01:38.146678 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071662 2577 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 14 02:01:38.146678 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071669 2577 flags.go:64] FLAG: --enable-load-reader="false" Jul 14 02:01:38.146678 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071673 2577 flags.go:64] FLAG: --enable-server="true" Jul 14 02:01:38.146678 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071677 2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 14 02:01:38.146678 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071683 2577 flags.go:64] FLAG: --event-burst="100" Jul 14 02:01:38.146678 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071688 2577 flags.go:64] FLAG: --event-qps="50" Jul 14 02:01:38.146678 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071692 2577 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 14 02:01:38.146678 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071696 2577 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 14 02:01:38.146678 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071701 2577 flags.go:64] FLAG: --eviction-hard="" Jul 14 02:01:38.146678 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071706 2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 14 02:01:38.146678 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071710 2577 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 14 02:01:38.146678 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071714 2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 14 02:01:38.146678 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071719 2577 flags.go:64] FLAG: --eviction-soft="" Jul 14 02:01:38.146678 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071723 2577 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 14 02:01:38.146678 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071728 2577 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 14 02:01:38.146678 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071733 2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 14 02:01:38.146678 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071737 2577 flags.go:64] FLAG: --experimental-mounter-path="" Jul 14 02:01:38.146678 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071741 2577 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 14 02:01:38.146678 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071746 2577 flags.go:64] FLAG: --fail-swap-on="true" Jul 14 02:01:38.146678 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071750 2577 flags.go:64] FLAG: --feature-gates="" Jul 14 02:01:38.146678 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071756 2577 flags.go:64] FLAG: --file-check-frequency="20s" Jul 14 02:01:38.146678 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071760 2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 14 02:01:38.147505 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071765 2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 14 02:01:38.147505 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071770 2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 14 02:01:38.147505 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071774 2577 flags.go:64] FLAG: --healthz-port="10248" Jul 14 02:01:38.147505 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071778 2577 flags.go:64] FLAG: --help="false" Jul 14 02:01:38.147505 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071782 2577 flags.go:64] FLAG: --hostname-override="ip-10-0-131-25.ec2.internal" Jul 14 02:01:38.147505 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071787 2577 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 14 02:01:38.147505 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071791 2577 flags.go:64] FLAG: --http-check-frequency="20s" Jul 14 02:01:38.147505 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071795 2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 14 02:01:38.147505 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071800 2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 14 02:01:38.147505 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071805 2577 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 14 02:01:38.147505 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071810 2577 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 14 02:01:38.147505 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071816 2577 flags.go:64] FLAG: --image-service-endpoint="" Jul 14 02:01:38.147505 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071820 2577 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 14 02:01:38.147505 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071825 2577 flags.go:64] FLAG: --kube-api-burst="100" Jul 14 02:01:38.147505 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071829 2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 14 02:01:38.147505 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071833 2577 flags.go:64] FLAG: --kube-api-qps="50" Jul 14 02:01:38.147505 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071838 2577 flags.go:64] FLAG: --kube-reserved="" Jul 14 02:01:38.147505 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071842 2577 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 14 02:01:38.147505 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071846 2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 14 02:01:38.147505 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071851 2577 flags.go:64] FLAG: --kubelet-cgroups="" Jul 14 02:01:38.147505 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071855 2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 14 02:01:38.147505 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071859 2577 flags.go:64] FLAG: --lock-file="" Jul 14 02:01:38.147505 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071863 2577 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 14 02:01:38.147505 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071868 2577 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 14 02:01:38.148656 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071872 2577 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 14 02:01:38.148656 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071880 2577 flags.go:64] FLAG: --log-json-split-stream="false" Jul 14 02:01:38.148656 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071885 2577 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 14 02:01:38.148656 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071889 2577 flags.go:64] FLAG: --log-text-split-stream="false" Jul 14 02:01:38.148656 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071893 2577 flags.go:64] FLAG: --logging-format="text" Jul 14 02:01:38.148656 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071897 2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 14 02:01:38.148656 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071903 2577 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 14 02:01:38.148656 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071907 2577 flags.go:64] FLAG: --manifest-url="" Jul 14 02:01:38.148656 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071911 2577 flags.go:64] FLAG: --manifest-url-header="" Jul 14 02:01:38.148656 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071917 2577 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 14 02:01:38.148656 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071921 2577 flags.go:64] FLAG: --max-open-files="1000000" Jul 14 02:01:38.148656 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071927 2577 flags.go:64] FLAG: --max-pods="110" Jul 14 02:01:38.148656 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071932 2577 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 14 02:01:38.148656 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071936 2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 14 02:01:38.148656 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071940 2577 flags.go:64] FLAG: --memory-manager-policy="None" Jul 14 02:01:38.148656 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071945 2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 14 02:01:38.148656 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071949 2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 14 02:01:38.148656 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071955 2577 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 14 02:01:38.148656 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071962 2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 14 02:01:38.148656 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071975 2577 flags.go:64] FLAG: --node-status-max-images="50" Jul 14 02:01:38.148656 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071979 2577 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 14 02:01:38.148656 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071983 2577 flags.go:64] FLAG: --oom-score-adj="-999" Jul 14 02:01:38.148656 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071987 2577 flags.go:64] FLAG: --pod-cidr="" Jul 14 02:01:38.149621 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071992 2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 14 02:01:38.149621 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.071999 2577 flags.go:64] FLAG: --pod-manifest-path="" Jul 14 02:01:38.149621 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072004 2577 flags.go:64] FLAG: --pod-max-pids="-1" Jul 14 02:01:38.149621 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072008 2577 flags.go:64] FLAG: --pods-per-core="0" Jul 14 02:01:38.149621 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072012 2577 flags.go:64] FLAG: --port="10250" Jul 14 02:01:38.149621 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072016 2577 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 14 02:01:38.149621 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072020 2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08282b987f87bdd56" Jul 14 02:01:38.149621 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072025 2577 flags.go:64] FLAG: --qos-reserved="" Jul 14 02:01:38.149621 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072030 2577 flags.go:64] FLAG: --read-only-port="10255" Jul 14 02:01:38.149621 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072034 2577 flags.go:64] FLAG: --register-node="true" Jul 14 02:01:38.149621 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072038 2577 flags.go:64] FLAG: --register-with-taints="" Jul 14 02:01:38.149621 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072044 2577 flags.go:64] FLAG: --registry-burst="10" Jul 14 02:01:38.149621 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072049 2577 flags.go:64] FLAG: --registry-qps="5" Jul 14 02:01:38.149621 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072053 2577 flags.go:64] FLAG: --reserved-cpus="" Jul 14 02:01:38.149621 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072057 2577 flags.go:64] FLAG: --reserved-memory="" Jul 14 02:01:38.149621 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072062 2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 14 02:01:38.149621 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072067 2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 14 02:01:38.149621 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072071 2577 flags.go:64] FLAG: --rotate-certificates="false" Jul 14 02:01:38.149621 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072076 2577 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 14 02:01:38.149621 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072080 2577 flags.go:64] FLAG: --runonce="false" Jul 14 02:01:38.149621 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072084 2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 14 02:01:38.149621 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072089 2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 14 02:01:38.149621 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072093 2577 flags.go:64] FLAG: --seccomp-default="false" Jul 14 02:01:38.149621 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072097 2577 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 14 02:01:38.149621 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072102 2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 14 02:01:38.150938 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072106 2577 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 14 02:01:38.150938 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072110 2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 14 02:01:38.150938 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072115 2577 flags.go:64] FLAG: --storage-driver-password="root" Jul 14 02:01:38.150938 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072122 2577 flags.go:64] FLAG: --storage-driver-secure="false" Jul 14 02:01:38.150938 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072126 2577 flags.go:64] FLAG: --storage-driver-table="stats" Jul 14 02:01:38.150938 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072131 2577 flags.go:64] FLAG: --storage-driver-user="root" Jul 14 02:01:38.150938 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072135 2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 14 02:01:38.150938 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072139 2577 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 14 02:01:38.150938 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072143 2577 flags.go:64] FLAG: --system-cgroups="" Jul 14 02:01:38.150938 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072148 2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 14 02:01:38.150938 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072156 2577 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 14 02:01:38.150938 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072160 2577 flags.go:64] FLAG: --tls-cert-file="" Jul 14 02:01:38.150938 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072164 2577 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 14 02:01:38.150938 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072171 2577 flags.go:64] FLAG: --tls-min-version="" Jul 14 02:01:38.150938 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072175 2577 flags.go:64] FLAG: --tls-private-key-file="" Jul 14 02:01:38.150938 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072179 2577 flags.go:64] FLAG: --topology-manager-policy="none" Jul 14 02:01:38.150938 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072184 2577 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 14 02:01:38.150938 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072188 2577 flags.go:64] FLAG: --topology-manager-scope="container" Jul 14 02:01:38.150938 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072192 2577 flags.go:64] FLAG: --v="2" Jul 14 02:01:38.150938 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072198 2577 flags.go:64] FLAG: --version="false" Jul 14 02:01:38.150938 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072205 2577 flags.go:64] FLAG: --vmodule="" Jul 14 02:01:38.150938 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072210 2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 14 02:01:38.150938 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.072215 2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 14 02:01:38.150938 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072394 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 02:01:38.150938 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072399 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 02:01:38.152077 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072403 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 02:01:38.152077 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072407 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 02:01:38.152077 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072411 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 02:01:38.152077 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072415 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 02:01:38.152077 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072419 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 02:01:38.152077 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072423 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 02:01:38.152077 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072426 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 02:01:38.152077 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072430 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 02:01:38.152077 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072434 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 02:01:38.152077 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072438 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 02:01:38.152077 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072445 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 02:01:38.152077 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072449 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 02:01:38.152077 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072453 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 02:01:38.152077 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072457 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 02:01:38.152077 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072461 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 02:01:38.152077 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072465 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 02:01:38.152077 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072469 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 14 02:01:38.152077 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072473 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 02:01:38.152077 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072477 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 02:01:38.152077 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072480 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 02:01:38.153103 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072484 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 02:01:38.153103 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072487 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 02:01:38.153103 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072491 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 02:01:38.153103 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072495 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 02:01:38.153103 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072498 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 02:01:38.153103 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072502 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 02:01:38.153103 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072506 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 02:01:38.153103 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072510 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 02:01:38.153103 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072514 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 02:01:38.153103 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072518 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 02:01:38.153103 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072522 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 02:01:38.153103 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072526 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 02:01:38.153103 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072529 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 02:01:38.153103 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072533 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 02:01:38.153103 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072537 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 02:01:38.153103 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072541 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 02:01:38.153103 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072544 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 02:01:38.153103 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072548 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 02:01:38.153103 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072551 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 02:01:38.153103 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072556 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 02:01:38.154149 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072560 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 02:01:38.154149 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072563 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 02:01:38.154149 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072570 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 02:01:38.154149 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072574 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 02:01:38.154149 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072578 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 02:01:38.154149 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072582 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 02:01:38.154149 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072585 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 02:01:38.154149 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072589 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 02:01:38.154149 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072593 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 02:01:38.154149 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072597 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 02:01:38.154149 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072601 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 02:01:38.154149 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072604 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 02:01:38.154149 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072608 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 02:01:38.154149 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072612 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 02:01:38.154149 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072616 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 02:01:38.154149 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072623 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 02:01:38.154149 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072628 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 02:01:38.154149 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072633 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 02:01:38.154149 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072637 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 02:01:38.154775 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072641 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 02:01:38.154775 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072645 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 02:01:38.154775 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072650 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 02:01:38.154775 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072654 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 02:01:38.154775 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072658 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 02:01:38.154775 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072662 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 02:01:38.154775 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072666 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 02:01:38.154775 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072670 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 02:01:38.154775 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072674 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 02:01:38.154775 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072677 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 02:01:38.154775 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072681 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 02:01:38.154775 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072687 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 02:01:38.154775 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072691 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 02:01:38.154775 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072696 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 02:01:38.154775 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072700 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 02:01:38.154775 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072707 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 02:01:38.154775 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072712 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 02:01:38.154775 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072716 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 02:01:38.154775 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072731 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 02:01:38.155357 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072734 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 02:01:38.155357 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072740 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 02:01:38.155357 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072745 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 02:01:38.155357 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072749 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 02:01:38.155357 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072753 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 02:01:38.155357 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072758 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 02:01:38.155357 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072761 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 02:01:38.155357 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072765 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 02:01:38.155357 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072770 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 02:01:38.155357 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072774 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 02:01:38.155357 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072777 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 02:01:38.155357 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072781 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 02:01:38.155357 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072785 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 02:01:38.155357 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072788 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 02:01:38.155357 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072792 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 02:01:38.155357 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072796 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 02:01:38.155357 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072800 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 02:01:38.155357 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072804 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 02:01:38.155357 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.072807 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 02:01:38.320443 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.073281 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 02:01:38.320443 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.079821 2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 14 02:01:38.320443 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.079838 2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 14 02:01:38.320443 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.079852 2577 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 14 02:01:38.320443 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.079863 2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 14 02:01:38.320443 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.079919 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 02:01:38.320443 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.079925 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 02:01:38.320443 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.079928 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 02:01:38.320443 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.079931 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 02:01:38.320443 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.079936 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 02:01:38.320443 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.079940 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 02:01:38.320443 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.079944 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 02:01:38.320443 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.079947 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 02:01:38.320443 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.079950 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 02:01:38.345850 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.079953 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 02:01:38.345850 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.079955 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 02:01:38.345850 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.079958 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 14 02:01:38.345850 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.079961 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 02:01:38.345850 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.079964 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 02:01:38.345850 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.079967 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 02:01:38.345850 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.079969 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 02:01:38.345850 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.079972 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 02:01:38.345850 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.079974 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 02:01:38.345850 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.079977 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 02:01:38.345850 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.079979 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 02:01:38.345850 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.079982 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 02:01:38.345850 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.079985 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 02:01:38.345850 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.079988 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 02:01:38.345850 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.079990 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 02:01:38.345850 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.079993 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 02:01:38.345850 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.079997 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 02:01:38.345850 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080000 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 02:01:38.345850 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080002 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 02:01:38.345850 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080005 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 02:01:38.346617 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080008 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 02:01:38.346617 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080011 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 02:01:38.346617 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080017 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 02:01:38.346617 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080019 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 02:01:38.346617 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080022 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 02:01:38.346617 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080025 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 02:01:38.346617 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080027 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 02:01:38.346617 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080030 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 02:01:38.346617 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080033 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 02:01:38.346617 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080035 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 02:01:38.346617 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080038 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 02:01:38.346617 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080043 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 02:01:38.346617 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080047 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 02:01:38.346617 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080051 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 02:01:38.346617 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080054 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 02:01:38.346617 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080057 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 02:01:38.346617 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080060 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 02:01:38.346617 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080064 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 02:01:38.346617 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080066 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 02:01:38.346617 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080069 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 02:01:38.347333 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080071 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 02:01:38.347333 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080074 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 02:01:38.347333 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080077 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 02:01:38.347333 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080079 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 02:01:38.347333 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080082 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 02:01:38.347333 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080085 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 02:01:38.347333 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080087 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 02:01:38.347333 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080089 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 02:01:38.347333 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080092 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 02:01:38.347333 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080095 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 02:01:38.347333 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080097 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 02:01:38.347333 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080101 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 02:01:38.347333 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080105 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 02:01:38.347333 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080109 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 02:01:38.347333 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080113 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 02:01:38.347333 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080116 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 02:01:38.347333 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080119 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 02:01:38.347333 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080122 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 02:01:38.347333 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080125 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 02:01:38.348009 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080128 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 02:01:38.348009 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080131 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 02:01:38.348009 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080134 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 02:01:38.348009 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080136 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 02:01:38.348009 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080141 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 02:01:38.348009 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080143 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 02:01:38.348009 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080146 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 02:01:38.348009 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080149 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 02:01:38.348009 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080151 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 02:01:38.348009 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080154 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 02:01:38.348009 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080156 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 02:01:38.348009 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080159 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 02:01:38.348009 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080161 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 02:01:38.348009 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080164 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 02:01:38.348009 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080166 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 02:01:38.348009 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080169 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 02:01:38.348009 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080171 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 02:01:38.348009 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080174 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 02:01:38.348009 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080176 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 02:01:38.348009 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080179 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 02:01:38.348718 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080181 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 02:01:38.348718 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080184 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 02:01:38.348718 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080186 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 02:01:38.348718 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080188 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 02:01:38.348718 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080191 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 02:01:38.348718 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080194 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 02:01:38.348718 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080196 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 02:01:38.348718 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080200 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 02:01:38.348718 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080203 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 02:01:38.348718 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080206 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 02:01:38.348718 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080209 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 02:01:38.348718 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.080213 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 02:01:38.348718 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080350 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 02:01:38.348718 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080355 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 02:01:38.348718 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080358 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 02:01:38.349263 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080362 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 02:01:38.349263 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080365 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 02:01:38.349263 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080371 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 02:01:38.349263 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080374 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 02:01:38.349263 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080377 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 02:01:38.349263 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080380 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 02:01:38.349263 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080383 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 02:01:38.349263 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080385 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 02:01:38.349263 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080388 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 02:01:38.349263 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080390 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 02:01:38.349263 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080393 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 02:01:38.349263 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080396 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 02:01:38.349263 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080398 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 02:01:38.349263 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080401 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 02:01:38.349263 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080403 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 02:01:38.349263 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080406 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 02:01:38.349263 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080408 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 02:01:38.349263 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080411 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 14 02:01:38.349263 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080414 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 02:01:38.349958 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080418 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 02:01:38.349958 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080421 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 02:01:38.349958 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080424 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 02:01:38.349958 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080426 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 02:01:38.349958 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080429 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 02:01:38.349958 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080433 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 02:01:38.349958 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080436 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 02:01:38.349958 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080438 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 02:01:38.349958 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080441 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 02:01:38.349958 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080444 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 02:01:38.349958 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080446 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 02:01:38.349958 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080449 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 02:01:38.349958 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080451 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 02:01:38.349958 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080454 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 02:01:38.349958 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080456 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 02:01:38.349958 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080460 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 02:01:38.349958 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080463 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 02:01:38.349958 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080465 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 02:01:38.349958 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080468 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 02:01:38.349958 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080470 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 02:01:38.350847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080473 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 02:01:38.350847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080476 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 02:01:38.350847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080479 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 02:01:38.350847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080481 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 02:01:38.350847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080484 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 02:01:38.350847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080487 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 02:01:38.350847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080490 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 02:01:38.350847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080492 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 02:01:38.350847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080495 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 02:01:38.350847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080497 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 02:01:38.350847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080500 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 02:01:38.350847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080502 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 02:01:38.350847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080505 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 02:01:38.350847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080507 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 02:01:38.350847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080510 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 02:01:38.350847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080512 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 02:01:38.350847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080515 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 02:01:38.350847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080520 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 02:01:38.350847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080522 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 02:01:38.350847 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080525 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 02:01:38.351965 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080528 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 02:01:38.351965 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080531 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 02:01:38.351965 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080535 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 02:01:38.351965 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080537 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 02:01:38.351965 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080540 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 02:01:38.351965 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080542 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 02:01:38.351965 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080545 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 02:01:38.351965 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080549 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 02:01:38.351965 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080551 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 02:01:38.351965 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080554 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 02:01:38.351965 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080556 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 02:01:38.351965 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080559 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 02:01:38.351965 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080561 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 02:01:38.351965 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080564 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 02:01:38.351965 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080566 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 02:01:38.351965 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080569 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 02:01:38.351965 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080572 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 02:01:38.351965 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080574 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 02:01:38.351965 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080578 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 02:01:38.352625 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080581 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 02:01:38.352625 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080584 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 02:01:38.352625 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080587 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 02:01:38.352625 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080590 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 02:01:38.352625 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080593 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 02:01:38.352625 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080595 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 02:01:38.352625 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080598 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 02:01:38.352625 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080601 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 02:01:38.352625 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080604 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 02:01:38.352625 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080606 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 02:01:38.352625 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080612 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 02:01:38.352625 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080614 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 02:01:38.352625 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080617 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 02:01:38.352625 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080620 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 02:01:38.352625 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080622 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 02:01:38.352625 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080625 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 02:01:38.352625 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080627 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 02:01:38.352625 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:38.080630 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 02:01:38.353703 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.080635 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 02:01:38.353703 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.080766 2577 server.go:962] "Client rotation is on, will bootstrap in background" Jul 14 02:01:38.353703 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.083211 2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 14 02:01:38.353703 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.084297 2577 server.go:1019] "Starting client certificate rotation" Jul 14 02:01:38.353703 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.084877 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 02:01:38.353703 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.085433 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 02:01:38.353703 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.086242 2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 02:01:38.353703 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.089264 2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 02:01:38.353703 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.098255 2577 log.go:25] "Validated CRI v1 runtime API" Jul 14 02:01:38.353703 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.100607 2577 log.go:25] "Validated CRI v1 image API" Jul 14 02:01:38.353703 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.101373 2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 14 02:01:38.353703 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.103394 2577 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 cd929b6d-af44-4cea-8c74-31928c8b1505:/dev/nvme0n1p3 d881e355-c9b1-4753-8d9b-8590acbb2298:/dev/nvme0n1p4] Jul 14 02:01:38.354144 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.103411 2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 14 02:01:38.354263 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.108002 2577 manager.go:216] Machine: {Timestamp:2026-07-14 02:01:38.106837349 +0000 UTC m=+0.261673987 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102942 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec237575bc855e34f4a81b1e2fa8220d SystemUUID:ec237575-bc85-5e34-f4a8-1b1e2fa8220d BootID:c33465a6-19c9-4cfb-9550-b387b9e82e48 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:90:92:c1:93:c3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:90:92:c1:93:c3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:36:74:3f:c0:5e:ba Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 14 02:01:38.354263 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.108109 2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 14 02:01:38.354263 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.108236 2577 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 14 02:01:38.354263 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.110609 2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 14 02:01:38.354263 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.110637 2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-25.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 14 02:01:38.354263 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.111240 2577 topology_manager.go:138] "Creating topology manager with none policy" Jul 14 02:01:38.354263 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.111250 2577 container_manager_linux.go:309] "Creating device plugin manager" Jul 14 02:01:38.354263 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.111258 2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 02:01:38.354707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.111746 2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 02:01:38.354707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.113821 2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 14 02:01:38.354707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.119515 2577 state_mem.go:36] "Initialized new in-memory state store" Jul 14 02:01:38.354707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.119648 2577 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 14 02:01:38.354707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.119991 2577 kubelet.go:486] "Attempting to sync node with API server" Jul 14 02:01:38.354707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.120009 2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 14 02:01:38.354707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.120024 2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 14 02:01:38.354707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.120033 2577 kubelet.go:398] "Adding apiserver pod source" Jul 14 02:01:38.354707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.120043 2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 14 02:01:38.354707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.120536 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 02:01:38.354707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.120569 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 02:01:38.354707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.122445 2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 14 02:01:38.354707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.122474 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 02:01:38.354707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.123826 2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 14 02:01:38.354707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.123858 2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 14 02:01:38.354707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.124388 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 14 02:01:38.354707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.124401 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 14 02:01:38.354707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.124409 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 14 02:01:38.354707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.124414 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 14 02:01:38.354707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.124422 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 14 02:01:38.354707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.124428 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 14 02:01:38.354707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.124435 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 14 02:01:38.354707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.124440 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 14 02:01:38.354707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.124447 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 14 02:01:38.354707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.124452 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 14 02:01:38.354707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.124467 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 14 02:01:38.354707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.124472 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 14 02:01:38.354707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.125089 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 14 02:01:38.354707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.125097 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 14 02:01:38.354707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.129030 2577 server.go:1268] "Started kubelet" Jul 14 02:01:38.354707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.129595 2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 14 02:01:38.354707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.131389 2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 14 02:01:38.354707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.131456 2577 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 14 02:01:38.354707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.132249 2577 server.go:310] "Adding debug handlers to kubelet server" Jul 14 02:01:38.354707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.132452 2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 14 02:01:38.354707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.133132 2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-25.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 14 02:01:38.354707 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:38.133824 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-25.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 14 02:01:38.354707 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:38.133906 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 14 02:01:38.355891 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:38.136292 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-25.ec2.internal.18c2058b463af207 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-25.ec2.internal,UID:ip-10-0-131-25.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-25.ec2.internal,},FirstTimestamp:2026-07-14 02:01:38.128998919 +0000 UTC m=+0.283835592,LastTimestamp:2026-07-14 02:01:38.128998919 +0000 UTC m=+0.283835592,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-25.ec2.internal,}" Jul 14 02:01:38.355891 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:38.137956 2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 14 02:01:38.355891 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.138519 2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 14 02:01:38.355891 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.138533 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 14 02:01:38.355891 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:38.138596 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-25.ec2.internal\" not found" Jul 14 02:01:38.355891 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.139002 2577 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 14 02:01:38.355891 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.139008 2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 14 02:01:38.355891 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.139018 2577 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 14 02:01:38.355891 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.139123 2577 reconstruct.go:97] "Volume reconstruction finished" Jul 14 02:01:38.355891 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.139130 2577 reconciler.go:29] "Reconciler: start to sync state" Jul 14 02:01:38.355891 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.139522 2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 14 02:01:38.355891 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.139535 2577 factory.go:55] Registering systemd factory Jul 14 02:01:38.355891 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.139544 2577 factory.go:223] Registration of the systemd container factory successfully Jul 14 02:01:38.355891 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.139769 2577 factory.go:153] Registering CRI-O factory Jul 14 02:01:38.355891 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.139776 2577 factory.go:223] Registration of the crio container factory successfully Jul 14 02:01:38.355891 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.139793 2577 factory.go:103] Registering Raw factory Jul 14 02:01:38.355891 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.139804 2577 manager.go:1196] Started watching for new ooms in manager Jul 14 02:01:38.355891 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.140612 2577 manager.go:349] Starting recovery of all containers Jul 14 02:01:38.355891 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:38.140619 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 14 02:01:38.355891 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:38.140657 2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-25.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 14 02:01:38.355891 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.152904 2577 manager.go:354] Recovery completed Jul 14 02:01:38.355891 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.154212 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-b6mlj" Jul 14 02:01:38.355891 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.157009 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 02:01:38.355891 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.159295 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-b6mlj" Jul 14 02:01:38.355891 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.159572 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-25.ec2.internal" event="NodeHasSufficientMemory" Jul 14 02:01:38.355891 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.159599 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-25.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 02:01:38.355891 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.159613 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-25.ec2.internal" event="NodeHasSufficientPID" Jul 14 02:01:38.355891 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.160174 2577 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 14 02:01:38.355891 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.160187 2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 14 02:01:38.355891 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.160205 2577 state_mem.go:36] "Initialized new in-memory state store" Jul 14 02:01:38.355891 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.160229 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 02:01:38.355891 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.160926 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-25.ec2.internal" event="NodeHasSufficientMemory" Jul 14 02:01:38.356890 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.160949 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-25.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 02:01:38.356890 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.160961 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-25.ec2.internal" event="NodeHasSufficientPID" Jul 14 02:01:38.356890 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.162440 2577 policy_none.go:49] "None policy: Start" Jul 14 02:01:38.356890 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.162456 2577 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 14 02:01:38.356890 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.162469 2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 14 02:01:38.356890 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.164453 2577 policy_none.go:47] "Start" Jul 14 02:01:38.356890 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.210049 2577 manager.go:337] "Starting Device Plugin manager" Jul 14 02:01:38.356890 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:38.210085 2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 14 02:01:38.356890 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.210096 2577 server.go:87] "Starting device plugin registration server" Jul 14 02:01:38.356890 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.210429 2577 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 14 02:01:38.356890 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.210446 2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 14 02:01:38.356890 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.210551 2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 14 02:01:38.356890 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.210625 2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 14 02:01:38.356890 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.210633 2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 14 02:01:38.356890 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:38.211196 2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 14 02:01:38.356890 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:38.211243 2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-25.ec2.internal\" not found" Jul 14 02:01:38.356890 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.256840 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 14 02:01:38.356890 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.258136 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 14 02:01:38.356890 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.258183 2577 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 14 02:01:38.356890 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.258211 2577 kubelet.go:2448] "Starting kubelet main sync loop" Jul 14 02:01:38.356890 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:38.258255 2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 14 02:01:38.356890 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.260740 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 02:01:38.356890 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.311701 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 02:01:38.356890 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.312732 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-25.ec2.internal" event="NodeHasSufficientMemory" Jul 14 02:01:38.356890 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.312763 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-25.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 02:01:38.356890 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.312775 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-25.ec2.internal" event="NodeHasSufficientPID" Jul 14 02:01:38.356890 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.312801 2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-25.ec2.internal" Jul 14 02:01:38.356890 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.321481 2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-25.ec2.internal" Jul 14 02:01:38.356890 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:38.321503 2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-25.ec2.internal\": node \"ip-10-0-131-25.ec2.internal\" not found" Jul 14 02:01:38.356890 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:38.341114 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-25.ec2.internal\" not found" Jul 14 02:01:38.559660 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.358360 2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-131-25.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-25.ec2.internal"] Jul 14 02:01:38.559660 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.358431 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 02:01:38.559660 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.359961 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-25.ec2.internal" event="NodeHasSufficientMemory" Jul 14 02:01:38.559660 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.359990 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-25.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 02:01:38.559660 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.360002 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-25.ec2.internal" event="NodeHasSufficientPID" Jul 14 02:01:38.559660 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.362492 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 02:01:38.559660 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.362623 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-25.ec2.internal" Jul 14 02:01:38.559660 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.362654 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 02:01:38.559660 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.363218 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-25.ec2.internal" event="NodeHasSufficientMemory" Jul 14 02:01:38.559660 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.363226 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-25.ec2.internal" event="NodeHasSufficientMemory" Jul 14 02:01:38.559660 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.363246 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-25.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 02:01:38.559660 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.363246 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-25.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 02:01:38.559660 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.363257 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-25.ec2.internal" event="NodeHasSufficientPID" Jul 14 02:01:38.559660 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.363264 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-25.ec2.internal" event="NodeHasSufficientPID" Jul 14 02:01:38.559660 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.365679 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-25.ec2.internal" Jul 14 02:01:38.559660 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.365704 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 02:01:38.559660 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.366364 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-25.ec2.internal" event="NodeHasSufficientMemory" Jul 14 02:01:38.559660 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.366388 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-25.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 02:01:38.559660 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.366402 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-25.ec2.internal" event="NodeHasSufficientPID" Jul 14 02:01:38.559660 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:38.402178 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-25.ec2.internal\" not found" node="ip-10-0-131-25.ec2.internal" Jul 14 02:01:38.559660 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:38.405979 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-25.ec2.internal\" not found" node="ip-10-0-131-25.ec2.internal" Jul 14 02:01:38.559660 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:38.441292 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-25.ec2.internal\" not found" Jul 14 02:01:38.559660 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.441350 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/68e660ef911ade3343c0551cb4b95404-config\") pod \"kube-apiserver-proxy-ip-10-0-131-25.ec2.internal\" (UID: \"68e660ef911ade3343c0551cb4b95404\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-25.ec2.internal" Jul 14 02:01:38.559660 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.441376 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/61db6160ce2d2c08f4b14efb99d52013-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-25.ec2.internal\" (UID: \"61db6160ce2d2c08f4b14efb99d52013\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-25.ec2.internal" Jul 14 02:01:38.559660 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.441403 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61db6160ce2d2c08f4b14efb99d52013-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-25.ec2.internal\" (UID: \"61db6160ce2d2c08f4b14efb99d52013\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-25.ec2.internal" Jul 14 02:01:38.559660 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:38.542028 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-25.ec2.internal\" not found" Jul 14 02:01:38.561148 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.542101 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61db6160ce2d2c08f4b14efb99d52013-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-25.ec2.internal\" (UID: \"61db6160ce2d2c08f4b14efb99d52013\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-25.ec2.internal" Jul 14 02:01:38.561148 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.542144 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/68e660ef911ade3343c0551cb4b95404-config\") pod \"kube-apiserver-proxy-ip-10-0-131-25.ec2.internal\" (UID: \"68e660ef911ade3343c0551cb4b95404\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-25.ec2.internal" Jul 14 02:01:38.561148 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.542181 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/61db6160ce2d2c08f4b14efb99d52013-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-25.ec2.internal\" (UID: \"61db6160ce2d2c08f4b14efb99d52013\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-25.ec2.internal" Jul 14 02:01:38.561148 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.542203 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61db6160ce2d2c08f4b14efb99d52013-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-25.ec2.internal\" (UID: \"61db6160ce2d2c08f4b14efb99d52013\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-25.ec2.internal" Jul 14 02:01:38.561148 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.542239 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/61db6160ce2d2c08f4b14efb99d52013-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-25.ec2.internal\" (UID: \"61db6160ce2d2c08f4b14efb99d52013\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-25.ec2.internal" Jul 14 02:01:38.561148 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.542291 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/68e660ef911ade3343c0551cb4b95404-config\") pod \"kube-apiserver-proxy-ip-10-0-131-25.ec2.internal\" (UID: \"68e660ef911ade3343c0551cb4b95404\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-25.ec2.internal" Jul 14 02:01:38.706745 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:38.643026 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-25.ec2.internal\" not found" Jul 14 02:01:38.708916 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.708165 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-25.ec2.internal" Jul 14 02:01:38.712097 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:38.710330 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-25.ec2.internal" Jul 14 02:01:38.794223 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:38.743622 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-25.ec2.internal\" not found" Jul 14 02:01:39.021970 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:38.844749 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-25.ec2.internal\" not found" Jul 14 02:01:39.021970 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:38.945515 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-25.ec2.internal\" not found" Jul 14 02:01:39.264744 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:39.040014 2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 02:01:39.264744 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:39.046521 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-25.ec2.internal\" not found" Jul 14 02:01:39.264744 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:39.084869 2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 14 02:01:39.264744 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:39.085029 2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 14 02:01:39.264744 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:39.085071 2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 14 02:01:39.264744 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:39.147246 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-25.ec2.internal\" not found" Jul 14 02:01:39.264744 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:39.164410 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-13 01:56:39 +0000 UTC" deadline="2028-03-03 06:57:55.681916208 +0000 UTC" Jul 14 02:01:39.264744 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:39.164441 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14356h56m16.517478753s" Jul 14 02:01:39.264744 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:39.248378 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-25.ec2.internal\" not found" Jul 14 02:01:39.437785 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:39.348741 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-25.ec2.internal\" not found" Jul 14 02:01:39.437785 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:39.412809 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61db6160ce2d2c08f4b14efb99d52013.slice/crio-aa2467cc48024d3286ff854769d4b7cf58cc12d14c9275c59f7776d596752bf9 WatchSource:0}: Error finding container aa2467cc48024d3286ff854769d4b7cf58cc12d14c9275c59f7776d596752bf9: Status 404 returned error can't find the container with id aa2467cc48024d3286ff854769d4b7cf58cc12d14c9275c59f7776d596752bf9 Jul 14 02:01:39.437785 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:39.419772 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 02:01:39.437785 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:39.425232 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68e660ef911ade3343c0551cb4b95404.slice/crio-98dad50ed3e6174b6157f9833ce75b2ab32e0b3de6b1c2ff174d610e96430211 WatchSource:0}: Error finding container 98dad50ed3e6174b6157f9833ce75b2ab32e0b3de6b1c2ff174d610e96430211: Status 404 returned error can't find the container with id 98dad50ed3e6174b6157f9833ce75b2ab32e0b3de6b1c2ff174d610e96430211 Jul 14 02:01:39.449602 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:39.449568 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-25.ec2.internal\" not found" Jul 14 02:01:39.550266 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:39.550223 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-25.ec2.internal\" not found" Jul 14 02:01:39.626491 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:39.626465 2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 02:01:39.650794 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:39.650736 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-25.ec2.internal\" not found" Jul 14 02:01:39.719415 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:39.719328 2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 02:01:39.738718 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:39.738680 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-25.ec2.internal" Jul 14 02:01:39.747900 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:39.747878 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 02:01:39.748696 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:39.748673 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-25.ec2.internal" Jul 14 02:01:39.757646 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:39.757626 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 02:01:40.014608 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.014514 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 02:01:40.121470 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.121424 2577 apiserver.go:52] "Watching apiserver" Jul 14 02:01:40.128193 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.128156 2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 14 02:01:40.129130 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.129077 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-zc4lr","kube-system/konnectivity-agent-bs5bn","openshift-cluster-node-tuning-operator/tuned-42rhx","openshift-dns/node-resolver-k4w2h","openshift-multus/multus-additional-cni-plugins-gvljc","openshift-multus/network-metrics-daemon-t9bnr","openshift-network-diagnostics/network-check-target-c6wmg","openshift-network-operator/iptables-alerter-g966p","openshift-ovn-kubernetes/ovnkube-node-bnpgh","kube-system/global-pull-secret-syncer-wmd5q","kube-system/kube-apiserver-proxy-ip-10-0-131-25.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbb4f","openshift-image-registry/node-ca-rpl6g","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-25.ec2.internal"] Jul 14 02:01:40.131866 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.131829 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6wmg" Jul 14 02:01:40.131993 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:40.131932 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6wmg" podUID="f018324f-f63f-41e0-9a3f-d6fed83b4f1c" Jul 14 02:01:40.134265 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.134165 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rpl6g" Jul 14 02:01:40.138594 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.138568 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-5r6pp\"" Jul 14 02:01:40.138854 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.138836 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 14 02:01:40.139089 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.139067 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 14 02:01:40.139297 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.139282 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 14 02:01:40.139851 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.139833 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.140865 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.140542 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 14 02:01:40.141698 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.141299 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gvljc" Jul 14 02:01:40.143090 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.143058 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 14 02:01:40.143410 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.143388 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-bkwgl\"" Jul 14 02:01:40.146864 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.143667 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 14 02:01:40.146864 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.144089 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-jmbj7\"" Jul 14 02:01:40.146864 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.144359 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 14 02:01:40.146864 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.144674 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 14 02:01:40.146864 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.144863 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 14 02:01:40.146864 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.145065 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 14 02:01:40.146864 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.145379 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 14 02:01:40.146864 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.145898 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9bnr" Jul 14 02:01:40.146864 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:40.145985 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t9bnr" podUID="8d6a986a-c066-4ce3-a49c-4bab5be2f075" Jul 14 02:01:40.146864 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.146478 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.146864 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.146611 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-k4w2h" Jul 14 02:01:40.148349 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.148322 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 14 02:01:40.149733 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.148652 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-f47w5\"" Jul 14 02:01:40.149733 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.149005 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 14 02:01:40.149733 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.149176 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-f6p4h\"" Jul 14 02:01:40.149733 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.149435 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.151181 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.151159 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 14 02:01:40.152542 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.151813 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bs5bn" Jul 14 02:01:40.158171 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.158003 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 14 02:01:40.158822 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.158796 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 14 02:01:40.164370 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.159225 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 14 02:01:40.164370 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.159568 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 14 02:01:40.164370 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.160336 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-d26rw\"" Jul 14 02:01:40.164370 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.160874 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 14 02:01:40.164370 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.161579 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmd5q" Jul 14 02:01:40.164370 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:40.161686 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wmd5q" podUID="45d07b54-1b6f-4f51-bf7c-cc8d287e5789" Jul 14 02:01:40.164370 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.162140 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 14 02:01:40.164370 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.162716 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 02:01:40.164370 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.162739 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 14 02:01:40.164370 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.162791 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 14 02:01:40.164370 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.163138 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-bb4ff\"" Jul 14 02:01:40.164370 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.163217 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-g966p" Jul 14 02:01:40.164370 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.163441 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94pcq\" (UniqueName: \"kubernetes.io/projected/94fa401d-46a7-45c2-bc57-88c112508139-kube-api-access-94pcq\") pod \"node-ca-rpl6g\" (UID: \"94fa401d-46a7-45c2-bc57-88c112508139\") " pod="openshift-image-registry/node-ca-rpl6g" Jul 14 02:01:40.164370 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.163475 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/772c82e2-fe88-43ef-b9d1-069856ce9282-etc-sysconfig\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.164370 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.163509 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/772c82e2-fe88-43ef-b9d1-069856ce9282-etc-kubernetes\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.164370 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.163540 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/772c82e2-fe88-43ef-b9d1-069856ce9282-etc-sysctl-d\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.164370 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.163571 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/772c82e2-fe88-43ef-b9d1-069856ce9282-tmp\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.164370 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.163612 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3d396e3c-208e-4489-88ee-334cfd664280-os-release\") pod \"multus-additional-cni-plugins-gvljc\" (UID: \"3d396e3c-208e-4489-88ee-334cfd664280\") " pod="openshift-multus/multus-additional-cni-plugins-gvljc" Jul 14 02:01:40.164370 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.163648 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s985w\" (UniqueName: \"kubernetes.io/projected/f018324f-f63f-41e0-9a3f-d6fed83b4f1c-kube-api-access-s985w\") pod \"network-check-target-c6wmg\" (UID: \"f018324f-f63f-41e0-9a3f-d6fed83b4f1c\") " pod="openshift-network-diagnostics/network-check-target-c6wmg" Jul 14 02:01:40.164370 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.163679 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/94fa401d-46a7-45c2-bc57-88c112508139-serviceca\") pod \"node-ca-rpl6g\" (UID: \"94fa401d-46a7-45c2-bc57-88c112508139\") " pod="openshift-image-registry/node-ca-rpl6g" Jul 14 02:01:40.165347 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.163715 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3d396e3c-208e-4489-88ee-334cfd664280-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gvljc\" (UID: \"3d396e3c-208e-4489-88ee-334cfd664280\") " pod="openshift-multus/multus-additional-cni-plugins-gvljc" Jul 14 02:01:40.165347 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.163750 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/772c82e2-fe88-43ef-b9d1-069856ce9282-lib-modules\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.165347 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.163778 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/772c82e2-fe88-43ef-b9d1-069856ce9282-host\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.165347 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.163805 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3d396e3c-208e-4489-88ee-334cfd664280-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gvljc\" (UID: \"3d396e3c-208e-4489-88ee-334cfd664280\") " pod="openshift-multus/multus-additional-cni-plugins-gvljc" Jul 14 02:01:40.165347 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.163842 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/059e3b07-82d1-475b-a7c2-4e9345579238-tmp-dir\") pod \"node-resolver-k4w2h\" (UID: \"059e3b07-82d1-475b-a7c2-4e9345579238\") " pod="openshift-dns/node-resolver-k4w2h" Jul 14 02:01:40.165347 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.163874 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d696\" (UniqueName: \"kubernetes.io/projected/059e3b07-82d1-475b-a7c2-4e9345579238-kube-api-access-8d696\") pod \"node-resolver-k4w2h\" (UID: \"059e3b07-82d1-475b-a7c2-4e9345579238\") " pod="openshift-dns/node-resolver-k4w2h" Jul 14 02:01:40.165347 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.163905 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3d396e3c-208e-4489-88ee-334cfd664280-system-cni-dir\") pod \"multus-additional-cni-plugins-gvljc\" (UID: \"3d396e3c-208e-4489-88ee-334cfd664280\") " pod="openshift-multus/multus-additional-cni-plugins-gvljc" Jul 14 02:01:40.165347 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.163939 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3d396e3c-208e-4489-88ee-334cfd664280-cni-binary-copy\") pod \"multus-additional-cni-plugins-gvljc\" (UID: \"3d396e3c-208e-4489-88ee-334cfd664280\") " pod="openshift-multus/multus-additional-cni-plugins-gvljc" Jul 14 02:01:40.165347 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.163966 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/772c82e2-fe88-43ef-b9d1-069856ce9282-etc-modprobe-d\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.165347 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.164053 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/772c82e2-fe88-43ef-b9d1-069856ce9282-run\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.165347 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.164097 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/772c82e2-fe88-43ef-b9d1-069856ce9282-var-lib-kubelet\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.165347 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.164144 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbnpk\" (UniqueName: \"kubernetes.io/projected/772c82e2-fe88-43ef-b9d1-069856ce9282-kube-api-access-xbnpk\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.165347 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.164305 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3d396e3c-208e-4489-88ee-334cfd664280-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gvljc\" (UID: \"3d396e3c-208e-4489-88ee-334cfd664280\") " pod="openshift-multus/multus-additional-cni-plugins-gvljc" Jul 14 02:01:40.166010 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.164782 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/772c82e2-fe88-43ef-b9d1-069856ce9282-sys\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.166010 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.164850 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/772c82e2-fe88-43ef-b9d1-069856ce9282-etc-tuned\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.166010 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.164886 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/059e3b07-82d1-475b-a7c2-4e9345579238-hosts-file\") pod \"node-resolver-k4w2h\" (UID: \"059e3b07-82d1-475b-a7c2-4e9345579238\") " pod="openshift-dns/node-resolver-k4w2h" Jul 14 02:01:40.166010 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.165394 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 14 02:01:40.166010 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.165536 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3d396e3c-208e-4489-88ee-334cfd664280-cnibin\") pod \"multus-additional-cni-plugins-gvljc\" (UID: \"3d396e3c-208e-4489-88ee-334cfd664280\") " pod="openshift-multus/multus-additional-cni-plugins-gvljc" Jul 14 02:01:40.166010 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.165586 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/94fa401d-46a7-45c2-bc57-88c112508139-host\") pod \"node-ca-rpl6g\" (UID: \"94fa401d-46a7-45c2-bc57-88c112508139\") " pod="openshift-image-registry/node-ca-rpl6g" Jul 14 02:01:40.166010 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.165659 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/772c82e2-fe88-43ef-b9d1-069856ce9282-etc-sysctl-conf\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.166010 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.165694 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/772c82e2-fe88-43ef-b9d1-069856ce9282-etc-systemd\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.166010 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.165728 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s446x\" (UniqueName: \"kubernetes.io/projected/3d396e3c-208e-4489-88ee-334cfd664280-kube-api-access-s446x\") pod \"multus-additional-cni-plugins-gvljc\" (UID: \"3d396e3c-208e-4489-88ee-334cfd664280\") " pod="openshift-multus/multus-additional-cni-plugins-gvljc" Jul 14 02:01:40.166010 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.165877 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 14 02:01:40.166010 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.165999 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 14 02:01:40.166571 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.166288 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9fhrl\"" Jul 14 02:01:40.166831 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.166752 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbb4f" Jul 14 02:01:40.168726 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.168702 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 14 02:01:40.168895 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.168875 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-74h9l\"" Jul 14 02:01:40.168960 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.168924 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 14 02:01:40.168960 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.168941 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 14 02:01:40.174925 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.174881 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hrph8" Jul 14 02:01:40.182471 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.182448 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hrph8" Jul 14 02:01:40.241041 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.241018 2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 14 02:01:40.266256 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.265850 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-25.ec2.internal" event={"ID":"68e660ef911ade3343c0551cb4b95404","Type":"ContainerStarted","Data":"98dad50ed3e6174b6157f9833ce75b2ab32e0b3de6b1c2ff174d610e96430211"} Jul 14 02:01:40.266256 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.265988 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/772c82e2-fe88-43ef-b9d1-069856ce9282-tmp\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.266256 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.266023 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3d396e3c-208e-4489-88ee-334cfd664280-os-release\") pod \"multus-additional-cni-plugins-gvljc\" (UID: \"3d396e3c-208e-4489-88ee-334cfd664280\") " pod="openshift-multus/multus-additional-cni-plugins-gvljc" Jul 14 02:01:40.266256 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.266056 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f361ee09-a428-4b20-93c3-b9762d14f012-konnectivity-ca\") pod \"konnectivity-agent-bs5bn\" (UID: \"f361ee09-a428-4b20-93c3-b9762d14f012\") " pod="kube-system/konnectivity-agent-bs5bn" Jul 14 02:01:40.266256 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.266085 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-log-socket\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.266256 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.266110 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.266256 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.266139 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/94fa401d-46a7-45c2-bc57-88c112508139-serviceca\") pod \"node-ca-rpl6g\" (UID: \"94fa401d-46a7-45c2-bc57-88c112508139\") " pod="openshift-image-registry/node-ca-rpl6g" Jul 14 02:01:40.266256 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.266191 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-run-ovn\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.266256 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.266195 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3d396e3c-208e-4489-88ee-334cfd664280-os-release\") pod \"multus-additional-cni-plugins-gvljc\" (UID: \"3d396e3c-208e-4489-88ee-334cfd664280\") " pod="openshift-multus/multus-additional-cni-plugins-gvljc" Jul 14 02:01:40.266256 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.266215 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-ovn-node-metrics-cert\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.266256 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.266264 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/45d07b54-1b6f-4f51-bf7c-cc8d287e5789-kubelet-config\") pod \"global-pull-secret-syncer-wmd5q\" (UID: \"45d07b54-1b6f-4f51-bf7c-cc8d287e5789\") " pod="kube-system/global-pull-secret-syncer-wmd5q" Jul 14 02:01:40.266256 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.266289 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h7lv\" (UniqueName: \"kubernetes.io/projected/a6353ee6-d8ec-42fc-bf8f-3cc00662786b-kube-api-access-7h7lv\") pod \"iptables-alerter-g966p\" (UID: \"a6353ee6-d8ec-42fc-bf8f-3cc00662786b\") " pod="openshift-network-operator/iptables-alerter-g966p" Jul 14 02:01:40.267655 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.266315 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e60fc957-ce6a-4b22-a7f6-91dfd2ef0267-socket-dir\") pod \"aws-ebs-csi-driver-node-qbb4f\" (UID: \"e60fc957-ce6a-4b22-a7f6-91dfd2ef0267\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbb4f" Jul 14 02:01:40.267655 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.266337 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e60fc957-ce6a-4b22-a7f6-91dfd2ef0267-registration-dir\") pod \"aws-ebs-csi-driver-node-qbb4f\" (UID: \"e60fc957-ce6a-4b22-a7f6-91dfd2ef0267\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbb4f" Jul 14 02:01:40.267655 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.266386 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e60fc957-ce6a-4b22-a7f6-91dfd2ef0267-sys-fs\") pod \"aws-ebs-csi-driver-node-qbb4f\" (UID: \"e60fc957-ce6a-4b22-a7f6-91dfd2ef0267\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbb4f" Jul 14 02:01:40.267655 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.266418 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/772c82e2-fe88-43ef-b9d1-069856ce9282-lib-modules\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.267655 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.266440 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/772c82e2-fe88-43ef-b9d1-069856ce9282-host\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.267655 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.266462 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-run-systemd\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.267655 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.266490 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a6353ee6-d8ec-42fc-bf8f-3cc00662786b-host-slash\") pod \"iptables-alerter-g966p\" (UID: \"a6353ee6-d8ec-42fc-bf8f-3cc00662786b\") " pod="openshift-network-operator/iptables-alerter-g966p" Jul 14 02:01:40.267655 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.266517 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-system-cni-dir\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.267655 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.266543 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-multus-cni-dir\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.267655 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.266572 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-run-openvswitch\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.267655 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.266597 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e60fc957-ce6a-4b22-a7f6-91dfd2ef0267-device-dir\") pod \"aws-ebs-csi-driver-node-qbb4f\" (UID: \"e60fc957-ce6a-4b22-a7f6-91dfd2ef0267\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbb4f" Jul 14 02:01:40.267655 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.266625 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-host-var-lib-cni-multus\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.267655 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.266648 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-host-var-lib-kubelet\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.267655 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.266646 2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 14 02:01:40.268431 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.266708 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/059e3b07-82d1-475b-a7c2-4e9345579238-tmp-dir\") pod \"node-resolver-k4w2h\" (UID: \"059e3b07-82d1-475b-a7c2-4e9345579238\") " pod="openshift-dns/node-resolver-k4w2h" Jul 14 02:01:40.268431 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.266794 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8d696\" (UniqueName: \"kubernetes.io/projected/059e3b07-82d1-475b-a7c2-4e9345579238-kube-api-access-8d696\") pod \"node-resolver-k4w2h\" (UID: \"059e3b07-82d1-475b-a7c2-4e9345579238\") " pod="openshift-dns/node-resolver-k4w2h" Jul 14 02:01:40.268431 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.266824 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3d396e3c-208e-4489-88ee-334cfd664280-system-cni-dir\") pod \"multus-additional-cni-plugins-gvljc\" (UID: \"3d396e3c-208e-4489-88ee-334cfd664280\") " pod="openshift-multus/multus-additional-cni-plugins-gvljc" Jul 14 02:01:40.268431 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.266854 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-env-overrides\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.268431 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.266899 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45d07b54-1b6f-4f51-bf7c-cc8d287e5789-original-pull-secret\") pod \"global-pull-secret-syncer-wmd5q\" (UID: \"45d07b54-1b6f-4f51-bf7c-cc8d287e5789\") " pod="kube-system/global-pull-secret-syncer-wmd5q" Jul 14 02:01:40.268431 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.266928 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-multus-socket-dir-parent\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.268431 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.266958 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-host-var-lib-cni-bin\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.268431 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.267104 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3d396e3c-208e-4489-88ee-334cfd664280-system-cni-dir\") pod \"multus-additional-cni-plugins-gvljc\" (UID: \"3d396e3c-208e-4489-88ee-334cfd664280\") " pod="openshift-multus/multus-additional-cni-plugins-gvljc" Jul 14 02:01:40.268431 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.267166 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-host-run-multus-certs\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.268431 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.267195 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/45d07b54-1b6f-4f51-bf7c-cc8d287e5789-dbus\") pod \"global-pull-secret-syncer-wmd5q\" (UID: \"45d07b54-1b6f-4f51-bf7c-cc8d287e5789\") " pod="kube-system/global-pull-secret-syncer-wmd5q" Jul 14 02:01:40.268431 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.267220 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/772c82e2-fe88-43ef-b9d1-069856ce9282-run\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.268431 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.267237 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-ovnkube-config\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.268431 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.267253 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e60fc957-ce6a-4b22-a7f6-91dfd2ef0267-etc-selinux\") pod \"aws-ebs-csi-driver-node-qbb4f\" (UID: \"e60fc957-ce6a-4b22-a7f6-91dfd2ef0267\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbb4f" Jul 14 02:01:40.268431 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.267308 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/059e3b07-82d1-475b-a7c2-4e9345579238-tmp-dir\") pod \"node-resolver-k4w2h\" (UID: \"059e3b07-82d1-475b-a7c2-4e9345579238\") " pod="openshift-dns/node-resolver-k4w2h" Jul 14 02:01:40.269192 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.267314 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-hostroot\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.269192 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.267340 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/94fa401d-46a7-45c2-bc57-88c112508139-serviceca\") pod \"node-ca-rpl6g\" (UID: \"94fa401d-46a7-45c2-bc57-88c112508139\") " pod="openshift-image-registry/node-ca-rpl6g" Jul 14 02:01:40.269192 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.267355 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/772c82e2-fe88-43ef-b9d1-069856ce9282-etc-tuned\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.269192 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.267415 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/059e3b07-82d1-475b-a7c2-4e9345579238-hosts-file\") pod \"node-resolver-k4w2h\" (UID: \"059e3b07-82d1-475b-a7c2-4e9345579238\") " pod="openshift-dns/node-resolver-k4w2h" Jul 14 02:01:40.269192 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.267442 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/772c82e2-fe88-43ef-b9d1-069856ce9282-host\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.269192 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.267484 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/772c82e2-fe88-43ef-b9d1-069856ce9282-lib-modules\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.269192 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.267491 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-host-run-netns\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.269192 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.267518 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-host-run-ovn-kubernetes\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.269192 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.267528 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/059e3b07-82d1-475b-a7c2-4e9345579238-hosts-file\") pod \"node-resolver-k4w2h\" (UID: \"059e3b07-82d1-475b-a7c2-4e9345579238\") " pod="openshift-dns/node-resolver-k4w2h" Jul 14 02:01:40.269192 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.267489 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/772c82e2-fe88-43ef-b9d1-069856ce9282-run\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.269192 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.267556 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e60fc957-ce6a-4b22-a7f6-91dfd2ef0267-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qbb4f\" (UID: \"e60fc957-ce6a-4b22-a7f6-91dfd2ef0267\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbb4f" Jul 14 02:01:40.269192 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.267598 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d6a986a-c066-4ce3-a49c-4bab5be2f075-metrics-certs\") pod \"network-metrics-daemon-t9bnr\" (UID: \"8d6a986a-c066-4ce3-a49c-4bab5be2f075\") " pod="openshift-multus/network-metrics-daemon-t9bnr" Jul 14 02:01:40.269192 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.267644 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/62531cce-f69d-4ccd-a6c6-e090b2789022-multus-daemon-config\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.269192 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.267693 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/94fa401d-46a7-45c2-bc57-88c112508139-host\") pod \"node-ca-rpl6g\" (UID: \"94fa401d-46a7-45c2-bc57-88c112508139\") " pod="openshift-image-registry/node-ca-rpl6g" Jul 14 02:01:40.269192 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.267724 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/772c82e2-fe88-43ef-b9d1-069856ce9282-etc-sysctl-conf\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.269935 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.267755 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-host-slash\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.269935 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.267804 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/94fa401d-46a7-45c2-bc57-88c112508139-host\") pod \"node-ca-rpl6g\" (UID: \"94fa401d-46a7-45c2-bc57-88c112508139\") " pod="openshift-image-registry/node-ca-rpl6g" Jul 14 02:01:40.269935 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.267854 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-var-lib-openvswitch\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.269935 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.267884 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-host-cni-bin\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.269935 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.267929 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cng7c\" (UniqueName: \"kubernetes.io/projected/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-kube-api-access-cng7c\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.269935 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.267994 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv52q\" (UniqueName: \"kubernetes.io/projected/8d6a986a-c066-4ce3-a49c-4bab5be2f075-kube-api-access-nv52q\") pod \"network-metrics-daemon-t9bnr\" (UID: \"8d6a986a-c066-4ce3-a49c-4bab5be2f075\") " pod="openshift-multus/network-metrics-daemon-t9bnr" Jul 14 02:01:40.269935 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.267969 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/772c82e2-fe88-43ef-b9d1-069856ce9282-etc-sysctl-conf\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.269935 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.268036 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/772c82e2-fe88-43ef-b9d1-069856ce9282-etc-sysconfig\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.269935 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.268065 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s985w\" (UniqueName: \"kubernetes.io/projected/f018324f-f63f-41e0-9a3f-d6fed83b4f1c-kube-api-access-s985w\") pod \"network-check-target-c6wmg\" (UID: \"f018324f-f63f-41e0-9a3f-d6fed83b4f1c\") " pod="openshift-network-diagnostics/network-check-target-c6wmg" Jul 14 02:01:40.269935 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.268147 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-os-release\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.269935 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.268224 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/772c82e2-fe88-43ef-b9d1-069856ce9282-etc-sysconfig\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.269935 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.268300 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-multus-conf-dir\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.269935 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.268357 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ks9sf\" (UniqueName: \"kubernetes.io/projected/62531cce-f69d-4ccd-a6c6-e090b2789022-kube-api-access-ks9sf\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.269935 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.268411 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3d396e3c-208e-4489-88ee-334cfd664280-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gvljc\" (UID: \"3d396e3c-208e-4489-88ee-334cfd664280\") " pod="openshift-multus/multus-additional-cni-plugins-gvljc" Jul 14 02:01:40.270626 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.268441 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-etc-openvswitch\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.270626 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.268581 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3d396e3c-208e-4489-88ee-334cfd664280-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gvljc\" (UID: \"3d396e3c-208e-4489-88ee-334cfd664280\") " pod="openshift-multus/multus-additional-cni-plugins-gvljc" Jul 14 02:01:40.270626 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.268613 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-node-log\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.270626 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.268670 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-host-run-k8s-cni-cncf-io\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.270626 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.268699 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f361ee09-a428-4b20-93c3-b9762d14f012-agent-certs\") pod \"konnectivity-agent-bs5bn\" (UID: \"f361ee09-a428-4b20-93c3-b9762d14f012\") " pod="kube-system/konnectivity-agent-bs5bn" Jul 14 02:01:40.270626 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.268724 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a6353ee6-d8ec-42fc-bf8f-3cc00662786b-iptables-alerter-script\") pod \"iptables-alerter-g966p\" (UID: \"a6353ee6-d8ec-42fc-bf8f-3cc00662786b\") " pod="openshift-network-operator/iptables-alerter-g966p" Jul 14 02:01:40.270626 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.268765 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvmnb\" (UniqueName: \"kubernetes.io/projected/e60fc957-ce6a-4b22-a7f6-91dfd2ef0267-kube-api-access-vvmnb\") pod \"aws-ebs-csi-driver-node-qbb4f\" (UID: \"e60fc957-ce6a-4b22-a7f6-91dfd2ef0267\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbb4f" Jul 14 02:01:40.270626 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.268819 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3d396e3c-208e-4489-88ee-334cfd664280-cni-binary-copy\") pod \"multus-additional-cni-plugins-gvljc\" (UID: \"3d396e3c-208e-4489-88ee-334cfd664280\") " pod="openshift-multus/multus-additional-cni-plugins-gvljc" Jul 14 02:01:40.270626 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.268850 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-host-run-netns\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.270626 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.268880 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/772c82e2-fe88-43ef-b9d1-069856ce9282-etc-modprobe-d\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.270626 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.268906 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/772c82e2-fe88-43ef-b9d1-069856ce9282-var-lib-kubelet\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.270626 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.268938 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbnpk\" (UniqueName: \"kubernetes.io/projected/772c82e2-fe88-43ef-b9d1-069856ce9282-kube-api-access-xbnpk\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.270626 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.268965 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3d396e3c-208e-4489-88ee-334cfd664280-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gvljc\" (UID: \"3d396e3c-208e-4489-88ee-334cfd664280\") " pod="openshift-multus/multus-additional-cni-plugins-gvljc" Jul 14 02:01:40.270626 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.268991 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-host-kubelet\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.271330 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.269069 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3d396e3c-208e-4489-88ee-334cfd664280-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gvljc\" (UID: \"3d396e3c-208e-4489-88ee-334cfd664280\") " pod="openshift-multus/multus-additional-cni-plugins-gvljc" Jul 14 02:01:40.271330 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.269236 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3d396e3c-208e-4489-88ee-334cfd664280-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gvljc\" (UID: \"3d396e3c-208e-4489-88ee-334cfd664280\") " pod="openshift-multus/multus-additional-cni-plugins-gvljc" Jul 14 02:01:40.271330 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.269303 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-ovnkube-script-lib\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.271330 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.269331 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/62531cce-f69d-4ccd-a6c6-e090b2789022-cni-binary-copy\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.271330 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.269361 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/772c82e2-fe88-43ef-b9d1-069856ce9282-sys\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.271330 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.269395 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3d396e3c-208e-4489-88ee-334cfd664280-cnibin\") pod \"multus-additional-cni-plugins-gvljc\" (UID: \"3d396e3c-208e-4489-88ee-334cfd664280\") " pod="openshift-multus/multus-additional-cni-plugins-gvljc" Jul 14 02:01:40.271330 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.269434 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-host-cni-netd\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.271330 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.269492 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/772c82e2-fe88-43ef-b9d1-069856ce9282-var-lib-kubelet\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.271330 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.269556 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-etc-kubernetes\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.271330 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.269605 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/772c82e2-fe88-43ef-b9d1-069856ce9282-etc-systemd\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.271330 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.269652 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/772c82e2-fe88-43ef-b9d1-069856ce9282-sys\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.271330 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.269638 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s446x\" (UniqueName: \"kubernetes.io/projected/3d396e3c-208e-4489-88ee-334cfd664280-kube-api-access-s446x\") pod \"multus-additional-cni-plugins-gvljc\" (UID: \"3d396e3c-208e-4489-88ee-334cfd664280\") " pod="openshift-multus/multus-additional-cni-plugins-gvljc" Jul 14 02:01:40.271330 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.269677 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/772c82e2-fe88-43ef-b9d1-069856ce9282-etc-modprobe-d\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.271330 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.269738 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-systemd-units\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.272120 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.270017 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-cnibin\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.272120 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.270032 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3d396e3c-208e-4489-88ee-334cfd664280-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gvljc\" (UID: \"3d396e3c-208e-4489-88ee-334cfd664280\") " pod="openshift-multus/multus-additional-cni-plugins-gvljc" Jul 14 02:01:40.272120 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.270105 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94pcq\" (UniqueName: \"kubernetes.io/projected/94fa401d-46a7-45c2-bc57-88c112508139-kube-api-access-94pcq\") pod \"node-ca-rpl6g\" (UID: \"94fa401d-46a7-45c2-bc57-88c112508139\") " pod="openshift-image-registry/node-ca-rpl6g" Jul 14 02:01:40.272120 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.271268 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3d396e3c-208e-4489-88ee-334cfd664280-cni-binary-copy\") pod \"multus-additional-cni-plugins-gvljc\" (UID: \"3d396e3c-208e-4489-88ee-334cfd664280\") " pod="openshift-multus/multus-additional-cni-plugins-gvljc" Jul 14 02:01:40.272120 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.271311 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/772c82e2-fe88-43ef-b9d1-069856ce9282-etc-kubernetes\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.272120 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.271381 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/772c82e2-fe88-43ef-b9d1-069856ce9282-etc-sysctl-d\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.272120 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.271384 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/772c82e2-fe88-43ef-b9d1-069856ce9282-etc-systemd\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.272120 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.271492 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/772c82e2-fe88-43ef-b9d1-069856ce9282-etc-sysctl-d\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.272549 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.272201 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/772c82e2-fe88-43ef-b9d1-069856ce9282-tmp\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.272549 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.272292 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/772c82e2-fe88-43ef-b9d1-069856ce9282-etc-tuned\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.272549 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.272391 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3d396e3c-208e-4489-88ee-334cfd664280-cnibin\") pod \"multus-additional-cni-plugins-gvljc\" (UID: \"3d396e3c-208e-4489-88ee-334cfd664280\") " pod="openshift-multus/multus-additional-cni-plugins-gvljc" Jul 14 02:01:40.272549 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.272527 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/772c82e2-fe88-43ef-b9d1-069856ce9282-etc-kubernetes\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.272741 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.272550 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-25.ec2.internal" event={"ID":"61db6160ce2d2c08f4b14efb99d52013","Type":"ContainerStarted","Data":"aa2467cc48024d3286ff854769d4b7cf58cc12d14c9275c59f7776d596752bf9"} Jul 14 02:01:40.273978 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:40.273947 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 02:01:40.273978 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:40.273981 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 02:01:40.274157 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:40.273994 2577 projected.go:196] Error preparing data for projected volume kube-api-access-s985w for pod openshift-network-diagnostics/network-check-target-c6wmg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 02:01:40.274211 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:40.274167 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f018324f-f63f-41e0-9a3f-d6fed83b4f1c-kube-api-access-s985w podName:f018324f-f63f-41e0-9a3f-d6fed83b4f1c nodeName:}" failed. No retries permitted until 2026-07-14 02:01:40.774145991 +0000 UTC m=+2.928982643 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-s985w" (UniqueName: "kubernetes.io/projected/f018324f-f63f-41e0-9a3f-d6fed83b4f1c-kube-api-access-s985w") pod "network-check-target-c6wmg" (UID: "f018324f-f63f-41e0-9a3f-d6fed83b4f1c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 02:01:40.279673 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.279646 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbnpk\" (UniqueName: \"kubernetes.io/projected/772c82e2-fe88-43ef-b9d1-069856ce9282-kube-api-access-xbnpk\") pod \"tuned-42rhx\" (UID: \"772c82e2-fe88-43ef-b9d1-069856ce9282\") " pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.280172 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.280105 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94pcq\" (UniqueName: \"kubernetes.io/projected/94fa401d-46a7-45c2-bc57-88c112508139-kube-api-access-94pcq\") pod \"node-ca-rpl6g\" (UID: \"94fa401d-46a7-45c2-bc57-88c112508139\") " pod="openshift-image-registry/node-ca-rpl6g" Jul 14 02:01:40.280802 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.280779 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s446x\" (UniqueName: \"kubernetes.io/projected/3d396e3c-208e-4489-88ee-334cfd664280-kube-api-access-s446x\") pod \"multus-additional-cni-plugins-gvljc\" (UID: \"3d396e3c-208e-4489-88ee-334cfd664280\") " pod="openshift-multus/multus-additional-cni-plugins-gvljc" Jul 14 02:01:40.280802 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.280795 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d696\" (UniqueName: \"kubernetes.io/projected/059e3b07-82d1-475b-a7c2-4e9345579238-kube-api-access-8d696\") pod \"node-resolver-k4w2h\" (UID: \"059e3b07-82d1-475b-a7c2-4e9345579238\") " pod="openshift-dns/node-resolver-k4w2h" Jul 14 02:01:40.371812 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.371772 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-systemd-units\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.371812 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.371819 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-cnibin\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.372079 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.371850 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f361ee09-a428-4b20-93c3-b9762d14f012-konnectivity-ca\") pod \"konnectivity-agent-bs5bn\" (UID: \"f361ee09-a428-4b20-93c3-b9762d14f012\") " pod="kube-system/konnectivity-agent-bs5bn" Jul 14 02:01:40.372079 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.371877 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-log-socket\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.372079 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.371903 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.372079 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.371933 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-run-ovn\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.372079 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.371962 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-ovn-node-metrics-cert\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.372079 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.371988 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/45d07b54-1b6f-4f51-bf7c-cc8d287e5789-kubelet-config\") pod \"global-pull-secret-syncer-wmd5q\" (UID: \"45d07b54-1b6f-4f51-bf7c-cc8d287e5789\") " pod="kube-system/global-pull-secret-syncer-wmd5q" Jul 14 02:01:40.372079 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.372014 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h7lv\" (UniqueName: \"kubernetes.io/projected/a6353ee6-d8ec-42fc-bf8f-3cc00662786b-kube-api-access-7h7lv\") pod \"iptables-alerter-g966p\" (UID: \"a6353ee6-d8ec-42fc-bf8f-3cc00662786b\") " pod="openshift-network-operator/iptables-alerter-g966p" Jul 14 02:01:40.372079 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.372039 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e60fc957-ce6a-4b22-a7f6-91dfd2ef0267-socket-dir\") pod \"aws-ebs-csi-driver-node-qbb4f\" (UID: \"e60fc957-ce6a-4b22-a7f6-91dfd2ef0267\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbb4f" Jul 14 02:01:40.372079 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.372064 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e60fc957-ce6a-4b22-a7f6-91dfd2ef0267-registration-dir\") pod \"aws-ebs-csi-driver-node-qbb4f\" (UID: \"e60fc957-ce6a-4b22-a7f6-91dfd2ef0267\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbb4f" Jul 14 02:01:40.372616 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.372088 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e60fc957-ce6a-4b22-a7f6-91dfd2ef0267-sys-fs\") pod \"aws-ebs-csi-driver-node-qbb4f\" (UID: \"e60fc957-ce6a-4b22-a7f6-91dfd2ef0267\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbb4f" Jul 14 02:01:40.372616 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.372113 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-run-systemd\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.372616 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.372139 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a6353ee6-d8ec-42fc-bf8f-3cc00662786b-host-slash\") pod \"iptables-alerter-g966p\" (UID: \"a6353ee6-d8ec-42fc-bf8f-3cc00662786b\") " pod="openshift-network-operator/iptables-alerter-g966p" Jul 14 02:01:40.372616 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.372162 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-system-cni-dir\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.372616 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.372188 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-multus-cni-dir\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.372616 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.372212 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-run-openvswitch\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.372616 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.372238 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e60fc957-ce6a-4b22-a7f6-91dfd2ef0267-device-dir\") pod \"aws-ebs-csi-driver-node-qbb4f\" (UID: \"e60fc957-ce6a-4b22-a7f6-91dfd2ef0267\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbb4f" Jul 14 02:01:40.372616 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.372263 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-host-var-lib-cni-multus\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.372616 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.372290 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-host-var-lib-kubelet\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.372616 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.372338 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-env-overrides\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.372616 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.372366 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45d07b54-1b6f-4f51-bf7c-cc8d287e5789-original-pull-secret\") pod \"global-pull-secret-syncer-wmd5q\" (UID: \"45d07b54-1b6f-4f51-bf7c-cc8d287e5789\") " pod="kube-system/global-pull-secret-syncer-wmd5q" Jul 14 02:01:40.372616 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.372394 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-multus-socket-dir-parent\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.372616 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.372418 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-host-var-lib-cni-bin\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.372616 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.372447 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-host-run-multus-certs\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.372616 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.372481 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/45d07b54-1b6f-4f51-bf7c-cc8d287e5789-dbus\") pod \"global-pull-secret-syncer-wmd5q\" (UID: \"45d07b54-1b6f-4f51-bf7c-cc8d287e5789\") " pod="kube-system/global-pull-secret-syncer-wmd5q" Jul 14 02:01:40.373389 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.372511 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-ovnkube-config\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.373389 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.372536 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e60fc957-ce6a-4b22-a7f6-91dfd2ef0267-etc-selinux\") pod \"aws-ebs-csi-driver-node-qbb4f\" (UID: \"e60fc957-ce6a-4b22-a7f6-91dfd2ef0267\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbb4f" Jul 14 02:01:40.373389 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.372561 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-hostroot\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.373389 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.372596 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-host-run-netns\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.373389 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.372624 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-host-run-ovn-kubernetes\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.373389 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.372684 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e60fc957-ce6a-4b22-a7f6-91dfd2ef0267-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qbb4f\" (UID: \"e60fc957-ce6a-4b22-a7f6-91dfd2ef0267\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbb4f" Jul 14 02:01:40.373389 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.372711 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d6a986a-c066-4ce3-a49c-4bab5be2f075-metrics-certs\") pod \"network-metrics-daemon-t9bnr\" (UID: \"8d6a986a-c066-4ce3-a49c-4bab5be2f075\") " pod="openshift-multus/network-metrics-daemon-t9bnr" Jul 14 02:01:40.373389 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.372737 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/62531cce-f69d-4ccd-a6c6-e090b2789022-multus-daemon-config\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.373389 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.372760 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-system-cni-dir\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.373389 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.372777 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-host-slash\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.373389 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.372805 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-var-lib-openvswitch\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.373389 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.372833 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-cnibin\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.373389 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.372843 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-host-cni-bin\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.373389 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.372871 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cng7c\" (UniqueName: \"kubernetes.io/projected/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-kube-api-access-cng7c\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.374092 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.372900 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv52q\" (UniqueName: \"kubernetes.io/projected/8d6a986a-c066-4ce3-a49c-4bab5be2f075-kube-api-access-nv52q\") pod \"network-metrics-daemon-t9bnr\" (UID: \"8d6a986a-c066-4ce3-a49c-4bab5be2f075\") " pod="openshift-multus/network-metrics-daemon-t9bnr" Jul 14 02:01:40.374092 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.372952 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-os-release\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.374092 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.372975 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-multus-conf-dir\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.374092 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.372998 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ks9sf\" (UniqueName: \"kubernetes.io/projected/62531cce-f69d-4ccd-a6c6-e090b2789022-kube-api-access-ks9sf\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.374092 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.373028 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-etc-openvswitch\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.374092 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.373054 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-node-log\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.374092 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.373079 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-host-run-k8s-cni-cncf-io\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.374092 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.373203 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-multus-socket-dir-parent\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.374092 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.373251 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-multus-cni-dir\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.374092 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.373256 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/45d07b54-1b6f-4f51-bf7c-cc8d287e5789-dbus\") pod \"global-pull-secret-syncer-wmd5q\" (UID: \"45d07b54-1b6f-4f51-bf7c-cc8d287e5789\") " pod="kube-system/global-pull-secret-syncer-wmd5q" Jul 14 02:01:40.374092 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.373317 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-host-cni-bin\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.374092 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.373348 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-run-openvswitch\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.374092 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.373360 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-host-run-multus-certs\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.374092 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.373393 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-host-run-ovn-kubernetes\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.374092 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.373392 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-host-run-k8s-cni-cncf-io\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.374928 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.373424 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-host-var-lib-cni-multus\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.374928 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.373452 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e60fc957-ce6a-4b22-a7f6-91dfd2ef0267-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qbb4f\" (UID: \"e60fc957-ce6a-4b22-a7f6-91dfd2ef0267\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbb4f" Jul 14 02:01:40.374928 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.373471 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-host-var-lib-kubelet\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.374928 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:40.373545 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 02:01:40.374928 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:40.373624 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d6a986a-c066-4ce3-a49c-4bab5be2f075-metrics-certs podName:8d6a986a-c066-4ce3-a49c-4bab5be2f075 nodeName:}" failed. No retries permitted until 2026-07-14 02:01:40.873605991 +0000 UTC m=+3.028442640 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8d6a986a-c066-4ce3-a49c-4bab5be2f075-metrics-certs") pod "network-metrics-daemon-t9bnr" (UID: "8d6a986a-c066-4ce3-a49c-4bab5be2f075") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 02:01:40.374928 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.373891 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-multus-conf-dir\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.374928 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.373907 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f361ee09-a428-4b20-93c3-b9762d14f012-agent-certs\") pod \"konnectivity-agent-bs5bn\" (UID: \"f361ee09-a428-4b20-93c3-b9762d14f012\") " pod="kube-system/konnectivity-agent-bs5bn" Jul 14 02:01:40.374928 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.373940 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a6353ee6-d8ec-42fc-bf8f-3cc00662786b-iptables-alerter-script\") pod \"iptables-alerter-g966p\" (UID: \"a6353ee6-d8ec-42fc-bf8f-3cc00662786b\") " pod="openshift-network-operator/iptables-alerter-g966p" Jul 14 02:01:40.374928 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.373968 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvmnb\" (UniqueName: \"kubernetes.io/projected/e60fc957-ce6a-4b22-a7f6-91dfd2ef0267-kube-api-access-vvmnb\") pod \"aws-ebs-csi-driver-node-qbb4f\" (UID: \"e60fc957-ce6a-4b22-a7f6-91dfd2ef0267\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbb4f" Jul 14 02:01:40.374928 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.373998 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-host-run-netns\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.374928 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.374028 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-host-kubelet\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.374928 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.374048 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e60fc957-ce6a-4b22-a7f6-91dfd2ef0267-device-dir\") pod \"aws-ebs-csi-driver-node-qbb4f\" (UID: \"e60fc957-ce6a-4b22-a7f6-91dfd2ef0267\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbb4f" Jul 14 02:01:40.374928 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.374055 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-ovnkube-script-lib\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.374928 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.374100 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/62531cce-f69d-4ccd-a6c6-e090b2789022-cni-binary-copy\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.375646 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.374133 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-host-cni-netd\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.375646 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.374178 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-etc-kubernetes\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.375646 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.374230 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-os-release\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.375646 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.374266 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-etc-kubernetes\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.375646 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.374288 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-etc-openvswitch\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.375646 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.374387 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e60fc957-ce6a-4b22-a7f6-91dfd2ef0267-etc-selinux\") pod \"aws-ebs-csi-driver-node-qbb4f\" (UID: \"e60fc957-ce6a-4b22-a7f6-91dfd2ef0267\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbb4f" Jul 14 02:01:40.375646 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.374413 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f361ee09-a428-4b20-93c3-b9762d14f012-konnectivity-ca\") pod \"konnectivity-agent-bs5bn\" (UID: \"f361ee09-a428-4b20-93c3-b9762d14f012\") " pod="kube-system/konnectivity-agent-bs5bn" Jul 14 02:01:40.375646 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.374434 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-hostroot\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.375646 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.374476 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-host-run-netns\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.375646 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.374485 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.375646 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.374513 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-log-socket\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.375646 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.373316 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-host-var-lib-cni-bin\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.375646 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.374535 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-run-ovn\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.375646 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.374547 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-node-log\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.375646 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.374578 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-host-cni-netd\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.376458 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.374606 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e60fc957-ce6a-4b22-a7f6-91dfd2ef0267-registration-dir\") pod \"aws-ebs-csi-driver-node-qbb4f\" (UID: \"e60fc957-ce6a-4b22-a7f6-91dfd2ef0267\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbb4f" Jul 14 02:01:40.376458 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.374636 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e60fc957-ce6a-4b22-a7f6-91dfd2ef0267-sys-fs\") pod \"aws-ebs-csi-driver-node-qbb4f\" (UID: \"e60fc957-ce6a-4b22-a7f6-91dfd2ef0267\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbb4f" Jul 14 02:01:40.376458 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.374680 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-run-systemd\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.376458 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.374724 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a6353ee6-d8ec-42fc-bf8f-3cc00662786b-host-slash\") pod \"iptables-alerter-g966p\" (UID: \"a6353ee6-d8ec-42fc-bf8f-3cc00662786b\") " pod="openshift-network-operator/iptables-alerter-g966p" Jul 14 02:01:40.376458 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.374820 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-host-slash\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.376458 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.374879 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-systemd-units\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.376458 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.374912 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-env-overrides\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.376458 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.374942 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/45d07b54-1b6f-4f51-bf7c-cc8d287e5789-kubelet-config\") pod \"global-pull-secret-syncer-wmd5q\" (UID: \"45d07b54-1b6f-4f51-bf7c-cc8d287e5789\") " pod="kube-system/global-pull-secret-syncer-wmd5q" Jul 14 02:01:40.376458 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.374985 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-var-lib-openvswitch\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.376458 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.373996 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-ovnkube-config\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.376458 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:40.375023 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 02:01:40.376458 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.375067 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/62531cce-f69d-4ccd-a6c6-e090b2789022-host-run-netns\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.376458 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:40.375083 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45d07b54-1b6f-4f51-bf7c-cc8d287e5789-original-pull-secret podName:45d07b54-1b6f-4f51-bf7c-cc8d287e5789 nodeName:}" failed. No retries permitted until 2026-07-14 02:01:40.875070886 +0000 UTC m=+3.029907514 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/45d07b54-1b6f-4f51-bf7c-cc8d287e5789-original-pull-secret") pod "global-pull-secret-syncer-wmd5q" (UID: "45d07b54-1b6f-4f51-bf7c-cc8d287e5789") : object "kube-system"/"original-pull-secret" not registered Jul 14 02:01:40.376458 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.375148 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-host-kubelet\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.377175 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.375168 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e60fc957-ce6a-4b22-a7f6-91dfd2ef0267-socket-dir\") pod \"aws-ebs-csi-driver-node-qbb4f\" (UID: \"e60fc957-ce6a-4b22-a7f6-91dfd2ef0267\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbb4f" Jul 14 02:01:40.377175 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.375188 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/62531cce-f69d-4ccd-a6c6-e090b2789022-cni-binary-copy\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.377175 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.375365 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/62531cce-f69d-4ccd-a6c6-e090b2789022-multus-daemon-config\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.377175 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.375584 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-ovnkube-script-lib\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.377175 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.375625 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a6353ee6-d8ec-42fc-bf8f-3cc00662786b-iptables-alerter-script\") pod \"iptables-alerter-g966p\" (UID: \"a6353ee6-d8ec-42fc-bf8f-3cc00662786b\") " pod="openshift-network-operator/iptables-alerter-g966p" Jul 14 02:01:40.378046 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.377975 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f361ee09-a428-4b20-93c3-b9762d14f012-agent-certs\") pod \"konnectivity-agent-bs5bn\" (UID: \"f361ee09-a428-4b20-93c3-b9762d14f012\") " pod="kube-system/konnectivity-agent-bs5bn" Jul 14 02:01:40.378449 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.378424 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-ovn-node-metrics-cert\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.386094 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.386039 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ks9sf\" (UniqueName: \"kubernetes.io/projected/62531cce-f69d-4ccd-a6c6-e090b2789022-kube-api-access-ks9sf\") pod \"multus-zc4lr\" (UID: \"62531cce-f69d-4ccd-a6c6-e090b2789022\") " pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.386094 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.386054 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv52q\" (UniqueName: \"kubernetes.io/projected/8d6a986a-c066-4ce3-a49c-4bab5be2f075-kube-api-access-nv52q\") pod \"network-metrics-daemon-t9bnr\" (UID: \"8d6a986a-c066-4ce3-a49c-4bab5be2f075\") " pod="openshift-multus/network-metrics-daemon-t9bnr" Jul 14 02:01:40.386634 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.386603 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cng7c\" (UniqueName: \"kubernetes.io/projected/13c988f5-6d7a-4d10-87c3-d548e48ac1d6-kube-api-access-cng7c\") pod \"ovnkube-node-bnpgh\" (UID: \"13c988f5-6d7a-4d10-87c3-d548e48ac1d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.387837 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.387764 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvmnb\" (UniqueName: \"kubernetes.io/projected/e60fc957-ce6a-4b22-a7f6-91dfd2ef0267-kube-api-access-vvmnb\") pod \"aws-ebs-csi-driver-node-qbb4f\" (UID: \"e60fc957-ce6a-4b22-a7f6-91dfd2ef0267\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbb4f" Jul 14 02:01:40.388515 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.388468 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h7lv\" (UniqueName: \"kubernetes.io/projected/a6353ee6-d8ec-42fc-bf8f-3cc00662786b-kube-api-access-7h7lv\") pod \"iptables-alerter-g966p\" (UID: \"a6353ee6-d8ec-42fc-bf8f-3cc00662786b\") " pod="openshift-network-operator/iptables-alerter-g966p" Jul 14 02:01:40.466802 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.466760 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rpl6g" Jul 14 02:01:40.476659 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.476621 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-42rhx" Jul 14 02:01:40.486805 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.486779 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gvljc" Jul 14 02:01:40.503115 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.503080 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:01:40.512324 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.512240 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-k4w2h" Jul 14 02:01:40.521580 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.521507 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zc4lr" Jul 14 02:01:40.530629 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.530603 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bs5bn" Jul 14 02:01:40.543685 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.543657 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-g966p" Jul 14 02:01:40.549760 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.549678 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbb4f" Jul 14 02:01:40.776819 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.776717 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s985w\" (UniqueName: \"kubernetes.io/projected/f018324f-f63f-41e0-9a3f-d6fed83b4f1c-kube-api-access-s985w\") pod \"network-check-target-c6wmg\" (UID: \"f018324f-f63f-41e0-9a3f-d6fed83b4f1c\") " pod="openshift-network-diagnostics/network-check-target-c6wmg" Jul 14 02:01:40.776980 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:40.776908 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 02:01:40.776980 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:40.776944 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 02:01:40.776980 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:40.776958 2577 projected.go:196] Error preparing data for projected volume kube-api-access-s985w for pod openshift-network-diagnostics/network-check-target-c6wmg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 02:01:40.777114 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:40.777017 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f018324f-f63f-41e0-9a3f-d6fed83b4f1c-kube-api-access-s985w podName:f018324f-f63f-41e0-9a3f-d6fed83b4f1c nodeName:}" failed. No retries permitted until 2026-07-14 02:01:41.777001401 +0000 UTC m=+3.931838030 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-s985w" (UniqueName: "kubernetes.io/projected/f018324f-f63f-41e0-9a3f-d6fed83b4f1c-kube-api-access-s985w") pod "network-check-target-c6wmg" (UID: "f018324f-f63f-41e0-9a3f-d6fed83b4f1c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 02:01:40.877862 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.877820 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d6a986a-c066-4ce3-a49c-4bab5be2f075-metrics-certs\") pod \"network-metrics-daemon-t9bnr\" (UID: \"8d6a986a-c066-4ce3-a49c-4bab5be2f075\") " pod="openshift-multus/network-metrics-daemon-t9bnr" Jul 14 02:01:40.878046 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:40.877930 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45d07b54-1b6f-4f51-bf7c-cc8d287e5789-original-pull-secret\") pod \"global-pull-secret-syncer-wmd5q\" (UID: \"45d07b54-1b6f-4f51-bf7c-cc8d287e5789\") " pod="kube-system/global-pull-secret-syncer-wmd5q" Jul 14 02:01:40.878046 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:40.878013 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 02:01:40.878046 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:40.878029 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 02:01:40.878210 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:40.878098 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45d07b54-1b6f-4f51-bf7c-cc8d287e5789-original-pull-secret podName:45d07b54-1b6f-4f51-bf7c-cc8d287e5789 nodeName:}" failed. No retries permitted until 2026-07-14 02:01:41.878078605 +0000 UTC m=+4.032915235 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/45d07b54-1b6f-4f51-bf7c-cc8d287e5789-original-pull-secret") pod "global-pull-secret-syncer-wmd5q" (UID: "45d07b54-1b6f-4f51-bf7c-cc8d287e5789") : object "kube-system"/"original-pull-secret" not registered Jul 14 02:01:40.878210 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:40.878117 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d6a986a-c066-4ce3-a49c-4bab5be2f075-metrics-certs podName:8d6a986a-c066-4ce3-a49c-4bab5be2f075 nodeName:}" failed. No retries permitted until 2026-07-14 02:01:41.87810809 +0000 UTC m=+4.032944721 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8d6a986a-c066-4ce3-a49c-4bab5be2f075-metrics-certs") pod "network-metrics-daemon-t9bnr" (UID: "8d6a986a-c066-4ce3-a49c-4bab5be2f075") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 02:01:41.186241 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:41.186198 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 01:56:40 +0000 UTC" deadline="2027-12-07 20:43:14.907198771 +0000 UTC" Jul 14 02:01:41.186241 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:41.186228 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12282h41m33.720974108s" Jul 14 02:01:41.259026 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:41.258987 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9bnr" Jul 14 02:01:41.259219 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:41.259003 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6wmg" Jul 14 02:01:41.259219 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:41.259128 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t9bnr" podUID="8d6a986a-c066-4ce3-a49c-4bab5be2f075" Jul 14 02:01:41.259219 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:41.259172 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6wmg" podUID="f018324f-f63f-41e0-9a3f-d6fed83b4f1c" Jul 14 02:01:41.455547 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:41.455516 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13c988f5_6d7a_4d10_87c3_d548e48ac1d6.slice/crio-cb1e08b771de25c1307fd828be8c1e1ac8c4f1046a33f3864ab172ee487f367b WatchSource:0}: Error finding container cb1e08b771de25c1307fd828be8c1e1ac8c4f1046a33f3864ab172ee487f367b: Status 404 returned error can't find the container with id cb1e08b771de25c1307fd828be8c1e1ac8c4f1046a33f3864ab172ee487f367b Jul 14 02:01:41.459546 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:41.459518 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94fa401d_46a7_45c2_bc57_88c112508139.slice/crio-18793d65cbd1555a3a70f2cd84c7c1e0180c207b8d2914636e8667a2ec7ed269 WatchSource:0}: Error finding container 18793d65cbd1555a3a70f2cd84c7c1e0180c207b8d2914636e8667a2ec7ed269: Status 404 returned error can't find the container with id 18793d65cbd1555a3a70f2cd84c7c1e0180c207b8d2914636e8667a2ec7ed269 Jul 14 02:01:41.460370 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:41.460346 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d396e3c_208e_4489_88ee_334cfd664280.slice/crio-644de8598637cbf37fa851fda3329d4b40d10d29a74b264a7bf94a05028781b0 WatchSource:0}: Error finding container 644de8598637cbf37fa851fda3329d4b40d10d29a74b264a7bf94a05028781b0: Status 404 returned error can't find the container with id 644de8598637cbf37fa851fda3329d4b40d10d29a74b264a7bf94a05028781b0 Jul 14 02:01:41.461571 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:41.461546 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod772c82e2_fe88_43ef_b9d1_069856ce9282.slice/crio-ddc492287c77c688d5dc8d963f4c74773cd6df768c74e7fc28a78cd0a1b01fc7 WatchSource:0}: Error finding container ddc492287c77c688d5dc8d963f4c74773cd6df768c74e7fc28a78cd0a1b01fc7: Status 404 returned error can't find the container with id ddc492287c77c688d5dc8d963f4c74773cd6df768c74e7fc28a78cd0a1b01fc7 Jul 14 02:01:41.462363 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:41.462299 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod059e3b07_82d1_475b_a7c2_4e9345579238.slice/crio-835e15264cbe4e9e6faa916705bdc06076f5a50c1e8689320e98aecaa4ac63f6 WatchSource:0}: Error finding container 835e15264cbe4e9e6faa916705bdc06076f5a50c1e8689320e98aecaa4ac63f6: Status 404 returned error can't find the container with id 835e15264cbe4e9e6faa916705bdc06076f5a50c1e8689320e98aecaa4ac63f6 Jul 14 02:01:41.465086 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:41.464827 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf361ee09_a428_4b20_93c3_b9762d14f012.slice/crio-4a412f46ace3f7b9c7197fd2cefc17a2e0d66b2674dbab170b8aa152cd569ec2 WatchSource:0}: Error finding container 4a412f46ace3f7b9c7197fd2cefc17a2e0d66b2674dbab170b8aa152cd569ec2: Status 404 returned error can't find the container with id 4a412f46ace3f7b9c7197fd2cefc17a2e0d66b2674dbab170b8aa152cd569ec2 Jul 14 02:01:41.465777 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:41.465753 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6353ee6_d8ec_42fc_bf8f_3cc00662786b.slice/crio-0933cf0a2a981d5b603203996f9e74d0239fdd3404fceb0631328fdd7c95be81 WatchSource:0}: Error finding container 0933cf0a2a981d5b603203996f9e74d0239fdd3404fceb0631328fdd7c95be81: Status 404 returned error can't find the container with id 0933cf0a2a981d5b603203996f9e74d0239fdd3404fceb0631328fdd7c95be81 Jul 14 02:01:41.467719 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:01:41.467691 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62531cce_f69d_4ccd_a6c6_e090b2789022.slice/crio-fc15f72b69fb0f19c05b9b6a4a21ad48a7e448e5a8d68938113be491f498d47c WatchSource:0}: Error finding container fc15f72b69fb0f19c05b9b6a4a21ad48a7e448e5a8d68938113be491f498d47c: Status 404 returned error can't find the container with id fc15f72b69fb0f19c05b9b6a4a21ad48a7e448e5a8d68938113be491f498d47c Jul 14 02:01:41.784246 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:41.784036 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s985w\" (UniqueName: \"kubernetes.io/projected/f018324f-f63f-41e0-9a3f-d6fed83b4f1c-kube-api-access-s985w\") pod \"network-check-target-c6wmg\" (UID: \"f018324f-f63f-41e0-9a3f-d6fed83b4f1c\") " pod="openshift-network-diagnostics/network-check-target-c6wmg" Jul 14 02:01:41.784459 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:41.784220 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 02:01:41.784459 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:41.784344 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 02:01:41.784459 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:41.784357 2577 projected.go:196] Error preparing data for projected volume kube-api-access-s985w for pod openshift-network-diagnostics/network-check-target-c6wmg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 02:01:41.784459 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:41.784413 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f018324f-f63f-41e0-9a3f-d6fed83b4f1c-kube-api-access-s985w podName:f018324f-f63f-41e0-9a3f-d6fed83b4f1c nodeName:}" failed. No retries permitted until 2026-07-14 02:01:43.784399407 +0000 UTC m=+5.939236035 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-s985w" (UniqueName: "kubernetes.io/projected/f018324f-f63f-41e0-9a3f-d6fed83b4f1c-kube-api-access-s985w") pod "network-check-target-c6wmg" (UID: "f018324f-f63f-41e0-9a3f-d6fed83b4f1c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 02:01:41.884610 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:41.884574 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d6a986a-c066-4ce3-a49c-4bab5be2f075-metrics-certs\") pod \"network-metrics-daemon-t9bnr\" (UID: \"8d6a986a-c066-4ce3-a49c-4bab5be2f075\") " pod="openshift-multus/network-metrics-daemon-t9bnr" Jul 14 02:01:41.884776 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:41.884753 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 02:01:41.884846 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:41.884823 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 02:01:41.884846 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:41.884760 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45d07b54-1b6f-4f51-bf7c-cc8d287e5789-original-pull-secret\") pod \"global-pull-secret-syncer-wmd5q\" (UID: \"45d07b54-1b6f-4f51-bf7c-cc8d287e5789\") " pod="kube-system/global-pull-secret-syncer-wmd5q" Jul 14 02:01:41.884930 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:41.884829 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d6a986a-c066-4ce3-a49c-4bab5be2f075-metrics-certs podName:8d6a986a-c066-4ce3-a49c-4bab5be2f075 nodeName:}" failed. No retries permitted until 2026-07-14 02:01:43.884809658 +0000 UTC m=+6.039646311 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8d6a986a-c066-4ce3-a49c-4bab5be2f075-metrics-certs") pod "network-metrics-daemon-t9bnr" (UID: "8d6a986a-c066-4ce3-a49c-4bab5be2f075") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 02:01:41.884930 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:41.884874 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45d07b54-1b6f-4f51-bf7c-cc8d287e5789-original-pull-secret podName:45d07b54-1b6f-4f51-bf7c-cc8d287e5789 nodeName:}" failed. No retries permitted until 2026-07-14 02:01:43.884863773 +0000 UTC m=+6.039700404 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/45d07b54-1b6f-4f51-bf7c-cc8d287e5789-original-pull-secret") pod "global-pull-secret-syncer-wmd5q" (UID: "45d07b54-1b6f-4f51-bf7c-cc8d287e5789") : object "kube-system"/"original-pull-secret" not registered Jul 14 02:01:42.079270 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:42.079187 2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 02:01:42.187066 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:42.186999 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 01:56:40 +0000 UTC" deadline="2028-04-07 07:43:09.339527038 +0000 UTC" Jul 14 02:01:42.187066 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:42.187064 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15197h41m27.152467891s" Jul 14 02:01:42.262059 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:42.262026 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmd5q" Jul 14 02:01:42.262231 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:42.262170 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wmd5q" podUID="45d07b54-1b6f-4f51-bf7c-cc8d287e5789" Jul 14 02:01:42.287696 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:42.286853 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-25.ec2.internal" event={"ID":"68e660ef911ade3343c0551cb4b95404","Type":"ContainerStarted","Data":"1030518c61310b9d7cc9d121c7b9edcb161c4711ceb3d9b57235888192f20fe9"} Jul 14 02:01:42.299396 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:42.298678 2577 generic.go:356] "Generic (PLEG): container finished" podID="61db6160ce2d2c08f4b14efb99d52013" containerID="a0f9e699078f9df44a5370d6090d42aa3adaf41b6d3332eba323457a8b6c4390" exitCode=0 Jul 14 02:01:42.299396 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:42.298770 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-25.ec2.internal" event={"ID":"61db6160ce2d2c08f4b14efb99d52013","Type":"ContainerDied","Data":"a0f9e699078f9df44a5370d6090d42aa3adaf41b6d3332eba323457a8b6c4390"} Jul 14 02:01:42.305273 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:42.304179 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-25.ec2.internal" podStartSLOduration=2.304159221 podStartE2EDuration="2.304159221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 02:01:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:01:42.303260297 +0000 UTC m=+4.458096951" watchObservedRunningTime="2026-07-14 02:01:42.304159221 +0000 UTC m=+4.458995874" Jul 14 02:01:42.319087 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:42.318998 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-g966p" event={"ID":"a6353ee6-d8ec-42fc-bf8f-3cc00662786b","Type":"ContainerStarted","Data":"0933cf0a2a981d5b603203996f9e74d0239fdd3404fceb0631328fdd7c95be81"} Jul 14 02:01:42.349085 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:42.349013 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-42rhx" event={"ID":"772c82e2-fe88-43ef-b9d1-069856ce9282","Type":"ContainerStarted","Data":"ddc492287c77c688d5dc8d963f4c74773cd6df768c74e7fc28a78cd0a1b01fc7"} Jul 14 02:01:42.369510 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:42.369194 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gvljc" event={"ID":"3d396e3c-208e-4489-88ee-334cfd664280","Type":"ContainerStarted","Data":"644de8598637cbf37fa851fda3329d4b40d10d29a74b264a7bf94a05028781b0"} Jul 14 02:01:42.398591 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:42.398475 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rpl6g" event={"ID":"94fa401d-46a7-45c2-bc57-88c112508139","Type":"ContainerStarted","Data":"18793d65cbd1555a3a70f2cd84c7c1e0180c207b8d2914636e8667a2ec7ed269"} Jul 14 02:01:42.409175 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:42.409136 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbb4f" event={"ID":"e60fc957-ce6a-4b22-a7f6-91dfd2ef0267","Type":"ContainerStarted","Data":"908d9479c37f735beaafb2c2d04c181ec85a759b50bfb85fde85770a73a9b849"} Jul 14 02:01:42.421646 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:42.421606 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zc4lr" event={"ID":"62531cce-f69d-4ccd-a6c6-e090b2789022","Type":"ContainerStarted","Data":"fc15f72b69fb0f19c05b9b6a4a21ad48a7e448e5a8d68938113be491f498d47c"} Jul 14 02:01:42.438634 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:42.438591 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bs5bn" event={"ID":"f361ee09-a428-4b20-93c3-b9762d14f012","Type":"ContainerStarted","Data":"4a412f46ace3f7b9c7197fd2cefc17a2e0d66b2674dbab170b8aa152cd569ec2"} Jul 14 02:01:42.442073 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:42.442034 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-k4w2h" event={"ID":"059e3b07-82d1-475b-a7c2-4e9345579238","Type":"ContainerStarted","Data":"835e15264cbe4e9e6faa916705bdc06076f5a50c1e8689320e98aecaa4ac63f6"} Jul 14 02:01:42.460328 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:42.460290 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" event={"ID":"13c988f5-6d7a-4d10-87c3-d548e48ac1d6","Type":"ContainerStarted","Data":"cb1e08b771de25c1307fd828be8c1e1ac8c4f1046a33f3864ab172ee487f367b"} Jul 14 02:01:43.260447 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:43.259623 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6wmg" Jul 14 02:01:43.260447 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:43.259758 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6wmg" podUID="f018324f-f63f-41e0-9a3f-d6fed83b4f1c" Jul 14 02:01:43.260447 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:43.260157 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9bnr" Jul 14 02:01:43.260447 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:43.260278 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t9bnr" podUID="8d6a986a-c066-4ce3-a49c-4bab5be2f075" Jul 14 02:01:43.513500 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:43.513404 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-25.ec2.internal" event={"ID":"61db6160ce2d2c08f4b14efb99d52013","Type":"ContainerStarted","Data":"491482284b8632360edc7853052ee239e08a35594d9ee6ae9740e4e1e0294e89"} Jul 14 02:01:43.802957 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:43.802181 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s985w\" (UniqueName: \"kubernetes.io/projected/f018324f-f63f-41e0-9a3f-d6fed83b4f1c-kube-api-access-s985w\") pod \"network-check-target-c6wmg\" (UID: \"f018324f-f63f-41e0-9a3f-d6fed83b4f1c\") " pod="openshift-network-diagnostics/network-check-target-c6wmg" Jul 14 02:01:43.802957 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:43.802389 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 02:01:43.802957 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:43.802412 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 02:01:43.802957 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:43.802426 2577 projected.go:196] Error preparing data for projected volume kube-api-access-s985w for pod openshift-network-diagnostics/network-check-target-c6wmg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 02:01:43.802957 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:43.802498 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f018324f-f63f-41e0-9a3f-d6fed83b4f1c-kube-api-access-s985w podName:f018324f-f63f-41e0-9a3f-d6fed83b4f1c nodeName:}" failed. No retries permitted until 2026-07-14 02:01:47.802475366 +0000 UTC m=+9.957312055 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-s985w" (UniqueName: "kubernetes.io/projected/f018324f-f63f-41e0-9a3f-d6fed83b4f1c-kube-api-access-s985w") pod "network-check-target-c6wmg" (UID: "f018324f-f63f-41e0-9a3f-d6fed83b4f1c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 02:01:43.903739 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:43.903691 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d6a986a-c066-4ce3-a49c-4bab5be2f075-metrics-certs\") pod \"network-metrics-daemon-t9bnr\" (UID: \"8d6a986a-c066-4ce3-a49c-4bab5be2f075\") " pod="openshift-multus/network-metrics-daemon-t9bnr" Jul 14 02:01:43.903927 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:43.903821 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45d07b54-1b6f-4f51-bf7c-cc8d287e5789-original-pull-secret\") pod \"global-pull-secret-syncer-wmd5q\" (UID: \"45d07b54-1b6f-4f51-bf7c-cc8d287e5789\") " pod="kube-system/global-pull-secret-syncer-wmd5q" Jul 14 02:01:43.904007 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:43.903961 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 02:01:43.904067 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:43.904030 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45d07b54-1b6f-4f51-bf7c-cc8d287e5789-original-pull-secret podName:45d07b54-1b6f-4f51-bf7c-cc8d287e5789 nodeName:}" failed. No retries permitted until 2026-07-14 02:01:47.904009288 +0000 UTC m=+10.058845934 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/45d07b54-1b6f-4f51-bf7c-cc8d287e5789-original-pull-secret") pod "global-pull-secret-syncer-wmd5q" (UID: "45d07b54-1b6f-4f51-bf7c-cc8d287e5789") : object "kube-system"/"original-pull-secret" not registered Jul 14 02:01:43.904529 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:43.904505 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 02:01:43.904644 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:43.904567 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d6a986a-c066-4ce3-a49c-4bab5be2f075-metrics-certs podName:8d6a986a-c066-4ce3-a49c-4bab5be2f075 nodeName:}" failed. No retries permitted until 2026-07-14 02:01:47.904550142 +0000 UTC m=+10.059386773 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8d6a986a-c066-4ce3-a49c-4bab5be2f075-metrics-certs") pod "network-metrics-daemon-t9bnr" (UID: "8d6a986a-c066-4ce3-a49c-4bab5be2f075") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 02:01:44.261980 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:44.261469 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmd5q" Jul 14 02:01:44.261980 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:44.261607 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wmd5q" podUID="45d07b54-1b6f-4f51-bf7c-cc8d287e5789" Jul 14 02:01:45.259100 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:45.258574 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9bnr" Jul 14 02:01:45.259100 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:45.258738 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t9bnr" podUID="8d6a986a-c066-4ce3-a49c-4bab5be2f075" Jul 14 02:01:45.259100 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:45.258921 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6wmg" Jul 14 02:01:45.259100 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:45.259016 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6wmg" podUID="f018324f-f63f-41e0-9a3f-d6fed83b4f1c" Jul 14 02:01:46.261660 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:46.261102 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmd5q" Jul 14 02:01:46.261660 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:46.261242 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wmd5q" podUID="45d07b54-1b6f-4f51-bf7c-cc8d287e5789" Jul 14 02:01:47.258648 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:47.258593 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6wmg" Jul 14 02:01:47.258648 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:47.258632 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9bnr" Jul 14 02:01:47.258908 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:47.258733 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6wmg" podUID="f018324f-f63f-41e0-9a3f-d6fed83b4f1c" Jul 14 02:01:47.258908 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:47.258825 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t9bnr" podUID="8d6a986a-c066-4ce3-a49c-4bab5be2f075" Jul 14 02:01:47.840249 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:47.839547 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s985w\" (UniqueName: \"kubernetes.io/projected/f018324f-f63f-41e0-9a3f-d6fed83b4f1c-kube-api-access-s985w\") pod \"network-check-target-c6wmg\" (UID: \"f018324f-f63f-41e0-9a3f-d6fed83b4f1c\") " pod="openshift-network-diagnostics/network-check-target-c6wmg" Jul 14 02:01:47.840249 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:47.839769 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 02:01:47.840249 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:47.839793 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 02:01:47.840249 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:47.839807 2577 projected.go:196] Error preparing data for projected volume kube-api-access-s985w for pod openshift-network-diagnostics/network-check-target-c6wmg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 02:01:47.840249 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:47.839868 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f018324f-f63f-41e0-9a3f-d6fed83b4f1c-kube-api-access-s985w podName:f018324f-f63f-41e0-9a3f-d6fed83b4f1c nodeName:}" failed. No retries permitted until 2026-07-14 02:01:55.839848199 +0000 UTC m=+17.994684845 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-s985w" (UniqueName: "kubernetes.io/projected/f018324f-f63f-41e0-9a3f-d6fed83b4f1c-kube-api-access-s985w") pod "network-check-target-c6wmg" (UID: "f018324f-f63f-41e0-9a3f-d6fed83b4f1c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 02:01:47.940558 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:47.940513 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45d07b54-1b6f-4f51-bf7c-cc8d287e5789-original-pull-secret\") pod \"global-pull-secret-syncer-wmd5q\" (UID: \"45d07b54-1b6f-4f51-bf7c-cc8d287e5789\") " pod="kube-system/global-pull-secret-syncer-wmd5q" Jul 14 02:01:47.940754 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:47.940582 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d6a986a-c066-4ce3-a49c-4bab5be2f075-metrics-certs\") pod \"network-metrics-daemon-t9bnr\" (UID: \"8d6a986a-c066-4ce3-a49c-4bab5be2f075\") " pod="openshift-multus/network-metrics-daemon-t9bnr" Jul 14 02:01:47.940754 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:47.940739 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 02:01:47.940878 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:47.940803 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d6a986a-c066-4ce3-a49c-4bab5be2f075-metrics-certs podName:8d6a986a-c066-4ce3-a49c-4bab5be2f075 nodeName:}" failed. No retries permitted until 2026-07-14 02:01:55.940782996 +0000 UTC m=+18.095619626 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8d6a986a-c066-4ce3-a49c-4bab5be2f075-metrics-certs") pod "network-metrics-daemon-t9bnr" (UID: "8d6a986a-c066-4ce3-a49c-4bab5be2f075") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 02:01:47.940945 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:47.940910 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 02:01:47.941000 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:47.940979 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45d07b54-1b6f-4f51-bf7c-cc8d287e5789-original-pull-secret podName:45d07b54-1b6f-4f51-bf7c-cc8d287e5789 nodeName:}" failed. No retries permitted until 2026-07-14 02:01:55.940958698 +0000 UTC m=+18.095795329 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/45d07b54-1b6f-4f51-bf7c-cc8d287e5789-original-pull-secret") pod "global-pull-secret-syncer-wmd5q" (UID: "45d07b54-1b6f-4f51-bf7c-cc8d287e5789") : object "kube-system"/"original-pull-secret" not registered Jul 14 02:01:48.260286 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:48.260137 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmd5q" Jul 14 02:01:48.260286 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:48.260262 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wmd5q" podUID="45d07b54-1b6f-4f51-bf7c-cc8d287e5789" Jul 14 02:01:49.259252 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:49.259218 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6wmg" Jul 14 02:01:49.259823 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:49.259379 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6wmg" podUID="f018324f-f63f-41e0-9a3f-d6fed83b4f1c" Jul 14 02:01:49.259823 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:49.259218 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9bnr" Jul 14 02:01:49.259823 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:49.259514 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t9bnr" podUID="8d6a986a-c066-4ce3-a49c-4bab5be2f075" Jul 14 02:01:50.259653 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:50.259617 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmd5q" Jul 14 02:01:50.260150 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:50.259773 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wmd5q" podUID="45d07b54-1b6f-4f51-bf7c-cc8d287e5789" Jul 14 02:01:51.259535 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:51.259479 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9bnr" Jul 14 02:01:51.259784 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:51.259494 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6wmg" Jul 14 02:01:51.259784 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:51.259625 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t9bnr" podUID="8d6a986a-c066-4ce3-a49c-4bab5be2f075" Jul 14 02:01:51.259784 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:51.259761 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6wmg" podUID="f018324f-f63f-41e0-9a3f-d6fed83b4f1c" Jul 14 02:01:52.259314 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:52.259282 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmd5q" Jul 14 02:01:52.259472 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:52.259393 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wmd5q" podUID="45d07b54-1b6f-4f51-bf7c-cc8d287e5789" Jul 14 02:01:53.259285 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:53.259238 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6wmg" Jul 14 02:01:53.259285 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:53.259262 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9bnr" Jul 14 02:01:53.259763 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:53.259438 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6wmg" podUID="f018324f-f63f-41e0-9a3f-d6fed83b4f1c" Jul 14 02:01:53.259763 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:53.259580 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t9bnr" podUID="8d6a986a-c066-4ce3-a49c-4bab5be2f075" Jul 14 02:01:54.259364 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:54.259329 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmd5q" Jul 14 02:01:54.259813 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:54.259470 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wmd5q" podUID="45d07b54-1b6f-4f51-bf7c-cc8d287e5789" Jul 14 02:01:55.258614 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:55.258580 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9bnr" Jul 14 02:01:55.258614 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:55.258613 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6wmg" Jul 14 02:01:55.258867 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:55.258715 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t9bnr" podUID="8d6a986a-c066-4ce3-a49c-4bab5be2f075" Jul 14 02:01:55.258867 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:55.258819 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6wmg" podUID="f018324f-f63f-41e0-9a3f-d6fed83b4f1c" Jul 14 02:01:55.909634 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:55.909595 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s985w\" (UniqueName: \"kubernetes.io/projected/f018324f-f63f-41e0-9a3f-d6fed83b4f1c-kube-api-access-s985w\") pod \"network-check-target-c6wmg\" (UID: \"f018324f-f63f-41e0-9a3f-d6fed83b4f1c\") " pod="openshift-network-diagnostics/network-check-target-c6wmg" Jul 14 02:01:55.910024 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:55.909726 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 02:01:55.910024 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:55.909745 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 02:01:55.910024 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:55.909757 2577 projected.go:196] Error preparing data for projected volume kube-api-access-s985w for pod openshift-network-diagnostics/network-check-target-c6wmg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 02:01:55.910024 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:55.909810 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f018324f-f63f-41e0-9a3f-d6fed83b4f1c-kube-api-access-s985w podName:f018324f-f63f-41e0-9a3f-d6fed83b4f1c nodeName:}" failed. No retries permitted until 2026-07-14 02:02:11.90979577 +0000 UTC m=+34.064632395 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-s985w" (UniqueName: "kubernetes.io/projected/f018324f-f63f-41e0-9a3f-d6fed83b4f1c-kube-api-access-s985w") pod "network-check-target-c6wmg" (UID: "f018324f-f63f-41e0-9a3f-d6fed83b4f1c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 02:01:56.010568 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:56.010533 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45d07b54-1b6f-4f51-bf7c-cc8d287e5789-original-pull-secret\") pod \"global-pull-secret-syncer-wmd5q\" (UID: \"45d07b54-1b6f-4f51-bf7c-cc8d287e5789\") " pod="kube-system/global-pull-secret-syncer-wmd5q" Jul 14 02:01:56.010728 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:56.010580 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d6a986a-c066-4ce3-a49c-4bab5be2f075-metrics-certs\") pod \"network-metrics-daemon-t9bnr\" (UID: \"8d6a986a-c066-4ce3-a49c-4bab5be2f075\") " pod="openshift-multus/network-metrics-daemon-t9bnr" Jul 14 02:01:56.010728 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:56.010687 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 02:01:56.010728 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:56.010690 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 02:01:56.010943 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:56.010753 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d6a986a-c066-4ce3-a49c-4bab5be2f075-metrics-certs podName:8d6a986a-c066-4ce3-a49c-4bab5be2f075 nodeName:}" failed. No retries permitted until 2026-07-14 02:02:12.010734109 +0000 UTC m=+34.165570736 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8d6a986a-c066-4ce3-a49c-4bab5be2f075-metrics-certs") pod "network-metrics-daemon-t9bnr" (UID: "8d6a986a-c066-4ce3-a49c-4bab5be2f075") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 02:01:56.010943 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:56.010771 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45d07b54-1b6f-4f51-bf7c-cc8d287e5789-original-pull-secret podName:45d07b54-1b6f-4f51-bf7c-cc8d287e5789 nodeName:}" failed. No retries permitted until 2026-07-14 02:02:12.010762659 +0000 UTC m=+34.165599288 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/45d07b54-1b6f-4f51-bf7c-cc8d287e5789-original-pull-secret") pod "global-pull-secret-syncer-wmd5q" (UID: "45d07b54-1b6f-4f51-bf7c-cc8d287e5789") : object "kube-system"/"original-pull-secret" not registered Jul 14 02:01:56.258902 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:56.258867 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmd5q" Jul 14 02:01:56.259046 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:56.258987 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wmd5q" podUID="45d07b54-1b6f-4f51-bf7c-cc8d287e5789" Jul 14 02:01:57.258749 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:57.258717 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9bnr" Jul 14 02:01:57.259131 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:57.258761 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6wmg" Jul 14 02:01:57.259131 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:57.258841 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t9bnr" podUID="8d6a986a-c066-4ce3-a49c-4bab5be2f075" Jul 14 02:01:57.259131 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:57.258976 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6wmg" podUID="f018324f-f63f-41e0-9a3f-d6fed83b4f1c" Jul 14 02:01:58.259667 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:58.259621 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmd5q" Jul 14 02:01:58.260078 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:58.259724 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wmd5q" podUID="45d07b54-1b6f-4f51-bf7c-cc8d287e5789" Jul 14 02:01:59.259098 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:59.259054 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9bnr" Jul 14 02:01:59.259098 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:01:59.259075 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6wmg" Jul 14 02:01:59.259329 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:59.259186 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t9bnr" podUID="8d6a986a-c066-4ce3-a49c-4bab5be2f075" Jul 14 02:01:59.259385 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:01:59.259346 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6wmg" podUID="f018324f-f63f-41e0-9a3f-d6fed83b4f1c" Jul 14 02:02:00.258671 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:00.258639 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmd5q" Jul 14 02:02:00.259133 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:00.258762 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wmd5q" podUID="45d07b54-1b6f-4f51-bf7c-cc8d287e5789" Jul 14 02:02:01.259291 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:01.259088 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6wmg" Jul 14 02:02:01.259966 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:01.259116 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9bnr" Jul 14 02:02:01.259966 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:01.259380 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6wmg" podUID="f018324f-f63f-41e0-9a3f-d6fed83b4f1c" Jul 14 02:02:01.259966 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:01.259431 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t9bnr" podUID="8d6a986a-c066-4ce3-a49c-4bab5be2f075" Jul 14 02:02:01.556841 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:01.556800 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rpl6g" event={"ID":"94fa401d-46a7-45c2-bc57-88c112508139","Type":"ContainerStarted","Data":"c6e99b21173f00bc784c72cd3261c6fe94af738667150b8db8c9fff858b6936d"} Jul 14 02:02:01.558531 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:01.558502 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbb4f" event={"ID":"e60fc957-ce6a-4b22-a7f6-91dfd2ef0267","Type":"ContainerStarted","Data":"a3c11cca89b5d868fa4b670826b3f0438a28d61e931ec1e783d0f3a3035c508e"} Jul 14 02:02:01.560176 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:01.560140 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zc4lr" event={"ID":"62531cce-f69d-4ccd-a6c6-e090b2789022","Type":"ContainerStarted","Data":"ff267ecbffa1418a1dfda90b80d6f9e4adc97bc564a01fec34ffde8f0f5bcf48"} Jul 14 02:02:01.561757 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:01.561724 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bs5bn" event={"ID":"f361ee09-a428-4b20-93c3-b9762d14f012","Type":"ContainerStarted","Data":"bd9d52ccf7d696f1c50562e256f105899d06e5cf18afeaa7f945e32f948ac41d"} Jul 14 02:02:01.563524 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:01.563484 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-k4w2h" event={"ID":"059e3b07-82d1-475b-a7c2-4e9345579238","Type":"ContainerStarted","Data":"64ab04345d3b9a3f1465dcca959ad479388fcce0b50e1b0d846acf473fceded2"} Jul 14 02:02:01.566903 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:01.566877 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" event={"ID":"13c988f5-6d7a-4d10-87c3-d548e48ac1d6","Type":"ContainerStarted","Data":"92ab4410f780a0c105e743e88799663e755d34148236cd4323425beb4c14dac9"} Jul 14 02:02:01.567009 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:01.566955 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" event={"ID":"13c988f5-6d7a-4d10-87c3-d548e48ac1d6","Type":"ContainerStarted","Data":"e197b735c01f32b596c8531fdf536145ca0c551c8efaa1cd2562947b0cf52b30"} Jul 14 02:02:01.567073 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:01.567009 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" event={"ID":"13c988f5-6d7a-4d10-87c3-d548e48ac1d6","Type":"ContainerStarted","Data":"c4b9cd3231a30fe2fe0913ffc5328a6520bcb1adfe23d6f81e0bd4ca706cc2c1"} Jul 14 02:02:01.567073 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:01.567058 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" event={"ID":"13c988f5-6d7a-4d10-87c3-d548e48ac1d6","Type":"ContainerStarted","Data":"e9a898183e30e63e4ff2b0548b622b798869af327d70954e19702cec790dd6d4"} Jul 14 02:02:01.567180 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:01.567102 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" event={"ID":"13c988f5-6d7a-4d10-87c3-d548e48ac1d6","Type":"ContainerStarted","Data":"7044f916f12429593c41339339f43be11f78b8be87168ad9f0f524aec90558d9"} Jul 14 02:02:01.567180 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:01.567145 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" event={"ID":"13c988f5-6d7a-4d10-87c3-d548e48ac1d6","Type":"ContainerStarted","Data":"7e84eadcb97263699d96961ca070fa63165b8ed035d586fb95b1716a7ec2a557"} Jul 14 02:02:01.568557 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:01.568533 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-42rhx" event={"ID":"772c82e2-fe88-43ef-b9d1-069856ce9282","Type":"ContainerStarted","Data":"a0c52ec7eca07cad925edd855be1fa750a0b627c0ef3c4752cdc1e642761ce27"} Jul 14 02:02:01.569926 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:01.569881 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rpl6g" podStartSLOduration=3.304348303 podStartE2EDuration="22.569864744s" totalImagesPullingTime="19.265516441s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 02:01:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:02:01.56938718 +0000 UTC m=+23.724223830" watchObservedRunningTime="2026-07-14 02:02:01.569864744 +0000 UTC m=+23.724701403" Jul 14 02:02:01.570040 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:01.569978 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-25.ec2.internal" podStartSLOduration=21.569969891 podStartE2EDuration="21.569969891s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 02:01:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:01:43.529644205 +0000 UTC m=+5.684480861" watchObservedRunningTime="2026-07-14 02:02:01.569969891 +0000 UTC m=+23.724806543" Jul 14 02:02:01.570539 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:01.570513 2577 generic.go:356] "Generic (PLEG): container finished" podID="3d396e3c-208e-4489-88ee-334cfd664280" containerID="ecf3de44faa835416255ab0eb0300e337e35e2d9ae3192e107732b25dc4433b7" exitCode=0 Jul 14 02:02:01.570629 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:01.570550 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gvljc" event={"ID":"3d396e3c-208e-4489-88ee-334cfd664280","Type":"ContainerDied","Data":"ecf3de44faa835416255ab0eb0300e337e35e2d9ae3192e107732b25dc4433b7"} Jul 14 02:02:01.585890 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:01.585783 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-zc4lr" podStartSLOduration=3.285804698 podStartE2EDuration="22.585762491s" totalImagesPullingTime="19.299957793s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 02:01:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:02:01.584716472 +0000 UTC m=+23.739553121" watchObservedRunningTime="2026-07-14 02:02:01.585762491 +0000 UTC m=+23.740599140" Jul 14 02:02:01.609998 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:01.609940 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-bs5bn" podStartSLOduration=7.885511764 podStartE2EDuration="22.609923529s" totalImagesPullingTime="14.724411765s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:01:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:02:01.609848433 +0000 UTC m=+23.764685083" watchObservedRunningTime="2026-07-14 02:02:01.609923529 +0000 UTC m=+23.764760173" Jul 14 02:02:01.610163 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:01.610113 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-k4w2h" podStartSLOduration=3.346499226 podStartE2EDuration="22.610106679s" totalImagesPullingTime="19.263607453s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 02:01:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:02:01.597678647 +0000 UTC m=+23.752515298" watchObservedRunningTime="2026-07-14 02:02:01.610106679 +0000 UTC m=+23.764943361" Jul 14 02:02:01.645400 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:01.645340 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-42rhx" podStartSLOduration=3.3798167 podStartE2EDuration="22.645317122s" totalImagesPullingTime="19.265500422s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 02:01:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:02:01.625297505 +0000 UTC m=+23.780134150" watchObservedRunningTime="2026-07-14 02:02:01.645317122 +0000 UTC m=+23.800153771" Jul 14 02:02:01.741847 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:01.741795 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:01.804843 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:01.804816 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:02.232411 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:02.232378 2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 14 02:02:02.259265 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:02.259215 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmd5q" Jul 14 02:02:02.259436 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:02.259406 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wmd5q" podUID="45d07b54-1b6f-4f51-bf7c-cc8d287e5789" Jul 14 02:02:02.575026 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:02.574989 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbb4f" event={"ID":"e60fc957-ce6a-4b22-a7f6-91dfd2ef0267","Type":"ContainerStarted","Data":"d673652218f68ecd30078c3c4be3dabad9be4662315a916cc2fcd614a820adb0"} Jul 14 02:02:02.576677 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:02.576647 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-g966p" event={"ID":"a6353ee6-d8ec-42fc-bf8f-3cc00662786b","Type":"ContainerStarted","Data":"1f33e53acbf77ee35f7409915ee79711841e5f156b5d7175990f0123780cb3ae"} Jul 14 02:02:02.591361 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:02.591301 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-g966p" podStartSLOduration=4.360307296 podStartE2EDuration="23.591284241s" totalImagesPullingTime="19.230976945s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 02:01:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:02:02.59078064 +0000 UTC m=+24.745617291" watchObservedRunningTime="2026-07-14 02:02:02.591284241 +0000 UTC m=+24.746120880" Jul 14 02:02:02.709963 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:02.709921 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:02.742968 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:02.742925 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-bs5bn" Jul 14 02:02:02.743601 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:02.743567 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-bs5bn" Jul 14 02:02:02.761502 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:02.761473 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:03.226129 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:03.226028 2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-14T02:02:02.232405209Z","UUID":"0539c660-227d-42c9-b3fa-1f3b43026726","Handler":null,"Name":"","Endpoint":""} Jul 14 02:02:03.227717 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:03.227693 2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 14 02:02:03.227838 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:03.227736 2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 14 02:02:03.259248 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:03.259219 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6wmg" Jul 14 02:02:03.259470 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:03.259219 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9bnr" Jul 14 02:02:03.259470 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:03.259364 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6wmg" podUID="f018324f-f63f-41e0-9a3f-d6fed83b4f1c" Jul 14 02:02:03.259470 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:03.259396 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t9bnr" podUID="8d6a986a-c066-4ce3-a49c-4bab5be2f075" Jul 14 02:02:03.580954 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:03.580919 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbb4f" event={"ID":"e60fc957-ce6a-4b22-a7f6-91dfd2ef0267","Type":"ContainerStarted","Data":"f22ba1457970d3c3391ca83127521263bc6106c9221f232db7dc2c3547a2d52d"} Jul 14 02:02:03.581284 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:03.581250 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-bs5bn" Jul 14 02:02:03.581693 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:03.581674 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-bs5bn" Jul 14 02:02:03.597519 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:03.597468 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbb4f" podStartSLOduration=2.997354584 podStartE2EDuration="24.597450006s" totalImagesPullingTime="21.600095422s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 02:01:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:02:03.597427581 +0000 UTC m=+25.752264230" watchObservedRunningTime="2026-07-14 02:02:03.597450006 +0000 UTC m=+25.752286651" Jul 14 02:02:03.666407 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:03.666361 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:04.259421 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:04.259387 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmd5q" Jul 14 02:02:04.259702 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:04.259520 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wmd5q" podUID="45d07b54-1b6f-4f51-bf7c-cc8d287e5789" Jul 14 02:02:04.587642 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:04.587560 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" event={"ID":"13c988f5-6d7a-4d10-87c3-d548e48ac1d6","Type":"ContainerStarted","Data":"e473f429d9b4ccfd0d8fa4ddf222efce9d6df7f1e64a0f1b8682402a6fdac9f3"} Jul 14 02:02:05.258527 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:05.258494 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9bnr" Jul 14 02:02:05.258718 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:05.258539 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6wmg" Jul 14 02:02:05.258718 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:05.258642 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t9bnr" podUID="8d6a986a-c066-4ce3-a49c-4bab5be2f075" Jul 14 02:02:05.258827 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:05.258741 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6wmg" podUID="f018324f-f63f-41e0-9a3f-d6fed83b4f1c" Jul 14 02:02:06.258842 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:06.258605 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmd5q" Jul 14 02:02:06.259484 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:06.258845 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wmd5q" podUID="45d07b54-1b6f-4f51-bf7c-cc8d287e5789" Jul 14 02:02:06.594042 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:06.593963 2577 generic.go:356] "Generic (PLEG): container finished" podID="3d396e3c-208e-4489-88ee-334cfd664280" containerID="ff36684b59eae1ccb4a2c1076862d512814e41120360ccd5b3de27c92df03c32" exitCode=0 Jul 14 02:02:06.594262 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:06.594057 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gvljc" event={"ID":"3d396e3c-208e-4489-88ee-334cfd664280","Type":"ContainerDied","Data":"ff36684b59eae1ccb4a2c1076862d512814e41120360ccd5b3de27c92df03c32"} Jul 14 02:02:06.597872 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:06.597850 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" event={"ID":"13c988f5-6d7a-4d10-87c3-d548e48ac1d6","Type":"ContainerStarted","Data":"ec2cbe4abba23bd003d3617274d70d49b1fc3e628e8d99067cd35024a472f6ff"} Jul 14 02:02:06.598119 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:06.598102 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:02:06.598187 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:06.598157 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:02:06.613603 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:06.613580 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:02:06.648697 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:06.648647 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" podStartSLOduration=8.307943472 podStartE2EDuration="27.6486317s" totalImagesPullingTime="19.340688228s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 02:01:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:02:06.648396871 +0000 UTC m=+28.803233519" watchObservedRunningTime="2026-07-14 02:02:06.6486317 +0000 UTC m=+28.803468347" Jul 14 02:02:07.259432 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:07.259400 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9bnr" Jul 14 02:02:07.259883 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:07.259432 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6wmg" Jul 14 02:02:07.259883 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:07.259526 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t9bnr" podUID="8d6a986a-c066-4ce3-a49c-4bab5be2f075" Jul 14 02:02:07.259883 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:07.259606 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6wmg" podUID="f018324f-f63f-41e0-9a3f-d6fed83b4f1c" Jul 14 02:02:07.600654 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:07.600630 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:02:07.615919 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:07.615896 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:02:07.710783 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:07.710752 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wmd5q"] Jul 14 02:02:07.710937 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:07.710924 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmd5q" Jul 14 02:02:07.711048 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:07.711026 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wmd5q" podUID="45d07b54-1b6f-4f51-bf7c-cc8d287e5789" Jul 14 02:02:07.713480 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:07.713453 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-c6wmg"] Jul 14 02:02:07.713587 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:07.713557 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6wmg" Jul 14 02:02:07.713678 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:07.713658 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6wmg" podUID="f018324f-f63f-41e0-9a3f-d6fed83b4f1c" Jul 14 02:02:07.714027 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:07.714003 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-t9bnr"] Jul 14 02:02:07.714137 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:07.714109 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9bnr" Jul 14 02:02:07.714228 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:07.714212 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t9bnr" podUID="8d6a986a-c066-4ce3-a49c-4bab5be2f075" Jul 14 02:02:08.603981 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:08.603950 2577 generic.go:356] "Generic (PLEG): container finished" podID="3d396e3c-208e-4489-88ee-334cfd664280" containerID="135fac484f76f66513475fa15449593ec1db4424e6259c6c36d723068ddda92f" exitCode=0 Jul 14 02:02:08.604482 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:08.604049 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gvljc" event={"ID":"3d396e3c-208e-4489-88ee-334cfd664280","Type":"ContainerDied","Data":"135fac484f76f66513475fa15449593ec1db4424e6259c6c36d723068ddda92f"} Jul 14 02:02:09.231360 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:09.231166 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:09.259234 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:09.259193 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmd5q" Jul 14 02:02:09.259387 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:09.259240 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6wmg" Jul 14 02:02:09.259387 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:09.259241 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9bnr" Jul 14 02:02:09.259387 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:09.259353 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wmd5q" podUID="45d07b54-1b6f-4f51-bf7c-cc8d287e5789" Jul 14 02:02:09.259483 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:09.259456 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6wmg" podUID="f018324f-f63f-41e0-9a3f-d6fed83b4f1c" Jul 14 02:02:09.259552 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:09.259538 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t9bnr" podUID="8d6a986a-c066-4ce3-a49c-4bab5be2f075" Jul 14 02:02:09.334648 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:09.334611 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:09.392924 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:09.392892 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:10.293895 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:10.293860 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:10.348646 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:10.348618 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:10.611009 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:10.610983 2577 generic.go:356] "Generic (PLEG): container finished" podID="3d396e3c-208e-4489-88ee-334cfd664280" containerID="df73e9500547a2ef333558f2bf01439d10be1785dc47c39678d02ad0b0d61459" exitCode=0 Jul 14 02:02:10.611153 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:10.611038 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gvljc" event={"ID":"3d396e3c-208e-4489-88ee-334cfd664280","Type":"ContainerDied","Data":"df73e9500547a2ef333558f2bf01439d10be1785dc47c39678d02ad0b0d61459"} Jul 14 02:02:11.258999 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:11.258905 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmd5q" Jul 14 02:02:11.258999 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:11.258953 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6wmg" Jul 14 02:02:11.259244 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:11.259017 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9bnr" Jul 14 02:02:11.259244 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:11.259033 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wmd5q" podUID="45d07b54-1b6f-4f51-bf7c-cc8d287e5789" Jul 14 02:02:11.259244 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:11.259165 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t9bnr" podUID="8d6a986a-c066-4ce3-a49c-4bab5be2f075" Jul 14 02:02:11.259400 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:11.259256 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6wmg" podUID="f018324f-f63f-41e0-9a3f-d6fed83b4f1c" Jul 14 02:02:11.924391 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:11.924346 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s985w\" (UniqueName: \"kubernetes.io/projected/f018324f-f63f-41e0-9a3f-d6fed83b4f1c-kube-api-access-s985w\") pod \"network-check-target-c6wmg\" (UID: \"f018324f-f63f-41e0-9a3f-d6fed83b4f1c\") " pod="openshift-network-diagnostics/network-check-target-c6wmg" Jul 14 02:02:11.924994 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:11.924489 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 02:02:11.924994 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:11.924506 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 02:02:11.924994 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:11.924515 2577 projected.go:196] Error preparing data for projected volume kube-api-access-s985w for pod openshift-network-diagnostics/network-check-target-c6wmg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 02:02:11.924994 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:11.924578 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f018324f-f63f-41e0-9a3f-d6fed83b4f1c-kube-api-access-s985w podName:f018324f-f63f-41e0-9a3f-d6fed83b4f1c nodeName:}" failed. No retries permitted until 2026-07-14 02:02:43.924558912 +0000 UTC m=+66.079395553 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-s985w" (UniqueName: "kubernetes.io/projected/f018324f-f63f-41e0-9a3f-d6fed83b4f1c-kube-api-access-s985w") pod "network-check-target-c6wmg" (UID: "f018324f-f63f-41e0-9a3f-d6fed83b4f1c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 02:02:12.025589 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:12.025555 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45d07b54-1b6f-4f51-bf7c-cc8d287e5789-original-pull-secret\") pod \"global-pull-secret-syncer-wmd5q\" (UID: \"45d07b54-1b6f-4f51-bf7c-cc8d287e5789\") " pod="kube-system/global-pull-secret-syncer-wmd5q" Jul 14 02:02:12.025766 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:12.025605 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d6a986a-c066-4ce3-a49c-4bab5be2f075-metrics-certs\") pod \"network-metrics-daemon-t9bnr\" (UID: \"8d6a986a-c066-4ce3-a49c-4bab5be2f075\") " pod="openshift-multus/network-metrics-daemon-t9bnr" Jul 14 02:02:12.025766 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:12.025726 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 02:02:12.025877 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:12.025787 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d6a986a-c066-4ce3-a49c-4bab5be2f075-metrics-certs podName:8d6a986a-c066-4ce3-a49c-4bab5be2f075 nodeName:}" failed. No retries permitted until 2026-07-14 02:02:44.025769387 +0000 UTC m=+66.180606014 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8d6a986a-c066-4ce3-a49c-4bab5be2f075-metrics-certs") pod "network-metrics-daemon-t9bnr" (UID: "8d6a986a-c066-4ce3-a49c-4bab5be2f075") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 02:02:12.025877 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:12.025727 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 02:02:12.025877 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:12.025864 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45d07b54-1b6f-4f51-bf7c-cc8d287e5789-original-pull-secret podName:45d07b54-1b6f-4f51-bf7c-cc8d287e5789 nodeName:}" failed. No retries permitted until 2026-07-14 02:02:44.025847889 +0000 UTC m=+66.180684519 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/45d07b54-1b6f-4f51-bf7c-cc8d287e5789-original-pull-secret") pod "global-pull-secret-syncer-wmd5q" (UID: "45d07b54-1b6f-4f51-bf7c-cc8d287e5789") : object "kube-system"/"original-pull-secret" not registered Jul 14 02:02:12.250660 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:12.250581 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:13.259327 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:13.259299 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmd5q" Jul 14 02:02:13.259815 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:13.259299 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9bnr" Jul 14 02:02:13.259815 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:13.259413 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wmd5q" podUID="45d07b54-1b6f-4f51-bf7c-cc8d287e5789" Jul 14 02:02:13.259815 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:13.259524 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t9bnr" podUID="8d6a986a-c066-4ce3-a49c-4bab5be2f075" Jul 14 02:02:13.259815 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:13.259306 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6wmg" Jul 14 02:02:13.259815 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:13.259628 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6wmg" podUID="f018324f-f63f-41e0-9a3f-d6fed83b4f1c" Jul 14 02:02:13.572649 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:13.572584 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-25.ec2.internal" event="NodeReady" Jul 14 02:02:13.572778 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:13.572705 2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 14 02:02:13.614460 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:13.614432 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6pg9z"] Jul 14 02:02:13.619617 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:13.619594 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-j4k5p"] Jul 14 02:02:13.619754 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:13.619738 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6pg9z" Jul 14 02:02:13.621654 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:13.621632 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 14 02:02:13.621654 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:13.621649 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 14 02:02:13.621960 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:13.621947 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qrrwt\"" Jul 14 02:02:13.623030 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:13.623011 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-j4k5p" Jul 14 02:02:13.624853 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:13.624830 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 14 02:02:13.625003 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:13.624881 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 14 02:02:13.625003 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:13.624962 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jlvpc\"" Jul 14 02:02:13.625209 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:13.625192 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 14 02:02:13.626490 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:13.626455 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6pg9z"] Jul 14 02:02:13.629722 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:13.629677 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-j4k5p"] Jul 14 02:02:13.738214 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:13.738181 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpbdg\" (UniqueName: \"kubernetes.io/projected/29fa5f87-8e2f-4d38-bc0f-2d6584e8016b-kube-api-access-mpbdg\") pod \"ingress-canary-j4k5p\" (UID: \"29fa5f87-8e2f-4d38-bc0f-2d6584e8016b\") " pod="openshift-ingress-canary/ingress-canary-j4k5p" Jul 14 02:02:13.738434 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:13.738255 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29fa5f87-8e2f-4d38-bc0f-2d6584e8016b-cert\") pod \"ingress-canary-j4k5p\" (UID: \"29fa5f87-8e2f-4d38-bc0f-2d6584e8016b\") " pod="openshift-ingress-canary/ingress-canary-j4k5p" Jul 14 02:02:13.738434 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:13.738343 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a9eccb8f-2ee4-4534-a4f0-60a83fc0879b-config-volume\") pod \"dns-default-6pg9z\" (UID: \"a9eccb8f-2ee4-4534-a4f0-60a83fc0879b\") " pod="openshift-dns/dns-default-6pg9z" Jul 14 02:02:13.738434 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:13.738385 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4tgm\" (UniqueName: \"kubernetes.io/projected/a9eccb8f-2ee4-4534-a4f0-60a83fc0879b-kube-api-access-b4tgm\") pod \"dns-default-6pg9z\" (UID: \"a9eccb8f-2ee4-4534-a4f0-60a83fc0879b\") " pod="openshift-dns/dns-default-6pg9z" Jul 14 02:02:13.738434 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:13.738418 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9eccb8f-2ee4-4534-a4f0-60a83fc0879b-metrics-tls\") pod \"dns-default-6pg9z\" (UID: \"a9eccb8f-2ee4-4534-a4f0-60a83fc0879b\") " pod="openshift-dns/dns-default-6pg9z" Jul 14 02:02:13.738656 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:13.738456 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9eccb8f-2ee4-4534-a4f0-60a83fc0879b-tmp-dir\") pod \"dns-default-6pg9z\" (UID: \"a9eccb8f-2ee4-4534-a4f0-60a83fc0879b\") " pod="openshift-dns/dns-default-6pg9z" Jul 14 02:02:13.739410 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:13.739388 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:13.839813 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:13.839719 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpbdg\" (UniqueName: \"kubernetes.io/projected/29fa5f87-8e2f-4d38-bc0f-2d6584e8016b-kube-api-access-mpbdg\") pod \"ingress-canary-j4k5p\" (UID: \"29fa5f87-8e2f-4d38-bc0f-2d6584e8016b\") " pod="openshift-ingress-canary/ingress-canary-j4k5p" Jul 14 02:02:13.839813 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:13.839760 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29fa5f87-8e2f-4d38-bc0f-2d6584e8016b-cert\") pod \"ingress-canary-j4k5p\" (UID: \"29fa5f87-8e2f-4d38-bc0f-2d6584e8016b\") " pod="openshift-ingress-canary/ingress-canary-j4k5p" Jul 14 02:02:13.840038 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:13.839815 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a9eccb8f-2ee4-4534-a4f0-60a83fc0879b-config-volume\") pod \"dns-default-6pg9z\" (UID: \"a9eccb8f-2ee4-4534-a4f0-60a83fc0879b\") " pod="openshift-dns/dns-default-6pg9z" Jul 14 02:02:13.840146 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:13.840037 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4tgm\" (UniqueName: \"kubernetes.io/projected/a9eccb8f-2ee4-4534-a4f0-60a83fc0879b-kube-api-access-b4tgm\") pod \"dns-default-6pg9z\" (UID: \"a9eccb8f-2ee4-4534-a4f0-60a83fc0879b\") " pod="openshift-dns/dns-default-6pg9z" Jul 14 02:02:13.840146 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:13.840105 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9eccb8f-2ee4-4534-a4f0-60a83fc0879b-metrics-tls\") pod \"dns-default-6pg9z\" (UID: \"a9eccb8f-2ee4-4534-a4f0-60a83fc0879b\") " pod="openshift-dns/dns-default-6pg9z" Jul 14 02:02:13.840146 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:13.840131 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9eccb8f-2ee4-4534-a4f0-60a83fc0879b-tmp-dir\") pod \"dns-default-6pg9z\" (UID: \"a9eccb8f-2ee4-4534-a4f0-60a83fc0879b\") " pod="openshift-dns/dns-default-6pg9z" Jul 14 02:02:13.840266 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:13.840249 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 02:02:13.840356 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:13.840339 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9eccb8f-2ee4-4534-a4f0-60a83fc0879b-metrics-tls podName:a9eccb8f-2ee4-4534-a4f0-60a83fc0879b nodeName:}" failed. No retries permitted until 2026-07-14 02:02:14.340317047 +0000 UTC m=+36.495153687 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a9eccb8f-2ee4-4534-a4f0-60a83fc0879b-metrics-tls") pod "dns-default-6pg9z" (UID: "a9eccb8f-2ee4-4534-a4f0-60a83fc0879b") : secret "dns-default-metrics-tls" not found Jul 14 02:02:13.840540 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:13.840515 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9eccb8f-2ee4-4534-a4f0-60a83fc0879b-tmp-dir\") pod \"dns-default-6pg9z\" (UID: \"a9eccb8f-2ee4-4534-a4f0-60a83fc0879b\") " pod="openshift-dns/dns-default-6pg9z" Jul 14 02:02:13.840661 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:13.840522 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a9eccb8f-2ee4-4534-a4f0-60a83fc0879b-config-volume\") pod \"dns-default-6pg9z\" (UID: \"a9eccb8f-2ee4-4534-a4f0-60a83fc0879b\") " pod="openshift-dns/dns-default-6pg9z" Jul 14 02:02:13.844836 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:13.844814 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29fa5f87-8e2f-4d38-bc0f-2d6584e8016b-cert\") pod \"ingress-canary-j4k5p\" (UID: \"29fa5f87-8e2f-4d38-bc0f-2d6584e8016b\") " pod="openshift-ingress-canary/ingress-canary-j4k5p" Jul 14 02:02:13.850769 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:13.850739 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpbdg\" (UniqueName: \"kubernetes.io/projected/29fa5f87-8e2f-4d38-bc0f-2d6584e8016b-kube-api-access-mpbdg\") pod \"ingress-canary-j4k5p\" (UID: \"29fa5f87-8e2f-4d38-bc0f-2d6584e8016b\") " pod="openshift-ingress-canary/ingress-canary-j4k5p" Jul 14 02:02:13.853987 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:13.853959 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4tgm\" (UniqueName: \"kubernetes.io/projected/a9eccb8f-2ee4-4534-a4f0-60a83fc0879b-kube-api-access-b4tgm\") pod \"dns-default-6pg9z\" (UID: \"a9eccb8f-2ee4-4534-a4f0-60a83fc0879b\") " pod="openshift-dns/dns-default-6pg9z" Jul 14 02:02:13.937944 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:13.937907 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-j4k5p" Jul 14 02:02:14.343357 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:14.343322 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9eccb8f-2ee4-4534-a4f0-60a83fc0879b-metrics-tls\") pod \"dns-default-6pg9z\" (UID: \"a9eccb8f-2ee4-4534-a4f0-60a83fc0879b\") " pod="openshift-dns/dns-default-6pg9z" Jul 14 02:02:14.343897 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:14.343516 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 02:02:14.343897 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:14.343593 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9eccb8f-2ee4-4534-a4f0-60a83fc0879b-metrics-tls podName:a9eccb8f-2ee4-4534-a4f0-60a83fc0879b nodeName:}" failed. No retries permitted until 2026-07-14 02:02:15.343573583 +0000 UTC m=+37.498410215 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a9eccb8f-2ee4-4534-a4f0-60a83fc0879b-metrics-tls") pod "dns-default-6pg9z" (UID: "a9eccb8f-2ee4-4534-a4f0-60a83fc0879b") : secret "dns-default-metrics-tls" not found Jul 14 02:02:15.202179 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.201982 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-j4k5p"] Jul 14 02:02:15.205341 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:02:15.205312 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29fa5f87_8e2f_4d38_bc0f_2d6584e8016b.slice/crio-5574b4e794098770356b4fa88c3eea8e24857345161ab4789801aad46586c7fd WatchSource:0}: Error finding container 5574b4e794098770356b4fa88c3eea8e24857345161ab4789801aad46586c7fd: Status 404 returned error can't find the container with id 5574b4e794098770356b4fa88c3eea8e24857345161ab4789801aad46586c7fd Jul 14 02:02:15.258545 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.258524 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmd5q" Jul 14 02:02:15.258621 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.258560 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9bnr" Jul 14 02:02:15.258667 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.258635 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6wmg" Jul 14 02:02:15.260659 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.260640 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mrlfg\"" Jul 14 02:02:15.260659 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.260651 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 14 02:02:15.261309 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.261288 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 02:02:15.261422 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.261314 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 02:02:15.261422 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.261374 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-klxhd\"" Jul 14 02:02:15.262194 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.262179 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 14 02:02:15.348769 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.348747 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9eccb8f-2ee4-4534-a4f0-60a83fc0879b-metrics-tls\") pod \"dns-default-6pg9z\" (UID: \"a9eccb8f-2ee4-4534-a4f0-60a83fc0879b\") " pod="openshift-dns/dns-default-6pg9z" Jul 14 02:02:15.349019 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:15.348884 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 02:02:15.349019 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:15.348944 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9eccb8f-2ee4-4534-a4f0-60a83fc0879b-metrics-tls podName:a9eccb8f-2ee4-4534-a4f0-60a83fc0879b nodeName:}" failed. No retries permitted until 2026-07-14 02:02:17.348929821 +0000 UTC m=+39.503766446 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a9eccb8f-2ee4-4534-a4f0-60a83fc0879b-metrics-tls") pod "dns-default-6pg9z" (UID: "a9eccb8f-2ee4-4534-a4f0-60a83fc0879b") : secret "dns-default-metrics-tls" not found Jul 14 02:02:15.625703 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.625671 2577 generic.go:356] "Generic (PLEG): container finished" podID="3d396e3c-208e-4489-88ee-334cfd664280" containerID="b1ddb48930181a09209748c2a6aa715988190243bcf3a628212b1037ce5916dc" exitCode=0 Jul 14 02:02:15.625895 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.625746 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gvljc" event={"ID":"3d396e3c-208e-4489-88ee-334cfd664280","Type":"ContainerDied","Data":"b1ddb48930181a09209748c2a6aa715988190243bcf3a628212b1037ce5916dc"} Jul 14 02:02:15.626962 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.626939 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-j4k5p" event={"ID":"29fa5f87-8e2f-4d38-bc0f-2d6584e8016b","Type":"ContainerStarted","Data":"5574b4e794098770356b4fa88c3eea8e24857345161ab4789801aad46586c7fd"} Jul 14 02:02:15.720169 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.720144 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-pb8mx"] Jul 14 02:02:15.723648 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.723626 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-pb8mx" Jul 14 02:02:15.726927 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.726904 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-zvhdm\"" Jul 14 02:02:15.727664 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.727641 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rn5p6"] Jul 14 02:02:15.730702 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.730680 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6497d48c68-d9xm4"] Jul 14 02:02:15.730826 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.730806 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rn5p6" Jul 14 02:02:15.733588 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.733569 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-8f44474b6-jrxf5"] Jul 14 02:02:15.733726 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.733707 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" Jul 14 02:02:15.738576 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.734691 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 14 02:02:15.738576 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.734948 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-twgrp\"" Jul 14 02:02:15.738576 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.735190 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 14 02:02:15.738576 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.736011 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 14 02:02:15.738576 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.736029 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 14 02:02:15.738576 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.737106 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 14 02:02:15.738576 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.737369 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 14 02:02:15.738576 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.738391 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 14 02:02:15.738576 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.738486 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-fmxrd\"" Jul 14 02:02:15.739619 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.739552 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-8f44474b6-jrxf5" Jul 14 02:02:15.744619 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.744595 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 14 02:02:15.744890 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.744872 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-pb8mx"] Jul 14 02:02:15.745035 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.744949 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 14 02:02:15.745256 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.745237 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 14 02:02:15.745387 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.745368 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-lq5mv\"" Jul 14 02:02:15.745564 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.745547 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 14 02:02:15.745831 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.745726 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 14 02:02:15.746425 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.746405 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 14 02:02:15.750622 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.750135 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rn5p6"] Jul 14 02:02:15.751223 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.751199 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-8f44474b6-jrxf5"] Jul 14 02:02:15.751362 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.751337 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 14 02:02:15.758427 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.758405 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6497d48c68-d9xm4"] Jul 14 02:02:15.836774 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.836740 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-bkdjp"] Jul 14 02:02:15.839831 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.839814 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-bkdjp" Jul 14 02:02:15.841746 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.841726 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 14 02:02:15.841866 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.841761 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 14 02:02:15.841866 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.841831 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-s2g87\"" Jul 14 02:02:15.842049 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.842030 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 14 02:02:15.842142 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.842097 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 14 02:02:15.847885 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.847707 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 14 02:02:15.852360 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.852143 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/998e452c-dc1f-4e3d-862e-7365d5a216e7-metrics-certs\") pod \"router-default-8f44474b6-jrxf5\" (UID: \"998e452c-dc1f-4e3d-862e-7365d5a216e7\") " pod="openshift-ingress/router-default-8f44474b6-jrxf5" Jul 14 02:02:15.852735 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.852701 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/998e452c-dc1f-4e3d-862e-7365d5a216e7-default-certificate\") pod \"router-default-8f44474b6-jrxf5\" (UID: \"998e452c-dc1f-4e3d-862e-7365d5a216e7\") " pod="openshift-ingress/router-default-8f44474b6-jrxf5" Jul 14 02:02:15.852881 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.852854 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/998e452c-dc1f-4e3d-862e-7365d5a216e7-stats-auth\") pod \"router-default-8f44474b6-jrxf5\" (UID: \"998e452c-dc1f-4e3d-862e-7365d5a216e7\") " pod="openshift-ingress/router-default-8f44474b6-jrxf5" Jul 14 02:02:15.852980 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.852917 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-registry-certificates\") pod \"image-registry-6497d48c68-d9xm4\" (UID: \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\") " pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" Jul 14 02:02:15.852980 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.852942 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6j5cc\" (UniqueName: \"kubernetes.io/projected/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-kube-api-access-6j5cc\") pod \"image-registry-6497d48c68-d9xm4\" (UID: \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\") " pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" Jul 14 02:02:15.853092 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.853001 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tv7hl\" (UniqueName: \"kubernetes.io/projected/d76325b8-c23d-4872-b1ce-a612cac8ad43-kube-api-access-tv7hl\") pod \"network-check-source-869dcfcdcd-pb8mx\" (UID: \"d76325b8-c23d-4872-b1ce-a612cac8ad43\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-pb8mx" Jul 14 02:02:15.853092 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.853033 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-image-registry-private-configuration\") pod \"image-registry-6497d48c68-d9xm4\" (UID: \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\") " pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" Jul 14 02:02:15.853092 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.853061 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-registry-tls\") pod \"image-registry-6497d48c68-d9xm4\" (UID: \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\") " pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" Jul 14 02:02:15.853092 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.853083 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-trusted-ca\") pod \"image-registry-6497d48c68-d9xm4\" (UID: \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\") " pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" Jul 14 02:02:15.853318 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.853110 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz6w2\" (UniqueName: \"kubernetes.io/projected/998e452c-dc1f-4e3d-862e-7365d5a216e7-kube-api-access-jz6w2\") pod \"router-default-8f44474b6-jrxf5\" (UID: \"998e452c-dc1f-4e3d-862e-7365d5a216e7\") " pod="openshift-ingress/router-default-8f44474b6-jrxf5" Jul 14 02:02:15.853318 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.853134 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/78343924-8733-41ae-9863-94ec59df33d5-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-rn5p6\" (UID: \"78343924-8733-41ae-9863-94ec59df33d5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rn5p6" Jul 14 02:02:15.853318 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.853157 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srg6q\" (UniqueName: \"kubernetes.io/projected/78343924-8733-41ae-9863-94ec59df33d5-kube-api-access-srg6q\") pod \"kube-storage-version-migrator-operator-799cb7f958-rn5p6\" (UID: \"78343924-8733-41ae-9863-94ec59df33d5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rn5p6" Jul 14 02:02:15.853318 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.853224 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-ca-trust-extracted\") pod \"image-registry-6497d48c68-d9xm4\" (UID: \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\") " pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" Jul 14 02:02:15.853318 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.853255 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-bound-sa-token\") pod \"image-registry-6497d48c68-d9xm4\" (UID: \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\") " pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" Jul 14 02:02:15.853318 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.853310 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/998e452c-dc1f-4e3d-862e-7365d5a216e7-service-ca-bundle\") pod \"router-default-8f44474b6-jrxf5\" (UID: \"998e452c-dc1f-4e3d-862e-7365d5a216e7\") " pod="openshift-ingress/router-default-8f44474b6-jrxf5" Jul 14 02:02:15.853607 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.853341 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-installation-pull-secrets\") pod \"image-registry-6497d48c68-d9xm4\" (UID: \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\") " pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" Jul 14 02:02:15.853607 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.853401 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/78343924-8733-41ae-9863-94ec59df33d5-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-rn5p6\" (UID: \"78343924-8733-41ae-9863-94ec59df33d5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rn5p6" Jul 14 02:02:15.853834 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.853776 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-bkdjp"] Jul 14 02:02:15.910657 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.910579 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:15.929484 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.929450 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-w4lsz"] Jul 14 02:02:15.932694 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.932670 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-w4lsz" Jul 14 02:02:15.934966 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.934912 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 14 02:02:15.934966 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.934929 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 14 02:02:15.934966 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.934947 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-jztg2\"" Jul 14 02:02:15.935264 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.935246 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 14 02:02:15.945020 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.944996 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-w4lsz"] Jul 14 02:02:15.954868 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.954838 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-installation-pull-secrets\") pod \"image-registry-6497d48c68-d9xm4\" (UID: \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\") " pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" Jul 14 02:02:15.954973 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.954891 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/78343924-8733-41ae-9863-94ec59df33d5-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-rn5p6\" (UID: \"78343924-8733-41ae-9863-94ec59df33d5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rn5p6" Jul 14 02:02:15.954973 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.954929 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/958c7626-e6d4-4fdd-aed2-7632ac9f9fb0-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-w4lsz\" (UID: \"958c7626-e6d4-4fdd-aed2-7632ac9f9fb0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-w4lsz" Jul 14 02:02:15.954973 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.954968 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-ca-trust-extracted\") pod \"image-registry-6497d48c68-d9xm4\" (UID: \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\") " pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" Jul 14 02:02:15.955182 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.955005 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/998e452c-dc1f-4e3d-862e-7365d5a216e7-default-certificate\") pod \"router-default-8f44474b6-jrxf5\" (UID: \"998e452c-dc1f-4e3d-862e-7365d5a216e7\") " pod="openshift-ingress/router-default-8f44474b6-jrxf5" Jul 14 02:02:15.955182 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.955057 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bbc58bc3-8649-42d2-a277-c27b1c6670c9-tmp\") pod \"insights-operator-68d9dd94cf-bkdjp\" (UID: \"bbc58bc3-8649-42d2-a277-c27b1c6670c9\") " pod="openshift-insights/insights-operator-68d9dd94cf-bkdjp" Jul 14 02:02:15.955182 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.955105 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-image-registry-private-configuration\") pod \"image-registry-6497d48c68-d9xm4\" (UID: \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\") " pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" Jul 14 02:02:15.955182 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.955143 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv7hl\" (UniqueName: \"kubernetes.io/projected/d76325b8-c23d-4872-b1ce-a612cac8ad43-kube-api-access-tv7hl\") pod \"network-check-source-869dcfcdcd-pb8mx\" (UID: \"d76325b8-c23d-4872-b1ce-a612cac8ad43\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-pb8mx" Jul 14 02:02:15.955385 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.955171 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-registry-tls\") pod \"image-registry-6497d48c68-d9xm4\" (UID: \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\") " pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" Jul 14 02:02:15.955385 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.955233 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-trusted-ca\") pod \"image-registry-6497d48c68-d9xm4\" (UID: \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\") " pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" Jul 14 02:02:15.955385 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.955308 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tvzl\" (UniqueName: \"kubernetes.io/projected/958c7626-e6d4-4fdd-aed2-7632ac9f9fb0-kube-api-access-8tvzl\") pod \"cluster-samples-operator-65db858894-w4lsz\" (UID: \"958c7626-e6d4-4fdd-aed2-7632ac9f9fb0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-w4lsz" Jul 14 02:02:15.955385 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.955345 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-bound-sa-token\") pod \"image-registry-6497d48c68-d9xm4\" (UID: \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\") " pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" Jul 14 02:02:15.955552 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.955384 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bbc58bc3-8649-42d2-a277-c27b1c6670c9-snapshots\") pod \"insights-operator-68d9dd94cf-bkdjp\" (UID: \"bbc58bc3-8649-42d2-a277-c27b1c6670c9\") " pod="openshift-insights/insights-operator-68d9dd94cf-bkdjp" Jul 14 02:02:15.955552 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.955418 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/998e452c-dc1f-4e3d-862e-7365d5a216e7-stats-auth\") pod \"router-default-8f44474b6-jrxf5\" (UID: \"998e452c-dc1f-4e3d-862e-7365d5a216e7\") " pod="openshift-ingress/router-default-8f44474b6-jrxf5" Jul 14 02:02:15.955552 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.955448 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz6w2\" (UniqueName: \"kubernetes.io/projected/998e452c-dc1f-4e3d-862e-7365d5a216e7-kube-api-access-jz6w2\") pod \"router-default-8f44474b6-jrxf5\" (UID: \"998e452c-dc1f-4e3d-862e-7365d5a216e7\") " pod="openshift-ingress/router-default-8f44474b6-jrxf5" Jul 14 02:02:15.955552 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.955484 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/998e452c-dc1f-4e3d-862e-7365d5a216e7-metrics-certs\") pod \"router-default-8f44474b6-jrxf5\" (UID: \"998e452c-dc1f-4e3d-862e-7365d5a216e7\") " pod="openshift-ingress/router-default-8f44474b6-jrxf5" Jul 14 02:02:15.955552 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.955514 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/78343924-8733-41ae-9863-94ec59df33d5-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-rn5p6\" (UID: \"78343924-8733-41ae-9863-94ec59df33d5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rn5p6" Jul 14 02:02:15.955763 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.955560 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbc58bc3-8649-42d2-a277-c27b1c6670c9-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-bkdjp\" (UID: \"bbc58bc3-8649-42d2-a277-c27b1c6670c9\") " pod="openshift-insights/insights-operator-68d9dd94cf-bkdjp" Jul 14 02:02:15.955763 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.955596 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmcz8\" (UniqueName: \"kubernetes.io/projected/bbc58bc3-8649-42d2-a277-c27b1c6670c9-kube-api-access-cmcz8\") pod \"insights-operator-68d9dd94cf-bkdjp\" (UID: \"bbc58bc3-8649-42d2-a277-c27b1c6670c9\") " pod="openshift-insights/insights-operator-68d9dd94cf-bkdjp" Jul 14 02:02:15.955763 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.955659 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbc58bc3-8649-42d2-a277-c27b1c6670c9-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-bkdjp\" (UID: \"bbc58bc3-8649-42d2-a277-c27b1c6670c9\") " pod="openshift-insights/insights-operator-68d9dd94cf-bkdjp" Jul 14 02:02:15.955763 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.955709 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6j5cc\" (UniqueName: \"kubernetes.io/projected/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-kube-api-access-6j5cc\") pod \"image-registry-6497d48c68-d9xm4\" (UID: \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\") " pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" Jul 14 02:02:15.955977 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.955746 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-registry-certificates\") pod \"image-registry-6497d48c68-d9xm4\" (UID: \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\") " pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" Jul 14 02:02:15.955977 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.955831 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srg6q\" (UniqueName: \"kubernetes.io/projected/78343924-8733-41ae-9863-94ec59df33d5-kube-api-access-srg6q\") pod \"kube-storage-version-migrator-operator-799cb7f958-rn5p6\" (UID: \"78343924-8733-41ae-9863-94ec59df33d5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rn5p6" Jul 14 02:02:15.955977 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.955872 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/998e452c-dc1f-4e3d-862e-7365d5a216e7-service-ca-bundle\") pod \"router-default-8f44474b6-jrxf5\" (UID: \"998e452c-dc1f-4e3d-862e-7365d5a216e7\") " pod="openshift-ingress/router-default-8f44474b6-jrxf5" Jul 14 02:02:15.955977 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.955904 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bbc58bc3-8649-42d2-a277-c27b1c6670c9-serving-cert\") pod \"insights-operator-68d9dd94cf-bkdjp\" (UID: \"bbc58bc3-8649-42d2-a277-c27b1c6670c9\") " pod="openshift-insights/insights-operator-68d9dd94cf-bkdjp" Jul 14 02:02:15.956907 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.956617 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/78343924-8733-41ae-9863-94ec59df33d5-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-rn5p6\" (UID: \"78343924-8733-41ae-9863-94ec59df33d5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rn5p6" Jul 14 02:02:15.957480 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.957083 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-trusted-ca\") pod \"image-registry-6497d48c68-d9xm4\" (UID: \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\") " pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" Jul 14 02:02:15.957572 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.957545 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-registry-certificates\") pod \"image-registry-6497d48c68-d9xm4\" (UID: \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\") " pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" Jul 14 02:02:15.957711 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:15.957660 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 14 02:02:15.957784 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:15.957771 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/998e452c-dc1f-4e3d-862e-7365d5a216e7-metrics-certs podName:998e452c-dc1f-4e3d-862e-7365d5a216e7 nodeName:}" failed. No retries permitted until 2026-07-14 02:02:16.457751116 +0000 UTC m=+38.612587747 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/998e452c-dc1f-4e3d-862e-7365d5a216e7-metrics-certs") pod "router-default-8f44474b6-jrxf5" (UID: "998e452c-dc1f-4e3d-862e-7365d5a216e7") : secret "router-metrics-certs-default" not found Jul 14 02:02:15.958123 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:15.958005 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/998e452c-dc1f-4e3d-862e-7365d5a216e7-service-ca-bundle podName:998e452c-dc1f-4e3d-862e-7365d5a216e7 nodeName:}" failed. No retries permitted until 2026-07-14 02:02:16.457990156 +0000 UTC m=+38.612826799 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/998e452c-dc1f-4e3d-862e-7365d5a216e7-service-ca-bundle") pod "router-default-8f44474b6-jrxf5" (UID: "998e452c-dc1f-4e3d-862e-7365d5a216e7") : configmap references non-existent config key: service-ca.crt Jul 14 02:02:15.960071 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:15.959393 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 02:02:15.960071 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:15.959415 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6497d48c68-d9xm4: secret "image-registry-tls" not found Jul 14 02:02:15.960071 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:15.959464 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-registry-tls podName:b2ab7b4e-fa36-44f3-86e6-151a58a5631b nodeName:}" failed. No retries permitted until 2026-07-14 02:02:16.459449765 +0000 UTC m=+38.614286408 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-registry-tls") pod "image-registry-6497d48c68-d9xm4" (UID: "b2ab7b4e-fa36-44f3-86e6-151a58a5631b") : secret "image-registry-tls" not found Jul 14 02:02:15.960071 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.959766 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-ca-trust-extracted\") pod \"image-registry-6497d48c68-d9xm4\" (UID: \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\") " pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" Jul 14 02:02:15.962028 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.961846 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/78343924-8733-41ae-9863-94ec59df33d5-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-rn5p6\" (UID: \"78343924-8733-41ae-9863-94ec59df33d5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rn5p6" Jul 14 02:02:15.962028 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.961978 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-image-registry-private-configuration\") pod \"image-registry-6497d48c68-d9xm4\" (UID: \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\") " pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" Jul 14 02:02:15.962139 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.962030 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-installation-pull-secrets\") pod \"image-registry-6497d48c68-d9xm4\" (UID: \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\") " pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" Jul 14 02:02:15.962849 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.962827 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/998e452c-dc1f-4e3d-862e-7365d5a216e7-stats-auth\") pod \"router-default-8f44474b6-jrxf5\" (UID: \"998e452c-dc1f-4e3d-862e-7365d5a216e7\") " pod="openshift-ingress/router-default-8f44474b6-jrxf5" Jul 14 02:02:15.962910 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.962893 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/998e452c-dc1f-4e3d-862e-7365d5a216e7-default-certificate\") pod \"router-default-8f44474b6-jrxf5\" (UID: \"998e452c-dc1f-4e3d-862e-7365d5a216e7\") " pod="openshift-ingress/router-default-8f44474b6-jrxf5" Jul 14 02:02:15.967027 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.966985 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6j5cc\" (UniqueName: \"kubernetes.io/projected/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-kube-api-access-6j5cc\") pod \"image-registry-6497d48c68-d9xm4\" (UID: \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\") " pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" Jul 14 02:02:15.967136 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.967080 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-bound-sa-token\") pod \"image-registry-6497d48c68-d9xm4\" (UID: \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\") " pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" Jul 14 02:02:15.967626 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.967605 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv7hl\" (UniqueName: \"kubernetes.io/projected/d76325b8-c23d-4872-b1ce-a612cac8ad43-kube-api-access-tv7hl\") pod \"network-check-source-869dcfcdcd-pb8mx\" (UID: \"d76325b8-c23d-4872-b1ce-a612cac8ad43\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-pb8mx" Jul 14 02:02:15.967626 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.967615 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srg6q\" (UniqueName: \"kubernetes.io/projected/78343924-8733-41ae-9863-94ec59df33d5-kube-api-access-srg6q\") pod \"kube-storage-version-migrator-operator-799cb7f958-rn5p6\" (UID: \"78343924-8733-41ae-9863-94ec59df33d5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rn5p6" Jul 14 02:02:15.967784 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.967612 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz6w2\" (UniqueName: \"kubernetes.io/projected/998e452c-dc1f-4e3d-862e-7365d5a216e7-kube-api-access-jz6w2\") pod \"router-default-8f44474b6-jrxf5\" (UID: \"998e452c-dc1f-4e3d-862e-7365d5a216e7\") " pod="openshift-ingress/router-default-8f44474b6-jrxf5" Jul 14 02:02:15.984455 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:15.984430 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:16.056808 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:16.056766 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmcz8\" (UniqueName: \"kubernetes.io/projected/bbc58bc3-8649-42d2-a277-c27b1c6670c9-kube-api-access-cmcz8\") pod \"insights-operator-68d9dd94cf-bkdjp\" (UID: \"bbc58bc3-8649-42d2-a277-c27b1c6670c9\") " pod="openshift-insights/insights-operator-68d9dd94cf-bkdjp" Jul 14 02:02:16.056988 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:16.056834 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbc58bc3-8649-42d2-a277-c27b1c6670c9-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-bkdjp\" (UID: \"bbc58bc3-8649-42d2-a277-c27b1c6670c9\") " pod="openshift-insights/insights-operator-68d9dd94cf-bkdjp" Jul 14 02:02:16.056988 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:16.056900 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bbc58bc3-8649-42d2-a277-c27b1c6670c9-serving-cert\") pod \"insights-operator-68d9dd94cf-bkdjp\" (UID: \"bbc58bc3-8649-42d2-a277-c27b1c6670c9\") " pod="openshift-insights/insights-operator-68d9dd94cf-bkdjp" Jul 14 02:02:16.056988 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:16.056934 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/958c7626-e6d4-4fdd-aed2-7632ac9f9fb0-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-w4lsz\" (UID: \"958c7626-e6d4-4fdd-aed2-7632ac9f9fb0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-w4lsz" Jul 14 02:02:16.056988 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:16.056980 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bbc58bc3-8649-42d2-a277-c27b1c6670c9-tmp\") pod \"insights-operator-68d9dd94cf-bkdjp\" (UID: \"bbc58bc3-8649-42d2-a277-c27b1c6670c9\") " pod="openshift-insights/insights-operator-68d9dd94cf-bkdjp" Jul 14 02:02:16.057211 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:16.057078 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 02:02:16.057211 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:16.057136 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tvzl\" (UniqueName: \"kubernetes.io/projected/958c7626-e6d4-4fdd-aed2-7632ac9f9fb0-kube-api-access-8tvzl\") pod \"cluster-samples-operator-65db858894-w4lsz\" (UID: \"958c7626-e6d4-4fdd-aed2-7632ac9f9fb0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-w4lsz" Jul 14 02:02:16.057211 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:16.057160 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/958c7626-e6d4-4fdd-aed2-7632ac9f9fb0-samples-operator-tls podName:958c7626-e6d4-4fdd-aed2-7632ac9f9fb0 nodeName:}" failed. No retries permitted until 2026-07-14 02:02:16.557139829 +0000 UTC m=+38.711976469 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/958c7626-e6d4-4fdd-aed2-7632ac9f9fb0-samples-operator-tls") pod "cluster-samples-operator-65db858894-w4lsz" (UID: "958c7626-e6d4-4fdd-aed2-7632ac9f9fb0") : secret "samples-operator-tls" not found Jul 14 02:02:16.057211 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:16.057200 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bbc58bc3-8649-42d2-a277-c27b1c6670c9-snapshots\") pod \"insights-operator-68d9dd94cf-bkdjp\" (UID: \"bbc58bc3-8649-42d2-a277-c27b1c6670c9\") " pod="openshift-insights/insights-operator-68d9dd94cf-bkdjp" Jul 14 02:02:16.057446 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:16.057241 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbc58bc3-8649-42d2-a277-c27b1c6670c9-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-bkdjp\" (UID: \"bbc58bc3-8649-42d2-a277-c27b1c6670c9\") " pod="openshift-insights/insights-operator-68d9dd94cf-bkdjp" Jul 14 02:02:16.057446 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:16.057418 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bbc58bc3-8649-42d2-a277-c27b1c6670c9-tmp\") pod \"insights-operator-68d9dd94cf-bkdjp\" (UID: \"bbc58bc3-8649-42d2-a277-c27b1c6670c9\") " pod="openshift-insights/insights-operator-68d9dd94cf-bkdjp" Jul 14 02:02:16.057794 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:16.057750 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbc58bc3-8649-42d2-a277-c27b1c6670c9-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-bkdjp\" (UID: \"bbc58bc3-8649-42d2-a277-c27b1c6670c9\") " pod="openshift-insights/insights-operator-68d9dd94cf-bkdjp" Jul 14 02:02:16.057911 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:16.057806 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bbc58bc3-8649-42d2-a277-c27b1c6670c9-snapshots\") pod \"insights-operator-68d9dd94cf-bkdjp\" (UID: \"bbc58bc3-8649-42d2-a277-c27b1c6670c9\") " pod="openshift-insights/insights-operator-68d9dd94cf-bkdjp" Jul 14 02:02:16.057980 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:16.057958 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbc58bc3-8649-42d2-a277-c27b1c6670c9-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-bkdjp\" (UID: \"bbc58bc3-8649-42d2-a277-c27b1c6670c9\") " pod="openshift-insights/insights-operator-68d9dd94cf-bkdjp" Jul 14 02:02:16.059551 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:16.059528 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bbc58bc3-8649-42d2-a277-c27b1c6670c9-serving-cert\") pod \"insights-operator-68d9dd94cf-bkdjp\" (UID: \"bbc58bc3-8649-42d2-a277-c27b1c6670c9\") " pod="openshift-insights/insights-operator-68d9dd94cf-bkdjp" Jul 14 02:02:16.067064 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:16.067036 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tvzl\" (UniqueName: \"kubernetes.io/projected/958c7626-e6d4-4fdd-aed2-7632ac9f9fb0-kube-api-access-8tvzl\") pod \"cluster-samples-operator-65db858894-w4lsz\" (UID: \"958c7626-e6d4-4fdd-aed2-7632ac9f9fb0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-w4lsz" Jul 14 02:02:16.067190 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:16.067163 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmcz8\" (UniqueName: \"kubernetes.io/projected/bbc58bc3-8649-42d2-a277-c27b1c6670c9-kube-api-access-cmcz8\") pod \"insights-operator-68d9dd94cf-bkdjp\" (UID: \"bbc58bc3-8649-42d2-a277-c27b1c6670c9\") " pod="openshift-insights/insights-operator-68d9dd94cf-bkdjp" Jul 14 02:02:16.074933 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:16.074912 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-pb8mx" Jul 14 02:02:16.081487 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:16.081428 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rn5p6" Jul 14 02:02:16.085253 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:16.085227 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:16.150093 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:16.150045 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-bkdjp" Jul 14 02:02:16.220163 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:16.220038 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-pb8mx"] Jul 14 02:02:16.248031 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:16.247820 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rn5p6"] Jul 14 02:02:16.253183 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:02:16.253129 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78343924_8733_41ae_9863_94ec59df33d5.slice/crio-e3b3bcd9a8bb776646c56c6e65e7c11ffb42310f4cbf18bcd73a0c7dc8f21547 WatchSource:0}: Error finding container e3b3bcd9a8bb776646c56c6e65e7c11ffb42310f4cbf18bcd73a0c7dc8f21547: Status 404 returned error can't find the container with id e3b3bcd9a8bb776646c56c6e65e7c11ffb42310f4cbf18bcd73a0c7dc8f21547 Jul 14 02:02:16.313670 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:16.313640 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-bkdjp"] Jul 14 02:02:16.317006 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:02:16.316973 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbc58bc3_8649_42d2_a277_c27b1c6670c9.slice/crio-7ab55ebe47957f1f8f92a03aef3107c99d8f77393c061809d1e3b3f2b5be969f WatchSource:0}: Error finding container 7ab55ebe47957f1f8f92a03aef3107c99d8f77393c061809d1e3b3f2b5be969f: Status 404 returned error can't find the container with id 7ab55ebe47957f1f8f92a03aef3107c99d8f77393c061809d1e3b3f2b5be969f Jul 14 02:02:16.460680 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:16.460602 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/998e452c-dc1f-4e3d-862e-7365d5a216e7-metrics-certs\") pod \"router-default-8f44474b6-jrxf5\" (UID: \"998e452c-dc1f-4e3d-862e-7365d5a216e7\") " pod="openshift-ingress/router-default-8f44474b6-jrxf5" Jul 14 02:02:16.461105 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:16.460710 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/998e452c-dc1f-4e3d-862e-7365d5a216e7-service-ca-bundle\") pod \"router-default-8f44474b6-jrxf5\" (UID: \"998e452c-dc1f-4e3d-862e-7365d5a216e7\") " pod="openshift-ingress/router-default-8f44474b6-jrxf5" Jul 14 02:02:16.461105 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:16.460783 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-registry-tls\") pod \"image-registry-6497d48c68-d9xm4\" (UID: \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\") " pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" Jul 14 02:02:16.461105 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:16.460939 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/998e452c-dc1f-4e3d-862e-7365d5a216e7-service-ca-bundle podName:998e452c-dc1f-4e3d-862e-7365d5a216e7 nodeName:}" failed. No retries permitted until 2026-07-14 02:02:17.460919937 +0000 UTC m=+39.615756565 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/998e452c-dc1f-4e3d-862e-7365d5a216e7-service-ca-bundle") pod "router-default-8f44474b6-jrxf5" (UID: "998e452c-dc1f-4e3d-862e-7365d5a216e7") : configmap references non-existent config key: service-ca.crt Jul 14 02:02:16.461105 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:16.460946 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 14 02:02:16.461105 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:16.461009 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/998e452c-dc1f-4e3d-862e-7365d5a216e7-metrics-certs podName:998e452c-dc1f-4e3d-862e-7365d5a216e7 nodeName:}" failed. No retries permitted until 2026-07-14 02:02:17.460991247 +0000 UTC m=+39.615827893 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/998e452c-dc1f-4e3d-862e-7365d5a216e7-metrics-certs") pod "router-default-8f44474b6-jrxf5" (UID: "998e452c-dc1f-4e3d-862e-7365d5a216e7") : secret "router-metrics-certs-default" not found Jul 14 02:02:16.461105 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:16.461011 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 02:02:16.461105 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:16.461032 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6497d48c68-d9xm4: secret "image-registry-tls" not found Jul 14 02:02:16.461105 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:16.461079 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-registry-tls podName:b2ab7b4e-fa36-44f3-86e6-151a58a5631b nodeName:}" failed. No retries permitted until 2026-07-14 02:02:17.461063332 +0000 UTC m=+39.615899973 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-registry-tls") pod "image-registry-6497d48c68-d9xm4" (UID: "b2ab7b4e-fa36-44f3-86e6-151a58a5631b") : secret "image-registry-tls" not found Jul 14 02:02:16.562241 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:16.562192 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/958c7626-e6d4-4fdd-aed2-7632ac9f9fb0-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-w4lsz\" (UID: \"958c7626-e6d4-4fdd-aed2-7632ac9f9fb0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-w4lsz" Jul 14 02:02:16.562451 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:16.562435 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 02:02:16.562519 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:16.562511 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/958c7626-e6d4-4fdd-aed2-7632ac9f9fb0-samples-operator-tls podName:958c7626-e6d4-4fdd-aed2-7632ac9f9fb0 nodeName:}" failed. No retries permitted until 2026-07-14 02:02:17.562490929 +0000 UTC m=+39.717327561 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/958c7626-e6d4-4fdd-aed2-7632ac9f9fb0-samples-operator-tls") pod "cluster-samples-operator-65db858894-w4lsz" (UID: "958c7626-e6d4-4fdd-aed2-7632ac9f9fb0") : secret "samples-operator-tls" not found Jul 14 02:02:16.631463 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:16.631423 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rn5p6" event={"ID":"78343924-8733-41ae-9863-94ec59df33d5","Type":"ContainerStarted","Data":"e3b3bcd9a8bb776646c56c6e65e7c11ffb42310f4cbf18bcd73a0c7dc8f21547"} Jul 14 02:02:16.633097 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:16.633043 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-bkdjp" event={"ID":"bbc58bc3-8649-42d2-a277-c27b1c6670c9","Type":"ContainerStarted","Data":"7ab55ebe47957f1f8f92a03aef3107c99d8f77393c061809d1e3b3f2b5be969f"} Jul 14 02:02:16.636991 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:16.636910 2577 generic.go:356] "Generic (PLEG): container finished" podID="3d396e3c-208e-4489-88ee-334cfd664280" containerID="b668ef6747a34921084abae0c82536f2f8104906660dcb78ec0d3dc17638043a" exitCode=0 Jul 14 02:02:16.637097 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:16.636993 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gvljc" event={"ID":"3d396e3c-208e-4489-88ee-334cfd664280","Type":"ContainerDied","Data":"b668ef6747a34921084abae0c82536f2f8104906660dcb78ec0d3dc17638043a"} Jul 14 02:02:16.638613 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:16.638416 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-pb8mx" event={"ID":"d76325b8-c23d-4872-b1ce-a612cac8ad43","Type":"ContainerStarted","Data":"2a393007ad7c98df70663e96e3a68336350414d873f3c8ea8ce99f078363f24e"} Jul 14 02:02:17.371200 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:17.370407 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9eccb8f-2ee4-4534-a4f0-60a83fc0879b-metrics-tls\") pod \"dns-default-6pg9z\" (UID: \"a9eccb8f-2ee4-4534-a4f0-60a83fc0879b\") " pod="openshift-dns/dns-default-6pg9z" Jul 14 02:02:17.371200 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:17.370545 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 02:02:17.371200 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:17.370613 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9eccb8f-2ee4-4534-a4f0-60a83fc0879b-metrics-tls podName:a9eccb8f-2ee4-4534-a4f0-60a83fc0879b nodeName:}" failed. No retries permitted until 2026-07-14 02:02:21.370592829 +0000 UTC m=+43.525429459 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a9eccb8f-2ee4-4534-a4f0-60a83fc0879b-metrics-tls") pod "dns-default-6pg9z" (UID: "a9eccb8f-2ee4-4534-a4f0-60a83fc0879b") : secret "dns-default-metrics-tls" not found Jul 14 02:02:17.471152 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:17.471119 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-registry-tls\") pod \"image-registry-6497d48c68-d9xm4\" (UID: \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\") " pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" Jul 14 02:02:17.471904 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:17.471167 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/998e452c-dc1f-4e3d-862e-7365d5a216e7-metrics-certs\") pod \"router-default-8f44474b6-jrxf5\" (UID: \"998e452c-dc1f-4e3d-862e-7365d5a216e7\") " pod="openshift-ingress/router-default-8f44474b6-jrxf5" Jul 14 02:02:17.471904 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:17.471256 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 14 02:02:17.471904 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:17.471287 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/998e452c-dc1f-4e3d-862e-7365d5a216e7-service-ca-bundle\") pod \"router-default-8f44474b6-jrxf5\" (UID: \"998e452c-dc1f-4e3d-862e-7365d5a216e7\") " pod="openshift-ingress/router-default-8f44474b6-jrxf5" Jul 14 02:02:17.471904 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:17.471330 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/998e452c-dc1f-4e3d-862e-7365d5a216e7-metrics-certs podName:998e452c-dc1f-4e3d-862e-7365d5a216e7 nodeName:}" failed. No retries permitted until 2026-07-14 02:02:19.471312387 +0000 UTC m=+41.626149015 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/998e452c-dc1f-4e3d-862e-7365d5a216e7-metrics-certs") pod "router-default-8f44474b6-jrxf5" (UID: "998e452c-dc1f-4e3d-862e-7365d5a216e7") : secret "router-metrics-certs-default" not found Jul 14 02:02:17.471904 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:17.471385 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/998e452c-dc1f-4e3d-862e-7365d5a216e7-service-ca-bundle podName:998e452c-dc1f-4e3d-862e-7365d5a216e7 nodeName:}" failed. No retries permitted until 2026-07-14 02:02:19.471373878 +0000 UTC m=+41.626210504 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/998e452c-dc1f-4e3d-862e-7365d5a216e7-service-ca-bundle") pod "router-default-8f44474b6-jrxf5" (UID: "998e452c-dc1f-4e3d-862e-7365d5a216e7") : configmap references non-existent config key: service-ca.crt Jul 14 02:02:17.471904 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:17.471415 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 02:02:17.471904 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:17.471426 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6497d48c68-d9xm4: secret "image-registry-tls" not found Jul 14 02:02:17.471904 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:17.471461 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-registry-tls podName:b2ab7b4e-fa36-44f3-86e6-151a58a5631b nodeName:}" failed. No retries permitted until 2026-07-14 02:02:19.471447822 +0000 UTC m=+41.626284450 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-registry-tls") pod "image-registry-6497d48c68-d9xm4" (UID: "b2ab7b4e-fa36-44f3-86e6-151a58a5631b") : secret "image-registry-tls" not found Jul 14 02:02:17.572543 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:17.571779 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/958c7626-e6d4-4fdd-aed2-7632ac9f9fb0-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-w4lsz\" (UID: \"958c7626-e6d4-4fdd-aed2-7632ac9f9fb0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-w4lsz" Jul 14 02:02:17.572543 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:17.572017 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 02:02:17.572543 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:17.572081 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/958c7626-e6d4-4fdd-aed2-7632ac9f9fb0-samples-operator-tls podName:958c7626-e6d4-4fdd-aed2-7632ac9f9fb0 nodeName:}" failed. No retries permitted until 2026-07-14 02:02:19.572062139 +0000 UTC m=+41.726898771 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/958c7626-e6d4-4fdd-aed2-7632ac9f9fb0-samples-operator-tls") pod "cluster-samples-operator-65db858894-w4lsz" (UID: "958c7626-e6d4-4fdd-aed2-7632ac9f9fb0") : secret "samples-operator-tls" not found Jul 14 02:02:17.647112 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:17.647073 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gvljc" event={"ID":"3d396e3c-208e-4489-88ee-334cfd664280","Type":"ContainerStarted","Data":"ff89a7119bfb166e413798c12ee62a9d9f1c8609510168e6b449689e2dcef54f"} Jul 14 02:02:17.668896 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:17.668835 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-gvljc" podStartSLOduration=8.587539515 podStartE2EDuration="38.668815893s" totalImagesPullingTime="30.081276378s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 02:01:39 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:02:17.667131472 +0000 UTC m=+39.821968120" watchObservedRunningTime="2026-07-14 02:02:17.668815893 +0000 UTC m=+39.823652542" Jul 14 02:02:17.800042 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:17.792995 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:17.852965 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:17.852788 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:17.942951 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:17.942915 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:18.040143 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:18.040061 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:18.309149 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:18.309114 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:18.365504 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:18.365469 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:19.488730 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:19.488688 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-registry-tls\") pod \"image-registry-6497d48c68-d9xm4\" (UID: \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\") " pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" Jul 14 02:02:19.488730 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:19.488738 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/998e452c-dc1f-4e3d-862e-7365d5a216e7-metrics-certs\") pod \"router-default-8f44474b6-jrxf5\" (UID: \"998e452c-dc1f-4e3d-862e-7365d5a216e7\") " pod="openshift-ingress/router-default-8f44474b6-jrxf5" Jul 14 02:02:19.489331 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:19.488838 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/998e452c-dc1f-4e3d-862e-7365d5a216e7-service-ca-bundle\") pod \"router-default-8f44474b6-jrxf5\" (UID: \"998e452c-dc1f-4e3d-862e-7365d5a216e7\") " pod="openshift-ingress/router-default-8f44474b6-jrxf5" Jul 14 02:02:19.489331 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:19.488905 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 02:02:19.489331 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:19.488918 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6497d48c68-d9xm4: secret "image-registry-tls" not found Jul 14 02:02:19.489331 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:19.488966 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/998e452c-dc1f-4e3d-862e-7365d5a216e7-service-ca-bundle podName:998e452c-dc1f-4e3d-862e-7365d5a216e7 nodeName:}" failed. No retries permitted until 2026-07-14 02:02:23.488951673 +0000 UTC m=+45.643788300 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/998e452c-dc1f-4e3d-862e-7365d5a216e7-service-ca-bundle") pod "router-default-8f44474b6-jrxf5" (UID: "998e452c-dc1f-4e3d-862e-7365d5a216e7") : configmap references non-existent config key: service-ca.crt Jul 14 02:02:19.489331 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:19.488993 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-registry-tls podName:b2ab7b4e-fa36-44f3-86e6-151a58a5631b nodeName:}" failed. No retries permitted until 2026-07-14 02:02:23.488974434 +0000 UTC m=+45.643811060 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-registry-tls") pod "image-registry-6497d48c68-d9xm4" (UID: "b2ab7b4e-fa36-44f3-86e6-151a58a5631b") : secret "image-registry-tls" not found Jul 14 02:02:19.489331 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:19.489036 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 14 02:02:19.489331 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:19.489064 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/998e452c-dc1f-4e3d-862e-7365d5a216e7-metrics-certs podName:998e452c-dc1f-4e3d-862e-7365d5a216e7 nodeName:}" failed. No retries permitted until 2026-07-14 02:02:23.489057783 +0000 UTC m=+45.643894410 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/998e452c-dc1f-4e3d-862e-7365d5a216e7-metrics-certs") pod "router-default-8f44474b6-jrxf5" (UID: "998e452c-dc1f-4e3d-862e-7365d5a216e7") : secret "router-metrics-certs-default" not found Jul 14 02:02:19.589600 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:19.589569 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/958c7626-e6d4-4fdd-aed2-7632ac9f9fb0-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-w4lsz\" (UID: \"958c7626-e6d4-4fdd-aed2-7632ac9f9fb0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-w4lsz" Jul 14 02:02:19.589858 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:19.589736 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 02:02:19.589858 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:19.589810 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/958c7626-e6d4-4fdd-aed2-7632ac9f9fb0-samples-operator-tls podName:958c7626-e6d4-4fdd-aed2-7632ac9f9fb0 nodeName:}" failed. No retries permitted until 2026-07-14 02:02:23.589791558 +0000 UTC m=+45.744628201 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/958c7626-e6d4-4fdd-aed2-7632ac9f9fb0-samples-operator-tls") pod "cluster-samples-operator-65db858894-w4lsz" (UID: "958c7626-e6d4-4fdd-aed2-7632ac9f9fb0") : secret "samples-operator-tls" not found Jul 14 02:02:19.691434 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:19.691399 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:20.129154 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:20.129123 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:20.184456 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:20.184415 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:20.396826 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:20.396715 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:20.497246 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:20.497218 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:21.276654 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:21.276598 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:21.376355 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:21.376322 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:21.405655 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:21.405619 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9eccb8f-2ee4-4534-a4f0-60a83fc0879b-metrics-tls\") pod \"dns-default-6pg9z\" (UID: \"a9eccb8f-2ee4-4534-a4f0-60a83fc0879b\") " pod="openshift-dns/dns-default-6pg9z" Jul 14 02:02:21.405841 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:21.405793 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 02:02:21.405905 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:21.405872 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9eccb8f-2ee4-4534-a4f0-60a83fc0879b-metrics-tls podName:a9eccb8f-2ee4-4534-a4f0-60a83fc0879b nodeName:}" failed. No retries permitted until 2026-07-14 02:02:29.405851549 +0000 UTC m=+51.560688180 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a9eccb8f-2ee4-4534-a4f0-60a83fc0879b-metrics-tls") pod "dns-default-6pg9z" (UID: "a9eccb8f-2ee4-4534-a4f0-60a83fc0879b") : secret "dns-default-metrics-tls" not found Jul 14 02:02:21.659865 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:21.659756 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rn5p6" event={"ID":"78343924-8733-41ae-9863-94ec59df33d5","Type":"ContainerStarted","Data":"b737076df4093cd9e8d92d2780f51d7798ddae142038214ddd8e394955409cec"} Jul 14 02:02:21.661205 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:21.661170 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-j4k5p" event={"ID":"29fa5f87-8e2f-4d38-bc0f-2d6584e8016b","Type":"ContainerStarted","Data":"66f6c3f38d805972c4aeeb6f268f6f27de44b1696671ad394644867cc5eb8151"} Jul 14 02:02:21.662873 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:21.662851 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-bkdjp" event={"ID":"bbc58bc3-8649-42d2-a277-c27b1c6670c9","Type":"ContainerStarted","Data":"e42d635c9ff1ef5e3db5b1347ac9d9cebacaf5d7d749c57d8a2a3828856ebd32"} Jul 14 02:02:21.664451 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:21.664429 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-pb8mx" event={"ID":"d76325b8-c23d-4872-b1ce-a612cac8ad43","Type":"ContainerStarted","Data":"59561cc4bb60320c0dfee2b2ce36a380a8ef708388cb3c52c6d53b0dbd4c9096"} Jul 14 02:02:21.677530 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:21.677487 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rn5p6" podStartSLOduration=1.902885367 podStartE2EDuration="6.677475632s" totalImagesPullingTime="4.774590265s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:02:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:02:21.677132817 +0000 UTC m=+43.831969466" watchObservedRunningTime="2026-07-14 02:02:21.677475632 +0000 UTC m=+43.832312280" Jul 14 02:02:21.692220 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:21.692167 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-j4k5p" podStartSLOduration=2.87006523 podStartE2EDuration="8.692146908s" totalImagesPullingTime="5.822081678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:02:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:02:21.69116679 +0000 UTC m=+43.846003440" watchObservedRunningTime="2026-07-14 02:02:21.692146908 +0000 UTC m=+43.846983558" Jul 14 02:02:21.720741 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:21.720678 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-pb8mx" podStartSLOduration=1.925977614 podStartE2EDuration="6.720656967s" totalImagesPullingTime="4.794679353s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:02:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:02:21.720593881 +0000 UTC m=+43.875430531" watchObservedRunningTime="2026-07-14 02:02:21.720656967 +0000 UTC m=+43.875493607" Jul 14 02:02:21.720898 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:21.720817 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-bkdjp" podStartSLOduration=2.011151105 podStartE2EDuration="6.720806112s" totalImagesPullingTime="4.709655007s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:02:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:02:21.705975377 +0000 UTC m=+43.860812028" watchObservedRunningTime="2026-07-14 02:02:21.720806112 +0000 UTC m=+43.875642761" Jul 14 02:02:22.023580 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:22.023490 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:22.123600 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:22.123550 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:22.290970 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:22.290888 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:22.348990 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:22.348957 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:22.706968 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:22.706932 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:22.807418 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:22.807384 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:22.944978 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:22.944948 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:23.522196 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:23.522164 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-registry-tls\") pod \"image-registry-6497d48c68-d9xm4\" (UID: \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\") " pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" Jul 14 02:02:23.522402 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:23.522208 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/998e452c-dc1f-4e3d-862e-7365d5a216e7-metrics-certs\") pod \"router-default-8f44474b6-jrxf5\" (UID: \"998e452c-dc1f-4e3d-862e-7365d5a216e7\") " pod="openshift-ingress/router-default-8f44474b6-jrxf5" Jul 14 02:02:23.522402 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:23.522255 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/998e452c-dc1f-4e3d-862e-7365d5a216e7-service-ca-bundle\") pod \"router-default-8f44474b6-jrxf5\" (UID: \"998e452c-dc1f-4e3d-862e-7365d5a216e7\") " pod="openshift-ingress/router-default-8f44474b6-jrxf5" Jul 14 02:02:23.522402 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:23.522337 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 02:02:23.522402 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:23.522357 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6497d48c68-d9xm4: secret "image-registry-tls" not found Jul 14 02:02:23.522402 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:23.522381 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 14 02:02:23.522402 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:23.522397 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/998e452c-dc1f-4e3d-862e-7365d5a216e7-service-ca-bundle podName:998e452c-dc1f-4e3d-862e-7365d5a216e7 nodeName:}" failed. No retries permitted until 2026-07-14 02:02:31.522383342 +0000 UTC m=+53.677219968 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/998e452c-dc1f-4e3d-862e-7365d5a216e7-service-ca-bundle") pod "router-default-8f44474b6-jrxf5" (UID: "998e452c-dc1f-4e3d-862e-7365d5a216e7") : configmap references non-existent config key: service-ca.crt Jul 14 02:02:23.522618 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:23.522412 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-registry-tls podName:b2ab7b4e-fa36-44f3-86e6-151a58a5631b nodeName:}" failed. No retries permitted until 2026-07-14 02:02:31.522405949 +0000 UTC m=+53.677242575 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-registry-tls") pod "image-registry-6497d48c68-d9xm4" (UID: "b2ab7b4e-fa36-44f3-86e6-151a58a5631b") : secret "image-registry-tls" not found Jul 14 02:02:23.522618 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:23.522440 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/998e452c-dc1f-4e3d-862e-7365d5a216e7-metrics-certs podName:998e452c-dc1f-4e3d-862e-7365d5a216e7 nodeName:}" failed. No retries permitted until 2026-07-14 02:02:31.52242694 +0000 UTC m=+53.677263565 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/998e452c-dc1f-4e3d-862e-7365d5a216e7-metrics-certs") pod "router-default-8f44474b6-jrxf5" (UID: "998e452c-dc1f-4e3d-862e-7365d5a216e7") : secret "router-metrics-certs-default" not found Jul 14 02:02:23.623648 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:23.623616 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/958c7626-e6d4-4fdd-aed2-7632ac9f9fb0-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-w4lsz\" (UID: \"958c7626-e6d4-4fdd-aed2-7632ac9f9fb0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-w4lsz" Jul 14 02:02:23.623817 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:23.623770 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 02:02:23.623861 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:23.623832 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/958c7626-e6d4-4fdd-aed2-7632ac9f9fb0-samples-operator-tls podName:958c7626-e6d4-4fdd-aed2-7632ac9f9fb0 nodeName:}" failed. No retries permitted until 2026-07-14 02:02:31.623817526 +0000 UTC m=+53.778654152 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/958c7626-e6d4-4fdd-aed2-7632ac9f9fb0-samples-operator-tls") pod "cluster-samples-operator-65db858894-w4lsz" (UID: "958c7626-e6d4-4fdd-aed2-7632ac9f9fb0") : secret "samples-operator-tls" not found Jul 14 02:02:24.477895 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:24.477870 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-k4w2h_059e3b07-82d1-475b-a7c2-4e9345579238/dns-node-resolver/0.log" Jul 14 02:02:24.893921 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:24.893885 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:25.131009 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:25.130972 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:25.237636 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:25.237550 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:25.327351 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:25.327320 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:25.508962 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:25.508883 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:25.613790 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:25.613757 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:25.802102 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:25.802020 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:25.881893 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:25.881865 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rpl6g_94fa401d-46a7-45c2-bc57-88c112508139/node-ca/0.log" Jul 14 02:02:25.904817 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:25.904785 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:26.004952 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:26.004921 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:26.189346 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:26.189319 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:26.277826 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:26.277798 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-j4k5p_29fa5f87-8e2f-4d38-bc0f-2d6584e8016b/serve-healthcheck-canary/0.log" Jul 14 02:02:27.000378 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:27.000344 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:27.196304 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:27.196259 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:27.581605 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:27.581575 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:27.774768 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:27.774739 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:27.974525 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:27.974496 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:28.571962 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:28.571932 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:28.681891 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:28.681862 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:28.975732 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:28.975701 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:29.173453 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:29.173421 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:29.373459 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:29.373424 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:29.471205 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:29.471170 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9eccb8f-2ee4-4534-a4f0-60a83fc0879b-metrics-tls\") pod \"dns-default-6pg9z\" (UID: \"a9eccb8f-2ee4-4534-a4f0-60a83fc0879b\") " pod="openshift-dns/dns-default-6pg9z" Jul 14 02:02:29.472119 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:29.472094 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:29.474617 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:29.474599 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9eccb8f-2ee4-4534-a4f0-60a83fc0879b-metrics-tls\") pod \"dns-default-6pg9z\" (UID: \"a9eccb8f-2ee4-4534-a4f0-60a83fc0879b\") " pod="openshift-dns/dns-default-6pg9z" Jul 14 02:02:29.532636 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:29.532599 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6pg9z" Jul 14 02:02:29.579882 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:29.579856 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:29.667676 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:29.667593 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6pg9z"] Jul 14 02:02:29.684138 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:29.684109 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6pg9z" event={"ID":"a9eccb8f-2ee4-4534-a4f0-60a83fc0879b","Type":"ContainerStarted","Data":"44ddc0a9d61e81ea84484fafab28a53232cb01d8556383d98514fe74e4754be0"} Jul 14 02:02:30.779402 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:30.779340 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:30.970651 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:30.970622 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:31.171385 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:31.171352 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:31.274900 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:31.274866 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:31.383744 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:31.383712 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:31.573753 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:31.573675 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:31.589019 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:31.588993 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-registry-tls\") pod \"image-registry-6497d48c68-d9xm4\" (UID: \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\") " pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" Jul 14 02:02:31.589186 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:31.589031 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/998e452c-dc1f-4e3d-862e-7365d5a216e7-metrics-certs\") pod \"router-default-8f44474b6-jrxf5\" (UID: \"998e452c-dc1f-4e3d-862e-7365d5a216e7\") " pod="openshift-ingress/router-default-8f44474b6-jrxf5" Jul 14 02:02:31.589287 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:31.589252 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/998e452c-dc1f-4e3d-862e-7365d5a216e7-service-ca-bundle\") pod \"router-default-8f44474b6-jrxf5\" (UID: \"998e452c-dc1f-4e3d-862e-7365d5a216e7\") " pod="openshift-ingress/router-default-8f44474b6-jrxf5" Jul 14 02:02:31.589863 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:31.589842 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/998e452c-dc1f-4e3d-862e-7365d5a216e7-service-ca-bundle\") pod \"router-default-8f44474b6-jrxf5\" (UID: \"998e452c-dc1f-4e3d-862e-7365d5a216e7\") " pod="openshift-ingress/router-default-8f44474b6-jrxf5" Jul 14 02:02:31.591507 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:31.591481 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-registry-tls\") pod \"image-registry-6497d48c68-d9xm4\" (UID: \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\") " pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" Jul 14 02:02:31.591507 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:31.591497 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/998e452c-dc1f-4e3d-862e-7365d5a216e7-metrics-certs\") pod \"router-default-8f44474b6-jrxf5\" (UID: \"998e452c-dc1f-4e3d-862e-7365d5a216e7\") " pod="openshift-ingress/router-default-8f44474b6-jrxf5" Jul 14 02:02:31.687992 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:31.687945 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" Jul 14 02:02:31.689906 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:31.689880 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/958c7626-e6d4-4fdd-aed2-7632ac9f9fb0-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-w4lsz\" (UID: \"958c7626-e6d4-4fdd-aed2-7632ac9f9fb0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-w4lsz" Jul 14 02:02:31.692463 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:31.692438 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/958c7626-e6d4-4fdd-aed2-7632ac9f9fb0-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-w4lsz\" (UID: \"958c7626-e6d4-4fdd-aed2-7632ac9f9fb0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-w4lsz" Jul 14 02:02:31.694317 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:31.694295 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-8f44474b6-jrxf5" Jul 14 02:02:31.843606 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:31.843524 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-w4lsz" Jul 14 02:02:32.299550 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:32.299516 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-w4lsz"] Jul 14 02:02:32.524577 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:32.524505 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-8f44474b6-jrxf5"] Jul 14 02:02:32.527311 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:32.527217 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6497d48c68-d9xm4"] Jul 14 02:02:32.549631 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:02:32.549588 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod998e452c_dc1f_4e3d_862e_7365d5a216e7.slice/crio-9f112890a2d91f22117f2446a6338451a50b67c4171f156c7d2481a4645e8615 WatchSource:0}: Error finding container 9f112890a2d91f22117f2446a6338451a50b67c4171f156c7d2481a4645e8615: Status 404 returned error can't find the container with id 9f112890a2d91f22117f2446a6338451a50b67c4171f156c7d2481a4645e8615 Jul 14 02:02:32.550262 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:02:32.550240 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2ab7b4e_fa36_44f3_86e6_151a58a5631b.slice/crio-0b85ebd5d1986b77585304c75e8e65777aefc1a3e93d1c8254935b64d940eb58 WatchSource:0}: Error finding container 0b85ebd5d1986b77585304c75e8e65777aefc1a3e93d1c8254935b64d940eb58: Status 404 returned error can't find the container with id 0b85ebd5d1986b77585304c75e8e65777aefc1a3e93d1c8254935b64d940eb58 Jul 14 02:02:32.575261 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:32.575234 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:32.697503 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:32.696445 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-8f44474b6-jrxf5" event={"ID":"998e452c-dc1f-4e3d-862e-7365d5a216e7","Type":"ContainerStarted","Data":"51231af63ce29bf7b59b77479ab2b10bdabea025b6a0905eaef1b46e580cc78a"} Jul 14 02:02:32.697503 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:32.696573 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-8f44474b6-jrxf5" event={"ID":"998e452c-dc1f-4e3d-862e-7365d5a216e7","Type":"ContainerStarted","Data":"9f112890a2d91f22117f2446a6338451a50b67c4171f156c7d2481a4645e8615"} Jul 14 02:02:32.701423 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:32.701389 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" event={"ID":"b2ab7b4e-fa36-44f3-86e6-151a58a5631b","Type":"ContainerStarted","Data":"0b85ebd5d1986b77585304c75e8e65777aefc1a3e93d1c8254935b64d940eb58"} Jul 14 02:02:32.703186 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:32.703105 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-w4lsz" event={"ID":"958c7626-e6d4-4fdd-aed2-7632ac9f9fb0","Type":"ContainerStarted","Data":"c4a5338db3c9c6a9838a72b04fd616964ccee2d87b34d69bfb9d00ec217b7c60"} Jul 14 02:02:32.775833 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:32.775767 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:33.171649 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:33.171606 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:33.335965 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:33.335936 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:33.434088 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:33.433978 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:33.709874 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:33.709781 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6pg9z" event={"ID":"a9eccb8f-2ee4-4534-a4f0-60a83fc0879b","Type":"ContainerStarted","Data":"68bc101fc81a8f6b23ebfd8b9db9f8248530e03990b1e4cb7d0884a7036151fc"} Jul 14 02:02:33.709874 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:33.709836 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6pg9z" event={"ID":"a9eccb8f-2ee4-4534-a4f0-60a83fc0879b","Type":"ContainerStarted","Data":"8cc9f2b3ee2de8ae26b61748974835f8dcec42ce3ff31cc4e8dc20ef82722f62"} Jul 14 02:02:33.710089 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:33.709904 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6pg9z" Jul 14 02:02:33.711570 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:33.711542 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" event={"ID":"b2ab7b4e-fa36-44f3-86e6-151a58a5631b","Type":"ContainerStarted","Data":"5bc2c3019de4f3c9e7e48bd49be473e0ff91d9e75eb8b860612d364192ed01ad"} Jul 14 02:02:33.711695 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:33.711586 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" Jul 14 02:02:33.725543 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:33.725465 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-6pg9z" podStartSLOduration=17.811795327 podStartE2EDuration="20.725448033s" totalImagesPullingTime="2.913652706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:02:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:02:33.724916478 +0000 UTC m=+55.879753126" watchObservedRunningTime="2026-07-14 02:02:33.725448033 +0000 UTC m=+55.880284681" Jul 14 02:02:33.741318 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:33.741258 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" podStartSLOduration=18.741240878 podStartE2EDuration="18.741240878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:02:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:02:33.740782793 +0000 UTC m=+55.895619466" watchObservedRunningTime="2026-07-14 02:02:33.741240878 +0000 UTC m=+55.896077526" Jul 14 02:02:33.756865 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:33.756810 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-8f44474b6-jrxf5" podStartSLOduration=18.756800381 podStartE2EDuration="18.756800381s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:02:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:02:33.755775526 +0000 UTC m=+55.910612173" watchObservedRunningTime="2026-07-14 02:02:33.756800381 +0000 UTC m=+55.911637044" Jul 14 02:02:33.970129 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:33.970046 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:34.362627 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:34.362604 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:34.618895 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:34.618868 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:34.694762 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:34.694726 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-8f44474b6-jrxf5" Jul 14 02:02:34.697134 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:34.697112 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-8f44474b6-jrxf5" Jul 14 02:02:34.717019 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:34.716976 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-w4lsz" event={"ID":"958c7626-e6d4-4fdd-aed2-7632ac9f9fb0","Type":"ContainerStarted","Data":"23634008b4f140779fa92a9535a6bcee78b038307b4a4264fec9ac2b588ef2c6"} Jul 14 02:02:34.717348 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:34.717031 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-w4lsz" event={"ID":"958c7626-e6d4-4fdd-aed2-7632ac9f9fb0","Type":"ContainerStarted","Data":"c2e25c7fb82f702c6c2f6ce15a2532d01de41440866cbd8ea1d20791e9fbb34a"} Jul 14 02:02:34.717426 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:34.717399 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-8f44474b6-jrxf5" Jul 14 02:02:34.718242 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:34.718223 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-8f44474b6-jrxf5" Jul 14 02:02:34.734443 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:34.734397 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-w4lsz" podStartSLOduration=17.733308681 podStartE2EDuration="19.734385373s" totalImagesPullingTime="2.001076692s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:02:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:02:34.734297845 +0000 UTC m=+56.889134493" watchObservedRunningTime="2026-07-14 02:02:34.734385373 +0000 UTC m=+56.889222021" Jul 14 02:02:34.773893 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:34.773860 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:34.969374 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:34.969258 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:35.377173 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:35.377139 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:35.581616 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:35.581584 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:39.617917 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:39.617885 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bnpgh" Jul 14 02:02:40.902827 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:40.902783 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6497d48c68-d9xm4"] Jul 14 02:02:40.968514 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:40.968483 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7b4cb8c7f6-kn9xv"] Jul 14 02:02:40.987391 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:40.987363 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b4cb8c7f6-kn9xv"] Jul 14 02:02:40.987543 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:40.987480 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b4cb8c7f6-kn9xv" Jul 14 02:02:40.990148 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:40.990111 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:41.018750 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.018720 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-4v7x2"] Jul 14 02:02:41.020562 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.020548 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4v7x2" Jul 14 02:02:41.022587 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.022563 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 14 02:02:41.022711 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.022637 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xfgbl\"" Jul 14 02:02:41.023875 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.023854 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 14 02:02:41.038555 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.038522 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4v7x2"] Jul 14 02:02:41.068648 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.068613 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/193772c5-ef88-4b4d-afbe-c14e65e8e81f-ca-trust-extracted\") pod \"image-registry-7b4cb8c7f6-kn9xv\" (UID: \"193772c5-ef88-4b4d-afbe-c14e65e8e81f\") " pod="openshift-image-registry/image-registry-7b4cb8c7f6-kn9xv" Jul 14 02:02:41.068648 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.068648 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9162ca72-95c1-49d7-9812-c5705faa7ee8-data-volume\") pod \"insights-runtime-extractor-4v7x2\" (UID: \"9162ca72-95c1-49d7-9812-c5705faa7ee8\") " pod="openshift-insights/insights-runtime-extractor-4v7x2" Jul 14 02:02:41.068871 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.068674 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9162ca72-95c1-49d7-9812-c5705faa7ee8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4v7x2\" (UID: \"9162ca72-95c1-49d7-9812-c5705faa7ee8\") " pod="openshift-insights/insights-runtime-extractor-4v7x2" Jul 14 02:02:41.068871 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.068698 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/193772c5-ef88-4b4d-afbe-c14e65e8e81f-registry-tls\") pod \"image-registry-7b4cb8c7f6-kn9xv\" (UID: \"193772c5-ef88-4b4d-afbe-c14e65e8e81f\") " pod="openshift-image-registry/image-registry-7b4cb8c7f6-kn9xv" Jul 14 02:02:41.068871 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.068720 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bj9l\" (UniqueName: \"kubernetes.io/projected/193772c5-ef88-4b4d-afbe-c14e65e8e81f-kube-api-access-9bj9l\") pod \"image-registry-7b4cb8c7f6-kn9xv\" (UID: \"193772c5-ef88-4b4d-afbe-c14e65e8e81f\") " pod="openshift-image-registry/image-registry-7b4cb8c7f6-kn9xv" Jul 14 02:02:41.068871 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.068741 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/193772c5-ef88-4b4d-afbe-c14e65e8e81f-trusted-ca\") pod \"image-registry-7b4cb8c7f6-kn9xv\" (UID: \"193772c5-ef88-4b4d-afbe-c14e65e8e81f\") " pod="openshift-image-registry/image-registry-7b4cb8c7f6-kn9xv" Jul 14 02:02:41.068871 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.068831 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/193772c5-ef88-4b4d-afbe-c14e65e8e81f-image-registry-private-configuration\") pod \"image-registry-7b4cb8c7f6-kn9xv\" (UID: \"193772c5-ef88-4b4d-afbe-c14e65e8e81f\") " pod="openshift-image-registry/image-registry-7b4cb8c7f6-kn9xv" Jul 14 02:02:41.068871 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.068865 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/193772c5-ef88-4b4d-afbe-c14e65e8e81f-registry-certificates\") pod \"image-registry-7b4cb8c7f6-kn9xv\" (UID: \"193772c5-ef88-4b4d-afbe-c14e65e8e81f\") " pod="openshift-image-registry/image-registry-7b4cb8c7f6-kn9xv" Jul 14 02:02:41.069090 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.068886 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/193772c5-ef88-4b4d-afbe-c14e65e8e81f-installation-pull-secrets\") pod \"image-registry-7b4cb8c7f6-kn9xv\" (UID: \"193772c5-ef88-4b4d-afbe-c14e65e8e81f\") " pod="openshift-image-registry/image-registry-7b4cb8c7f6-kn9xv" Jul 14 02:02:41.069090 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.068951 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkmv6\" (UniqueName: \"kubernetes.io/projected/9162ca72-95c1-49d7-9812-c5705faa7ee8-kube-api-access-bkmv6\") pod \"insights-runtime-extractor-4v7x2\" (UID: \"9162ca72-95c1-49d7-9812-c5705faa7ee8\") " pod="openshift-insights/insights-runtime-extractor-4v7x2" Jul 14 02:02:41.069090 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.068974 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/193772c5-ef88-4b4d-afbe-c14e65e8e81f-bound-sa-token\") pod \"image-registry-7b4cb8c7f6-kn9xv\" (UID: \"193772c5-ef88-4b4d-afbe-c14e65e8e81f\") " pod="openshift-image-registry/image-registry-7b4cb8c7f6-kn9xv" Jul 14 02:02:41.069090 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.069038 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9162ca72-95c1-49d7-9812-c5705faa7ee8-crio-socket\") pod \"insights-runtime-extractor-4v7x2\" (UID: \"9162ca72-95c1-49d7-9812-c5705faa7ee8\") " pod="openshift-insights/insights-runtime-extractor-4v7x2" Jul 14 02:02:41.069090 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.069055 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9162ca72-95c1-49d7-9812-c5705faa7ee8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4v7x2\" (UID: \"9162ca72-95c1-49d7-9812-c5705faa7ee8\") " pod="openshift-insights/insights-runtime-extractor-4v7x2" Jul 14 02:02:41.088335 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.088303 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:41.169500 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.169414 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/193772c5-ef88-4b4d-afbe-c14e65e8e81f-image-registry-private-configuration\") pod \"image-registry-7b4cb8c7f6-kn9xv\" (UID: \"193772c5-ef88-4b4d-afbe-c14e65e8e81f\") " pod="openshift-image-registry/image-registry-7b4cb8c7f6-kn9xv" Jul 14 02:02:41.169500 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.169447 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/193772c5-ef88-4b4d-afbe-c14e65e8e81f-registry-certificates\") pod \"image-registry-7b4cb8c7f6-kn9xv\" (UID: \"193772c5-ef88-4b4d-afbe-c14e65e8e81f\") " pod="openshift-image-registry/image-registry-7b4cb8c7f6-kn9xv" Jul 14 02:02:41.169500 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.169467 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/193772c5-ef88-4b4d-afbe-c14e65e8e81f-installation-pull-secrets\") pod \"image-registry-7b4cb8c7f6-kn9xv\" (UID: \"193772c5-ef88-4b4d-afbe-c14e65e8e81f\") " pod="openshift-image-registry/image-registry-7b4cb8c7f6-kn9xv" Jul 14 02:02:41.169741 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.169517 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkmv6\" (UniqueName: \"kubernetes.io/projected/9162ca72-95c1-49d7-9812-c5705faa7ee8-kube-api-access-bkmv6\") pod \"insights-runtime-extractor-4v7x2\" (UID: \"9162ca72-95c1-49d7-9812-c5705faa7ee8\") " pod="openshift-insights/insights-runtime-extractor-4v7x2" Jul 14 02:02:41.169741 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.169544 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/193772c5-ef88-4b4d-afbe-c14e65e8e81f-bound-sa-token\") pod \"image-registry-7b4cb8c7f6-kn9xv\" (UID: \"193772c5-ef88-4b4d-afbe-c14e65e8e81f\") " pod="openshift-image-registry/image-registry-7b4cb8c7f6-kn9xv" Jul 14 02:02:41.169741 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.169611 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9162ca72-95c1-49d7-9812-c5705faa7ee8-crio-socket\") pod \"insights-runtime-extractor-4v7x2\" (UID: \"9162ca72-95c1-49d7-9812-c5705faa7ee8\") " pod="openshift-insights/insights-runtime-extractor-4v7x2" Jul 14 02:02:41.169741 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.169636 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9162ca72-95c1-49d7-9812-c5705faa7ee8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4v7x2\" (UID: \"9162ca72-95c1-49d7-9812-c5705faa7ee8\") " pod="openshift-insights/insights-runtime-extractor-4v7x2" Jul 14 02:02:41.169741 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.169667 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/193772c5-ef88-4b4d-afbe-c14e65e8e81f-ca-trust-extracted\") pod \"image-registry-7b4cb8c7f6-kn9xv\" (UID: \"193772c5-ef88-4b4d-afbe-c14e65e8e81f\") " pod="openshift-image-registry/image-registry-7b4cb8c7f6-kn9xv" Jul 14 02:02:41.169741 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.169725 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9162ca72-95c1-49d7-9812-c5705faa7ee8-data-volume\") pod \"insights-runtime-extractor-4v7x2\" (UID: \"9162ca72-95c1-49d7-9812-c5705faa7ee8\") " pod="openshift-insights/insights-runtime-extractor-4v7x2" Jul 14 02:02:41.171498 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.169772 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9162ca72-95c1-49d7-9812-c5705faa7ee8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4v7x2\" (UID: \"9162ca72-95c1-49d7-9812-c5705faa7ee8\") " pod="openshift-insights/insights-runtime-extractor-4v7x2" Jul 14 02:02:41.171498 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.170419 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/193772c5-ef88-4b4d-afbe-c14e65e8e81f-registry-tls\") pod \"image-registry-7b4cb8c7f6-kn9xv\" (UID: \"193772c5-ef88-4b4d-afbe-c14e65e8e81f\") " pod="openshift-image-registry/image-registry-7b4cb8c7f6-kn9xv" Jul 14 02:02:41.171498 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.170461 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bj9l\" (UniqueName: \"kubernetes.io/projected/193772c5-ef88-4b4d-afbe-c14e65e8e81f-kube-api-access-9bj9l\") pod \"image-registry-7b4cb8c7f6-kn9xv\" (UID: \"193772c5-ef88-4b4d-afbe-c14e65e8e81f\") " pod="openshift-image-registry/image-registry-7b4cb8c7f6-kn9xv" Jul 14 02:02:41.171498 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.170498 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/193772c5-ef88-4b4d-afbe-c14e65e8e81f-trusted-ca\") pod \"image-registry-7b4cb8c7f6-kn9xv\" (UID: \"193772c5-ef88-4b4d-afbe-c14e65e8e81f\") " pod="openshift-image-registry/image-registry-7b4cb8c7f6-kn9xv" Jul 14 02:02:41.171498 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.170564 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/193772c5-ef88-4b4d-afbe-c14e65e8e81f-ca-trust-extracted\") pod \"image-registry-7b4cb8c7f6-kn9xv\" (UID: \"193772c5-ef88-4b4d-afbe-c14e65e8e81f\") " pod="openshift-image-registry/image-registry-7b4cb8c7f6-kn9xv" Jul 14 02:02:41.171835 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.171688 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/193772c5-ef88-4b4d-afbe-c14e65e8e81f-registry-certificates\") pod \"image-registry-7b4cb8c7f6-kn9xv\" (UID: \"193772c5-ef88-4b4d-afbe-c14e65e8e81f\") " pod="openshift-image-registry/image-registry-7b4cb8c7f6-kn9xv" Jul 14 02:02:41.171835 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.171655 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9162ca72-95c1-49d7-9812-c5705faa7ee8-data-volume\") pod \"insights-runtime-extractor-4v7x2\" (UID: \"9162ca72-95c1-49d7-9812-c5705faa7ee8\") " pod="openshift-insights/insights-runtime-extractor-4v7x2" Jul 14 02:02:41.171948 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.171913 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9162ca72-95c1-49d7-9812-c5705faa7ee8-crio-socket\") pod \"insights-runtime-extractor-4v7x2\" (UID: \"9162ca72-95c1-49d7-9812-c5705faa7ee8\") " pod="openshift-insights/insights-runtime-extractor-4v7x2" Jul 14 02:02:41.173026 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.172107 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9162ca72-95c1-49d7-9812-c5705faa7ee8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4v7x2\" (UID: \"9162ca72-95c1-49d7-9812-c5705faa7ee8\") " pod="openshift-insights/insights-runtime-extractor-4v7x2" Jul 14 02:02:41.173236 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.173200 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9162ca72-95c1-49d7-9812-c5705faa7ee8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4v7x2\" (UID: \"9162ca72-95c1-49d7-9812-c5705faa7ee8\") " pod="openshift-insights/insights-runtime-extractor-4v7x2" Jul 14 02:02:41.176000 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.174468 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/193772c5-ef88-4b4d-afbe-c14e65e8e81f-image-registry-private-configuration\") pod \"image-registry-7b4cb8c7f6-kn9xv\" (UID: \"193772c5-ef88-4b4d-afbe-c14e65e8e81f\") " pod="openshift-image-registry/image-registry-7b4cb8c7f6-kn9xv" Jul 14 02:02:41.176599 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.176579 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/193772c5-ef88-4b4d-afbe-c14e65e8e81f-registry-tls\") pod \"image-registry-7b4cb8c7f6-kn9xv\" (UID: \"193772c5-ef88-4b4d-afbe-c14e65e8e81f\") " pod="openshift-image-registry/image-registry-7b4cb8c7f6-kn9xv" Jul 14 02:02:41.176812 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.176794 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/193772c5-ef88-4b4d-afbe-c14e65e8e81f-installation-pull-secrets\") pod \"image-registry-7b4cb8c7f6-kn9xv\" (UID: \"193772c5-ef88-4b4d-afbe-c14e65e8e81f\") " pod="openshift-image-registry/image-registry-7b4cb8c7f6-kn9xv" Jul 14 02:02:41.176968 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.176946 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/193772c5-ef88-4b4d-afbe-c14e65e8e81f-trusted-ca\") pod \"image-registry-7b4cb8c7f6-kn9xv\" (UID: \"193772c5-ef88-4b4d-afbe-c14e65e8e81f\") " pod="openshift-image-registry/image-registry-7b4cb8c7f6-kn9xv" Jul 14 02:02:41.181759 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.181733 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkmv6\" (UniqueName: \"kubernetes.io/projected/9162ca72-95c1-49d7-9812-c5705faa7ee8-kube-api-access-bkmv6\") pod \"insights-runtime-extractor-4v7x2\" (UID: \"9162ca72-95c1-49d7-9812-c5705faa7ee8\") " pod="openshift-insights/insights-runtime-extractor-4v7x2" Jul 14 02:02:41.181966 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.181948 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/193772c5-ef88-4b4d-afbe-c14e65e8e81f-bound-sa-token\") pod \"image-registry-7b4cb8c7f6-kn9xv\" (UID: \"193772c5-ef88-4b4d-afbe-c14e65e8e81f\") " pod="openshift-image-registry/image-registry-7b4cb8c7f6-kn9xv" Jul 14 02:02:41.182084 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.182067 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bj9l\" (UniqueName: \"kubernetes.io/projected/193772c5-ef88-4b4d-afbe-c14e65e8e81f-kube-api-access-9bj9l\") pod \"image-registry-7b4cb8c7f6-kn9xv\" (UID: \"193772c5-ef88-4b4d-afbe-c14e65e8e81f\") " pod="openshift-image-registry/image-registry-7b4cb8c7f6-kn9xv" Jul 14 02:02:41.189383 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.189363 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:41.284227 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.284194 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:41.296667 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.296646 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b4cb8c7f6-kn9xv" Jul 14 02:02:41.329910 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.329873 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4v7x2" Jul 14 02:02:41.355839 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.355800 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:41.430927 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.430701 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b4cb8c7f6-kn9xv"] Jul 14 02:02:41.433012 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:02:41.432979 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod193772c5_ef88_4b4d_afbe_c14e65e8e81f.slice/crio-9d4c728c941f219c1a65eb1ff51d282ad2b97b82b607998a73c9f89b23f7ac1f WatchSource:0}: Error finding container 9d4c728c941f219c1a65eb1ff51d282ad2b97b82b607998a73c9f89b23f7ac1f: Status 404 returned error can't find the container with id 9d4c728c941f219c1a65eb1ff51d282ad2b97b82b607998a73c9f89b23f7ac1f Jul 14 02:02:41.463612 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.463581 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4v7x2"] Jul 14 02:02:41.466571 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:02:41.466537 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9162ca72_95c1_49d7_9812_c5705faa7ee8.slice/crio-7d8e6a371db25b1d7fc60672d738e740b46ce95778602a0761bf042e46a07ee3 WatchSource:0}: Error finding container 7d8e6a371db25b1d7fc60672d738e740b46ce95778602a0761bf042e46a07ee3: Status 404 returned error can't find the container with id 7d8e6a371db25b1d7fc60672d738e740b46ce95778602a0761bf042e46a07ee3 Jul 14 02:02:41.741235 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.741132 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4v7x2" event={"ID":"9162ca72-95c1-49d7-9812-c5705faa7ee8","Type":"ContainerStarted","Data":"7d8e6a371db25b1d7fc60672d738e740b46ce95778602a0761bf042e46a07ee3"} Jul 14 02:02:41.742745 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.742715 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b4cb8c7f6-kn9xv" event={"ID":"193772c5-ef88-4b4d-afbe-c14e65e8e81f","Type":"ContainerStarted","Data":"47d4e65a90a1d946b16762eed6179aee3182a209444221395e123ae7e7e8dcca"} Jul 14 02:02:41.742934 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.742761 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b4cb8c7f6-kn9xv" event={"ID":"193772c5-ef88-4b4d-afbe-c14e65e8e81f","Type":"ContainerStarted","Data":"9d4c728c941f219c1a65eb1ff51d282ad2b97b82b607998a73c9f89b23f7ac1f"} Jul 14 02:02:41.742934 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.742832 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7b4cb8c7f6-kn9xv" Jul 14 02:02:41.761255 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:41.761202 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7b4cb8c7f6-kn9xv" podStartSLOduration=1.7611841419999998 podStartE2EDuration="1.761184142s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:02:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:02:41.760006922 +0000 UTC m=+63.914843569" watchObservedRunningTime="2026-07-14 02:02:41.761184142 +0000 UTC m=+63.916020790" Jul 14 02:02:43.414586 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:43.414553 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:43.466881 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:43.466839 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:43.719912 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:43.719838 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6pg9z" Jul 14 02:02:44.000252 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:44.000167 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s985w\" (UniqueName: \"kubernetes.io/projected/f018324f-f63f-41e0-9a3f-d6fed83b4f1c-kube-api-access-s985w\") pod \"network-check-target-c6wmg\" (UID: \"f018324f-f63f-41e0-9a3f-d6fed83b4f1c\") " pod="openshift-network-diagnostics/network-check-target-c6wmg" Jul 14 02:02:44.002797 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:44.002773 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s985w\" (UniqueName: \"kubernetes.io/projected/f018324f-f63f-41e0-9a3f-d6fed83b4f1c-kube-api-access-s985w\") pod \"network-check-target-c6wmg\" (UID: \"f018324f-f63f-41e0-9a3f-d6fed83b4f1c\") " pod="openshift-network-diagnostics/network-check-target-c6wmg" Jul 14 02:02:44.076740 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:44.076705 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mrlfg\"" Jul 14 02:02:44.085362 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:44.085339 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6wmg" Jul 14 02:02:44.101231 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:44.101197 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45d07b54-1b6f-4f51-bf7c-cc8d287e5789-original-pull-secret\") pod \"global-pull-secret-syncer-wmd5q\" (UID: \"45d07b54-1b6f-4f51-bf7c-cc8d287e5789\") " pod="kube-system/global-pull-secret-syncer-wmd5q" Jul 14 02:02:44.101375 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:44.101243 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d6a986a-c066-4ce3-a49c-4bab5be2f075-metrics-certs\") pod \"network-metrics-daemon-t9bnr\" (UID: \"8d6a986a-c066-4ce3-a49c-4bab5be2f075\") " pod="openshift-multus/network-metrics-daemon-t9bnr" Jul 14 02:02:44.103347 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:44.103329 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 02:02:44.103432 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:44.103342 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 02:02:44.114106 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:44.114081 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d6a986a-c066-4ce3-a49c-4bab5be2f075-metrics-certs\") pod \"network-metrics-daemon-t9bnr\" (UID: \"8d6a986a-c066-4ce3-a49c-4bab5be2f075\") " pod="openshift-multus/network-metrics-daemon-t9bnr" Jul 14 02:02:44.114214 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:44.114178 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45d07b54-1b6f-4f51-bf7c-cc8d287e5789-original-pull-secret\") pod \"global-pull-secret-syncer-wmd5q\" (UID: \"45d07b54-1b6f-4f51-bf7c-cc8d287e5789\") " pod="kube-system/global-pull-secret-syncer-wmd5q" Jul 14 02:02:44.368493 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:44.368453 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmd5q" Jul 14 02:02:44.381568 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:44.381543 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-klxhd\"" Jul 14 02:02:44.390026 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:44.390003 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9bnr" Jul 14 02:02:45.156229 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:45.156202 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-c6wmg"] Jul 14 02:02:45.160447 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:02:45.160414 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf018324f_f63f_41e0_9a3f_d6fed83b4f1c.slice/crio-470e1d83cf9d6431bc6e4a2651008dca9d0c5530ea9b8c8a00dad4f573471611 WatchSource:0}: Error finding container 470e1d83cf9d6431bc6e4a2651008dca9d0c5530ea9b8c8a00dad4f573471611: Status 404 returned error can't find the container with id 470e1d83cf9d6431bc6e4a2651008dca9d0c5530ea9b8c8a00dad4f573471611 Jul 14 02:02:45.177322 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:45.177296 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-t9bnr"] Jul 14 02:02:45.180092 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:02:45.180010 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d6a986a_c066_4ce3_a49c_4bab5be2f075.slice/crio-842564ce7af67e595ae94cb9e77d73a3fc2ac70b225c3883329d031b81f5bda9 WatchSource:0}: Error finding container 842564ce7af67e595ae94cb9e77d73a3fc2ac70b225c3883329d031b81f5bda9: Status 404 returned error can't find the container with id 842564ce7af67e595ae94cb9e77d73a3fc2ac70b225c3883329d031b81f5bda9 Jul 14 02:02:45.199155 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:45.199124 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wmd5q"] Jul 14 02:02:45.201766 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:02:45.201741 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45d07b54_1b6f_4f51_bf7c_cc8d287e5789.slice/crio-68aa98fa654eae8e3deeba8a1db549d57e5d7c614f389c0af6a7be2a5ecdef2f WatchSource:0}: Error finding container 68aa98fa654eae8e3deeba8a1db549d57e5d7c614f389c0af6a7be2a5ecdef2f: Status 404 returned error can't find the container with id 68aa98fa654eae8e3deeba8a1db549d57e5d7c614f389c0af6a7be2a5ecdef2f Jul 14 02:02:45.759306 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:45.759248 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t9bnr" event={"ID":"8d6a986a-c066-4ce3-a49c-4bab5be2f075","Type":"ContainerStarted","Data":"842564ce7af67e595ae94cb9e77d73a3fc2ac70b225c3883329d031b81f5bda9"} Jul 14 02:02:45.761104 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:45.761073 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-c6wmg" event={"ID":"f018324f-f63f-41e0-9a3f-d6fed83b4f1c","Type":"ContainerStarted","Data":"774c6ff4a912b64968b737cec251067b3ab16105b3f46a7413098503c385bdd9"} Jul 14 02:02:45.761217 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:45.761124 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-c6wmg" event={"ID":"f018324f-f63f-41e0-9a3f-d6fed83b4f1c","Type":"ContainerStarted","Data":"470e1d83cf9d6431bc6e4a2651008dca9d0c5530ea9b8c8a00dad4f573471611"} Jul 14 02:02:45.761217 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:45.761191 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-c6wmg" Jul 14 02:02:45.762750 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:45.762718 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wmd5q" event={"ID":"45d07b54-1b6f-4f51-bf7c-cc8d287e5789","Type":"ContainerStarted","Data":"68aa98fa654eae8e3deeba8a1db549d57e5d7c614f389c0af6a7be2a5ecdef2f"} Jul 14 02:02:45.764439 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:45.764414 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4v7x2" event={"ID":"9162ca72-95c1-49d7-9812-c5705faa7ee8","Type":"ContainerStarted","Data":"e2052bdab09d237b5add53be2c12230b19f8ca024bca352bc2f93661013084db"} Jul 14 02:02:45.776991 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:45.776940 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-c6wmg" podStartSLOduration=66.776922981 podStartE2EDuration="1m6.776922981s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:01:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:02:45.775404925 +0000 UTC m=+67.930241577" watchObservedRunningTime="2026-07-14 02:02:45.776922981 +0000 UTC m=+67.931759608" Jul 14 02:02:46.226416 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:46.226384 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:46.327259 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:46.327221 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:46.771109 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:46.771069 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4v7x2" event={"ID":"9162ca72-95c1-49d7-9812-c5705faa7ee8","Type":"ContainerStarted","Data":"e4120a6c250e82a2a7e2cf8fb41d45d44422c49a55a3a358dbb572e5911bb7d8"} Jul 14 02:02:46.773419 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:46.773390 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t9bnr" event={"ID":"8d6a986a-c066-4ce3-a49c-4bab5be2f075","Type":"ContainerStarted","Data":"5004bc960878b6037cd9cd39c207fde2fd63bce46c2ca9bfb96860185c160af8"} Jul 14 02:02:46.773550 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:46.773450 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t9bnr" event={"ID":"8d6a986a-c066-4ce3-a49c-4bab5be2f075","Type":"ContainerStarted","Data":"a1a9f6aa78f07ab99033b15fe74e7994155e01c9d2447322992dc21bb8ac4458"} Jul 14 02:02:46.789362 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:46.789309 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-t9bnr" podStartSLOduration=66.823563513 podStartE2EDuration="1m7.789262984s" totalImagesPullingTime="965.699471ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:01:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:02:46.788171151 +0000 UTC m=+68.943007801" watchObservedRunningTime="2026-07-14 02:02:46.789262984 +0000 UTC m=+68.944099634" Jul 14 02:02:46.907677 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:46.907637 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:46.962682 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:46.962645 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:48.330883 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:48.330852 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:48.417588 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:48.417548 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:48.479493 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:48.479465 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:50.471127 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:50.471092 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:50.582679 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:50.582642 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:50.790728 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:50.790625 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wmd5q" event={"ID":"45d07b54-1b6f-4f51-bf7c-cc8d287e5789","Type":"ContainerStarted","Data":"b8f00fef750b2d101c3b0109d20dc9bfd4e5a5200677b6139a7e0a2f019b9824"} Jul 14 02:02:50.792710 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:50.792685 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4v7x2" event={"ID":"9162ca72-95c1-49d7-9812-c5705faa7ee8","Type":"ContainerStarted","Data":"69b2c1b3d16bda2320b630b5bd9c5db599a1f95413a536c26408bb4f9cefbf31"} Jul 14 02:02:50.805359 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:50.805318 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-wmd5q" podStartSLOduration=66.496647945 podStartE2EDuration="1m11.805303934s" totalImagesPullingTime="5.308655989s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 02:01:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:02:50.803971369 +0000 UTC m=+72.958808033" watchObservedRunningTime="2026-07-14 02:02:50.805303934 +0000 UTC m=+72.960140560" Jul 14 02:02:50.819565 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:50.819516 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-4v7x2" podStartSLOduration=1.933741218 podStartE2EDuration="10.81949996s" totalImagesPullingTime="8.885758742s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 02:02:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:02:50.819094569 +0000 UTC m=+72.973931220" watchObservedRunningTime="2026-07-14 02:02:50.81949996 +0000 UTC m=+72.974336612" Jul 14 02:02:50.909190 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:50.909162 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" Jul 14 02:02:51.066084 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:51.065997 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:51.167030 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:51.166991 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:51.225573 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:51.225542 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fkbzn"] Jul 14 02:02:51.231145 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:51.231122 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:51.242429 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:51.242401 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fkbzn"] Jul 14 02:02:51.242558 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:51.242510 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fkbzn" Jul 14 02:02:51.244458 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:51.244437 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 14 02:02:51.244554 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:51.244528 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-jr2ff\"" Jul 14 02:02:51.332332 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:51.332244 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:51.362639 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:51.362608 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/bf9b9caa-ed64-44ce-b36f-4cab77cf304d-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-fkbzn\" (UID: \"bf9b9caa-ed64-44ce-b36f-4cab77cf304d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fkbzn" Jul 14 02:02:51.464022 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:51.463979 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/bf9b9caa-ed64-44ce-b36f-4cab77cf304d-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-fkbzn\" (UID: \"bf9b9caa-ed64-44ce-b36f-4cab77cf304d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fkbzn" Jul 14 02:02:51.466444 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:51.466421 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/bf9b9caa-ed64-44ce-b36f-4cab77cf304d-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-fkbzn\" (UID: \"bf9b9caa-ed64-44ce-b36f-4cab77cf304d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fkbzn" Jul 14 02:02:51.553007 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:51.552965 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fkbzn" Jul 14 02:02:51.702641 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:51.702611 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fkbzn"] Jul 14 02:02:51.706553 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:02:51.706527 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf9b9caa_ed64_44ce_b36f_4cab77cf304d.slice/crio-01f3fccbe54009617aab3d2eedabd37c16cd2398417793aecd19ac62125e1718 WatchSource:0}: Error finding container 01f3fccbe54009617aab3d2eedabd37c16cd2398417793aecd19ac62125e1718: Status 404 returned error can't find the container with id 01f3fccbe54009617aab3d2eedabd37c16cd2398417793aecd19ac62125e1718 Jul 14 02:02:51.797171 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:51.797133 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fkbzn" event={"ID":"bf9b9caa-ed64-44ce-b36f-4cab77cf304d","Type":"ContainerStarted","Data":"01f3fccbe54009617aab3d2eedabd37c16cd2398417793aecd19ac62125e1718"} Jul 14 02:02:52.387928 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:52.387895 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:52.443944 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:52.443909 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:53.088533 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:53.088502 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:53.190700 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:53.190663 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:53.292197 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:53.292166 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:53.806431 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:53.806351 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fkbzn" event={"ID":"bf9b9caa-ed64-44ce-b36f-4cab77cf304d","Type":"ContainerStarted","Data":"ad399b42e9a43c3b8f608a2c25009e942ad4c6998b718b0b22f9fdc5a6f1390c"} Jul 14 02:02:53.806579 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:53.806478 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fkbzn" Jul 14 02:02:53.811150 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:53.811125 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fkbzn" Jul 14 02:02:53.823698 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:53.823655 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fkbzn" podStartSLOduration=1.079429522 podStartE2EDuration="2.823640068s" totalImagesPullingTime="1.744210546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:02:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:02:53.823144347 +0000 UTC m=+75.977981013" watchObservedRunningTime="2026-07-14 02:02:53.823640068 +0000 UTC m=+75.978476713" Jul 14 02:02:54.242402 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:54.242374 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:54.319543 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:54.319515 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:54.377374 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:54.377338 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:55.504008 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:55.503976 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:55.559015 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:55.558979 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:56.254145 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:56.254115 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:57.712081 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:57.712054 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:57.761703 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:57.761671 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:58.368100 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.368069 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:58.465982 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.465947 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:58.601247 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.601215 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:58.646887 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.646808 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-sfkpv"] Jul 14 02:02:58.650157 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.650129 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-r2p5h"] Jul 14 02:02:58.650157 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.650159 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-sfkpv" Jul 14 02:02:58.652959 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.652932 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 14 02:02:58.652959 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.652951 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 14 02:02:58.653142 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.652956 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 14 02:02:58.653142 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.653001 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 14 02:02:58.653142 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.653070 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 14 02:02:58.653282 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.653248 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-drpzm\"" Jul 14 02:02:58.653448 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.653430 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-r2p5h" Jul 14 02:02:58.653544 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.653532 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 14 02:02:58.656837 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.656814 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-pgskw\"" Jul 14 02:02:58.656837 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.656833 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 14 02:02:58.656998 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.656814 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 14 02:02:58.656998 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.656907 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 14 02:02:58.660198 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.660179 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-sfkpv"] Jul 14 02:02:58.687425 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.687393 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:58.824542 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.824511 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvvd2\" (UniqueName: \"kubernetes.io/projected/d0fd1ddf-e24c-4639-a99f-28cc076262e5-kube-api-access-pvvd2\") pod \"node-exporter-r2p5h\" (UID: \"d0fd1ddf-e24c-4639-a99f-28cc076262e5\") " pod="openshift-monitoring/node-exporter-r2p5h" Jul 14 02:02:58.824999 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.824554 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/318d3e98-edc4-40e9-88fb-6f1558d193ad-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-sfkpv\" (UID: \"318d3e98-edc4-40e9-88fb-6f1558d193ad\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-sfkpv" Jul 14 02:02:58.824999 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.824581 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/318d3e98-edc4-40e9-88fb-6f1558d193ad-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-sfkpv\" (UID: \"318d3e98-edc4-40e9-88fb-6f1558d193ad\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-sfkpv" Jul 14 02:02:58.824999 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.824610 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d0fd1ddf-e24c-4639-a99f-28cc076262e5-root\") pod \"node-exporter-r2p5h\" (UID: \"d0fd1ddf-e24c-4639-a99f-28cc076262e5\") " pod="openshift-monitoring/node-exporter-r2p5h" Jul 14 02:02:58.824999 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.824634 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d0fd1ddf-e24c-4639-a99f-28cc076262e5-node-exporter-accelerators-collector-config\") pod \"node-exporter-r2p5h\" (UID: \"d0fd1ddf-e24c-4639-a99f-28cc076262e5\") " pod="openshift-monitoring/node-exporter-r2p5h" Jul 14 02:02:58.824999 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.824704 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/318d3e98-edc4-40e9-88fb-6f1558d193ad-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-sfkpv\" (UID: \"318d3e98-edc4-40e9-88fb-6f1558d193ad\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-sfkpv" Jul 14 02:02:58.824999 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.824760 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d0fd1ddf-e24c-4639-a99f-28cc076262e5-node-exporter-textfile\") pod \"node-exporter-r2p5h\" (UID: \"d0fd1ddf-e24c-4639-a99f-28cc076262e5\") " pod="openshift-monitoring/node-exporter-r2p5h" Jul 14 02:02:58.824999 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.824810 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d0fd1ddf-e24c-4639-a99f-28cc076262e5-node-exporter-wtmp\") pod \"node-exporter-r2p5h\" (UID: \"d0fd1ddf-e24c-4639-a99f-28cc076262e5\") " pod="openshift-monitoring/node-exporter-r2p5h" Jul 14 02:02:58.824999 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.824845 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0fd1ddf-e24c-4639-a99f-28cc076262e5-metrics-client-ca\") pod \"node-exporter-r2p5h\" (UID: \"d0fd1ddf-e24c-4639-a99f-28cc076262e5\") " pod="openshift-monitoring/node-exporter-r2p5h" Jul 14 02:02:58.824999 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.824877 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bpps\" (UniqueName: \"kubernetes.io/projected/318d3e98-edc4-40e9-88fb-6f1558d193ad-kube-api-access-6bpps\") pod \"kube-state-metrics-785778ff74-sfkpv\" (UID: \"318d3e98-edc4-40e9-88fb-6f1558d193ad\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-sfkpv" Jul 14 02:02:58.824999 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.824916 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/318d3e98-edc4-40e9-88fb-6f1558d193ad-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-sfkpv\" (UID: \"318d3e98-edc4-40e9-88fb-6f1558d193ad\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-sfkpv" Jul 14 02:02:58.824999 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.824936 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d0fd1ddf-e24c-4639-a99f-28cc076262e5-node-exporter-tls\") pod \"node-exporter-r2p5h\" (UID: \"d0fd1ddf-e24c-4639-a99f-28cc076262e5\") " pod="openshift-monitoring/node-exporter-r2p5h" Jul 14 02:02:58.824999 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.824959 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/318d3e98-edc4-40e9-88fb-6f1558d193ad-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-sfkpv\" (UID: \"318d3e98-edc4-40e9-88fb-6f1558d193ad\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-sfkpv" Jul 14 02:02:58.824999 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.824982 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d0fd1ddf-e24c-4639-a99f-28cc076262e5-sys\") pod \"node-exporter-r2p5h\" (UID: \"d0fd1ddf-e24c-4639-a99f-28cc076262e5\") " pod="openshift-monitoring/node-exporter-r2p5h" Jul 14 02:02:58.825511 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.825009 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d0fd1ddf-e24c-4639-a99f-28cc076262e5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r2p5h\" (UID: \"d0fd1ddf-e24c-4639-a99f-28cc076262e5\") " pod="openshift-monitoring/node-exporter-r2p5h" Jul 14 02:02:58.838007 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.837976 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:58.926347 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.926233 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/318d3e98-edc4-40e9-88fb-6f1558d193ad-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-sfkpv\" (UID: \"318d3e98-edc4-40e9-88fb-6f1558d193ad\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-sfkpv" Jul 14 02:02:58.926347 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.926296 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/318d3e98-edc4-40e9-88fb-6f1558d193ad-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-sfkpv\" (UID: \"318d3e98-edc4-40e9-88fb-6f1558d193ad\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-sfkpv" Jul 14 02:02:58.926347 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.926330 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d0fd1ddf-e24c-4639-a99f-28cc076262e5-root\") pod \"node-exporter-r2p5h\" (UID: \"d0fd1ddf-e24c-4639-a99f-28cc076262e5\") " pod="openshift-monitoring/node-exporter-r2p5h" Jul 14 02:02:58.926608 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.926353 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d0fd1ddf-e24c-4639-a99f-28cc076262e5-node-exporter-accelerators-collector-config\") pod \"node-exporter-r2p5h\" (UID: \"d0fd1ddf-e24c-4639-a99f-28cc076262e5\") " pod="openshift-monitoring/node-exporter-r2p5h" Jul 14 02:02:58.926608 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.926385 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/318d3e98-edc4-40e9-88fb-6f1558d193ad-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-sfkpv\" (UID: \"318d3e98-edc4-40e9-88fb-6f1558d193ad\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-sfkpv" Jul 14 02:02:58.926608 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.926419 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d0fd1ddf-e24c-4639-a99f-28cc076262e5-node-exporter-textfile\") pod \"node-exporter-r2p5h\" (UID: \"d0fd1ddf-e24c-4639-a99f-28cc076262e5\") " pod="openshift-monitoring/node-exporter-r2p5h" Jul 14 02:02:58.926608 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.926442 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d0fd1ddf-e24c-4639-a99f-28cc076262e5-node-exporter-wtmp\") pod \"node-exporter-r2p5h\" (UID: \"d0fd1ddf-e24c-4639-a99f-28cc076262e5\") " pod="openshift-monitoring/node-exporter-r2p5h" Jul 14 02:02:58.926608 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.926450 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d0fd1ddf-e24c-4639-a99f-28cc076262e5-root\") pod \"node-exporter-r2p5h\" (UID: \"d0fd1ddf-e24c-4639-a99f-28cc076262e5\") " pod="openshift-monitoring/node-exporter-r2p5h" Jul 14 02:02:58.926608 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.926462 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0fd1ddf-e24c-4639-a99f-28cc076262e5-metrics-client-ca\") pod \"node-exporter-r2p5h\" (UID: \"d0fd1ddf-e24c-4639-a99f-28cc076262e5\") " pod="openshift-monitoring/node-exporter-r2p5h" Jul 14 02:02:58.926608 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.926530 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bpps\" (UniqueName: \"kubernetes.io/projected/318d3e98-edc4-40e9-88fb-6f1558d193ad-kube-api-access-6bpps\") pod \"kube-state-metrics-785778ff74-sfkpv\" (UID: \"318d3e98-edc4-40e9-88fb-6f1558d193ad\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-sfkpv" Jul 14 02:02:58.926608 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.926577 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/318d3e98-edc4-40e9-88fb-6f1558d193ad-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-sfkpv\" (UID: \"318d3e98-edc4-40e9-88fb-6f1558d193ad\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-sfkpv" Jul 14 02:02:58.926608 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.926602 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d0fd1ddf-e24c-4639-a99f-28cc076262e5-node-exporter-tls\") pod \"node-exporter-r2p5h\" (UID: \"d0fd1ddf-e24c-4639-a99f-28cc076262e5\") " pod="openshift-monitoring/node-exporter-r2p5h" Jul 14 02:02:58.927041 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.926608 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d0fd1ddf-e24c-4639-a99f-28cc076262e5-node-exporter-wtmp\") pod \"node-exporter-r2p5h\" (UID: \"d0fd1ddf-e24c-4639-a99f-28cc076262e5\") " pod="openshift-monitoring/node-exporter-r2p5h" Jul 14 02:02:58.927041 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.926629 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/318d3e98-edc4-40e9-88fb-6f1558d193ad-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-sfkpv\" (UID: \"318d3e98-edc4-40e9-88fb-6f1558d193ad\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-sfkpv" Jul 14 02:02:58.927041 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.926657 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d0fd1ddf-e24c-4639-a99f-28cc076262e5-sys\") pod \"node-exporter-r2p5h\" (UID: \"d0fd1ddf-e24c-4639-a99f-28cc076262e5\") " pod="openshift-monitoring/node-exporter-r2p5h" Jul 14 02:02:58.927041 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.926688 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d0fd1ddf-e24c-4639-a99f-28cc076262e5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r2p5h\" (UID: \"d0fd1ddf-e24c-4639-a99f-28cc076262e5\") " pod="openshift-monitoring/node-exporter-r2p5h" Jul 14 02:02:58.927041 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.926688 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/318d3e98-edc4-40e9-88fb-6f1558d193ad-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-sfkpv\" (UID: \"318d3e98-edc4-40e9-88fb-6f1558d193ad\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-sfkpv" Jul 14 02:02:58.927041 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:58.926699 2577 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 14 02:02:58.927041 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.926741 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvvd2\" (UniqueName: \"kubernetes.io/projected/d0fd1ddf-e24c-4639-a99f-28cc076262e5-kube-api-access-pvvd2\") pod \"node-exporter-r2p5h\" (UID: \"d0fd1ddf-e24c-4639-a99f-28cc076262e5\") " pod="openshift-monitoring/node-exporter-r2p5h" Jul 14 02:02:58.927041 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:02:58.926770 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/318d3e98-edc4-40e9-88fb-6f1558d193ad-kube-state-metrics-tls podName:318d3e98-edc4-40e9-88fb-6f1558d193ad nodeName:}" failed. No retries permitted until 2026-07-14 02:02:59.426750049 +0000 UTC m=+81.581586690 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/318d3e98-edc4-40e9-88fb-6f1558d193ad-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-sfkpv" (UID: "318d3e98-edc4-40e9-88fb-6f1558d193ad") : secret "kube-state-metrics-tls" not found Jul 14 02:02:58.927041 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.926872 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d0fd1ddf-e24c-4639-a99f-28cc076262e5-node-exporter-textfile\") pod \"node-exporter-r2p5h\" (UID: \"d0fd1ddf-e24c-4639-a99f-28cc076262e5\") " pod="openshift-monitoring/node-exporter-r2p5h" Jul 14 02:02:58.927041 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.927009 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d0fd1ddf-e24c-4639-a99f-28cc076262e5-node-exporter-accelerators-collector-config\") pod \"node-exporter-r2p5h\" (UID: \"d0fd1ddf-e24c-4639-a99f-28cc076262e5\") " pod="openshift-monitoring/node-exporter-r2p5h" Jul 14 02:02:58.927547 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.927075 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0fd1ddf-e24c-4639-a99f-28cc076262e5-metrics-client-ca\") pod \"node-exporter-r2p5h\" (UID: \"d0fd1ddf-e24c-4639-a99f-28cc076262e5\") " pod="openshift-monitoring/node-exporter-r2p5h" Jul 14 02:02:58.927547 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.927085 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d0fd1ddf-e24c-4639-a99f-28cc076262e5-sys\") pod \"node-exporter-r2p5h\" (UID: \"d0fd1ddf-e24c-4639-a99f-28cc076262e5\") " pod="openshift-monitoring/node-exporter-r2p5h" Jul 14 02:02:58.927547 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.927129 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/318d3e98-edc4-40e9-88fb-6f1558d193ad-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-sfkpv\" (UID: \"318d3e98-edc4-40e9-88fb-6f1558d193ad\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-sfkpv" Jul 14 02:02:58.927547 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.927533 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/318d3e98-edc4-40e9-88fb-6f1558d193ad-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-sfkpv\" (UID: \"318d3e98-edc4-40e9-88fb-6f1558d193ad\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-sfkpv" Jul 14 02:02:58.929156 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.929125 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d0fd1ddf-e24c-4639-a99f-28cc076262e5-node-exporter-tls\") pod \"node-exporter-r2p5h\" (UID: \"d0fd1ddf-e24c-4639-a99f-28cc076262e5\") " pod="openshift-monitoring/node-exporter-r2p5h" Jul 14 02:02:58.929156 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.929151 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d0fd1ddf-e24c-4639-a99f-28cc076262e5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r2p5h\" (UID: \"d0fd1ddf-e24c-4639-a99f-28cc076262e5\") " pod="openshift-monitoring/node-exporter-r2p5h" Jul 14 02:02:58.929374 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.929317 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/318d3e98-edc4-40e9-88fb-6f1558d193ad-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-sfkpv\" (UID: \"318d3e98-edc4-40e9-88fb-6f1558d193ad\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-sfkpv" Jul 14 02:02:58.937989 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.937961 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvvd2\" (UniqueName: \"kubernetes.io/projected/d0fd1ddf-e24c-4639-a99f-28cc076262e5-kube-api-access-pvvd2\") pod \"node-exporter-r2p5h\" (UID: \"d0fd1ddf-e24c-4639-a99f-28cc076262e5\") " pod="openshift-monitoring/node-exporter-r2p5h" Jul 14 02:02:58.938134 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.938107 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bpps\" (UniqueName: \"kubernetes.io/projected/318d3e98-edc4-40e9-88fb-6f1558d193ad-kube-api-access-6bpps\") pod \"kube-state-metrics-785778ff74-sfkpv\" (UID: \"318d3e98-edc4-40e9-88fb-6f1558d193ad\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-sfkpv" Jul 14 02:02:58.966391 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.966355 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-r2p5h" Jul 14 02:02:58.971399 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:58.971375 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:58.978035 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:02:58.978003 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0fd1ddf_e24c_4639_a99f_28cc076262e5.slice/crio-25c8074c567fb95e360e76e828ab80528667a9c85ed1316a38d269a49ec60735 WatchSource:0}: Error finding container 25c8074c567fb95e360e76e828ab80528667a9c85ed1316a38d269a49ec60735: Status 404 returned error can't find the container with id 25c8074c567fb95e360e76e828ab80528667a9c85ed1316a38d269a49ec60735 Jul 14 02:02:59.072186 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:59.072152 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:59.240689 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:59.240605 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:59.342536 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:59.342507 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:59.431685 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:59.431652 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/318d3e98-edc4-40e9-88fb-6f1558d193ad-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-sfkpv\" (UID: \"318d3e98-edc4-40e9-88fb-6f1558d193ad\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-sfkpv" Jul 14 02:02:59.434355 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:59.434325 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/318d3e98-edc4-40e9-88fb-6f1558d193ad-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-sfkpv\" (UID: \"318d3e98-edc4-40e9-88fb-6f1558d193ad\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-sfkpv" Jul 14 02:02:59.561574 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:59.561494 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-sfkpv" Jul 14 02:02:59.584477 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:59.584444 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:59.665896 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:59.665847 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:59.703044 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:59.703017 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-sfkpv"] Jul 14 02:02:59.831431 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:59.831305 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r2p5h" event={"ID":"d0fd1ddf-e24c-4639-a99f-28cc076262e5","Type":"ContainerStarted","Data":"25c8074c567fb95e360e76e828ab80528667a9c85ed1316a38d269a49ec60735"} Jul 14 02:02:59.855558 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:02:59.855524 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod318d3e98_edc4_40e9_88fb_6f1558d193ad.slice/crio-d22dd744c561c98020fdb780a08d3ca15af9ffd66b2e878c3e2982d015467bdc WatchSource:0}: Error finding container d22dd744c561c98020fdb780a08d3ca15af9ffd66b2e878c3e2982d015467bdc: Status 404 returned error can't find the container with id d22dd744c561c98020fdb780a08d3ca15af9ffd66b2e878c3e2982d015467bdc Jul 14 02:02:59.882791 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:59.882764 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:02:59.984541 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:02:59.984518 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:00.086930 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:00.086848 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:00.836319 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:00.836282 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-sfkpv" event={"ID":"318d3e98-edc4-40e9-88fb-6f1558d193ad","Type":"ContainerStarted","Data":"d22dd744c561c98020fdb780a08d3ca15af9ffd66b2e878c3e2982d015467bdc"} Jul 14 02:03:00.837762 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:00.837737 2577 generic.go:356] "Generic (PLEG): container finished" podID="d0fd1ddf-e24c-4639-a99f-28cc076262e5" containerID="31067c0fac6fc53300a79f36eb164795528d074e5fe207b2bc769fcb1fcbcc38" exitCode=0 Jul 14 02:03:00.837908 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:00.837773 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r2p5h" event={"ID":"d0fd1ddf-e24c-4639-a99f-28cc076262e5","Type":"ContainerDied","Data":"31067c0fac6fc53300a79f36eb164795528d074e5fe207b2bc769fcb1fcbcc38"} Jul 14 02:03:01.654098 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:01.654056 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:01.715593 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:01.715552 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:01.844155 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:01.844054 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r2p5h" event={"ID":"d0fd1ddf-e24c-4639-a99f-28cc076262e5","Type":"ContainerStarted","Data":"fb249a4b1cc52d421db15f3fad1396e0ab695bd6ab2e368d18ecba84267532a7"} Jul 14 02:03:01.844635 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:01.844156 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r2p5h" event={"ID":"d0fd1ddf-e24c-4639-a99f-28cc076262e5","Type":"ContainerStarted","Data":"14abbea734e703d2487e61b53f148b0091544dffe1d6694aac32de772c38c834"} Jul 14 02:03:01.846663 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:01.846621 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-sfkpv" event={"ID":"318d3e98-edc4-40e9-88fb-6f1558d193ad","Type":"ContainerStarted","Data":"851a85b2e060ed10c816d719ed3c1ac868807c7d67e515d8540858efb4fd959f"} Jul 14 02:03:01.846788 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:01.846691 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-sfkpv" event={"ID":"318d3e98-edc4-40e9-88fb-6f1558d193ad","Type":"ContainerStarted","Data":"a31b636e9d0e7851bee31039e8f49862a79d8a349e827cb12d5d623fca8eb6bf"} Jul 14 02:03:01.846788 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:01.846718 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-sfkpv" event={"ID":"318d3e98-edc4-40e9-88fb-6f1558d193ad","Type":"ContainerStarted","Data":"4a9d9b122208319161c991b5bb088f5c7c8420cffce3537f25bb64bb2264917b"} Jul 14 02:03:01.864504 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:01.864448 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-r2p5h" podStartSLOduration=2.879659996 podStartE2EDuration="3.864432973s" totalImagesPullingTime="921.383739ms" totalInitContainerRuntime="63.389238ms" isStatefulPod=true podCreationTimestamp="2026-07-14 02:02:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:03:01.862654904 +0000 UTC m=+84.017491553" watchObservedRunningTime="2026-07-14 02:03:01.864432973 +0000 UTC m=+84.019269620" Jul 14 02:03:01.880886 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:01.880835 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-sfkpv" podStartSLOduration=2.443888311 podStartE2EDuration="3.880816698s" totalImagesPullingTime="1.436928387s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:02:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:03:01.879883975 +0000 UTC m=+84.034720634" watchObservedRunningTime="2026-07-14 02:03:01.880816698 +0000 UTC m=+84.035653346" Jul 14 02:03:01.932563 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:01.932534 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:02.135695 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:02.135663 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:02.751332 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:02.751295 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7b4cb8c7f6-kn9xv" Jul 14 02:03:02.886037 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:02.886004 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:02.988822 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:02.988786 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:03.087128 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:03.087041 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:03.404657 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:03.404618 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:03.757732 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:03.757658 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:04.107239 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.107165 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:04.207631 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.207605 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:04.915827 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.915788 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 02:03:04.920522 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.920499 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:04.923176 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.923151 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 14 02:03:04.923318 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.923183 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 14 02:03:04.923318 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.923183 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5rr5kl436dgm0\"" Jul 14 02:03:04.923446 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.923432 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 14 02:03:04.923641 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.923627 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 14 02:03:04.923761 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.923744 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 14 02:03:04.923761 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.923756 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 14 02:03:04.923881 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.923764 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-hp76r\"" Jul 14 02:03:04.923881 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.923788 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 14 02:03:04.923881 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.923797 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 14 02:03:04.923881 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.923817 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 14 02:03:04.924113 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.924099 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 14 02:03:04.924310 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.924290 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 14 02:03:04.924372 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.924343 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 14 02:03:04.924596 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.924579 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 14 02:03:04.924744 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.924728 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 14 02:03:04.932439 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.932418 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 14 02:03:04.934770 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.934752 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 02:03:04.940894 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.940872 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:04.976071 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.976042 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:04.976071 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.976074 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxvcs\" (UniqueName: \"kubernetes.io/projected/a15093fc-3e71-4af0-bb06-fd31cf42798c-kube-api-access-gxvcs\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:04.976258 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.976103 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:04.976258 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.976128 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a15093fc-3e71-4af0-bb06-fd31cf42798c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:04.976258 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.976155 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:04.976258 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.976169 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a15093fc-3e71-4af0-bb06-fd31cf42798c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:04.976258 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.976188 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:04.976425 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.976238 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-web-config\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:04.976425 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.976339 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:04.976425 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.976359 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:04.976425 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.976375 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:04.976425 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.976394 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:04.976425 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.976413 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:04.976659 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.976431 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:04.976659 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.976457 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:04.976659 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.976496 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:04.976659 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.976525 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:04.976659 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.976540 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-config\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:04.976659 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.976555 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a15093fc-3e71-4af0-bb06-fd31cf42798c-config-out\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:04.976659 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:04.976584 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.077248 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.077210 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.077248 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.077248 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxvcs\" (UniqueName: \"kubernetes.io/projected/a15093fc-3e71-4af0-bb06-fd31cf42798c-kube-api-access-gxvcs\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.077536 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.077305 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.077536 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.077325 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a15093fc-3e71-4af0-bb06-fd31cf42798c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.077536 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.077354 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.077536 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.077481 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a15093fc-3e71-4af0-bb06-fd31cf42798c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.077536 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.077537 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.077779 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.077568 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-web-config\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.077779 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.077613 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.077779 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.077652 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.077779 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.077686 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.077779 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.077711 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.077779 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.077742 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.077779 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.077773 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.078087 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.077799 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.078087 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.077840 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.078087 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.077887 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.078087 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.077913 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-config\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.078087 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.077913 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a15093fc-3e71-4af0-bb06-fd31cf42798c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.078087 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.077935 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a15093fc-3e71-4af0-bb06-fd31cf42798c-config-out\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.078087 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.077994 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.078625 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.078590 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.078936 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.078913 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.080613 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.079394 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.080613 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.080419 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a15093fc-3e71-4af0-bb06-fd31cf42798c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.080788 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.080648 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-web-config\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.080788 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.080693 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a15093fc-3e71-4af0-bb06-fd31cf42798c-config-out\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.080996 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.080972 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.081713 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.081605 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.081811 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.081767 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.081955 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.081933 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.082366 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.082341 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.082684 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.082664 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.082809 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.082766 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.082992 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.082966 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.083147 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.083123 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.083738 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.083716 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.083874 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.083853 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-config\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.083914 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.083862 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.085409 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.085388 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxvcs\" (UniqueName: \"kubernetes.io/projected/a15093fc-3e71-4af0-bb06-fd31cf42798c-kube-api-access-gxvcs\") pod \"prometheus-k8s-0\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.234496 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.234414 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:05.369198 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.369029 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 02:03:05.371842 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:03:05.371819 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda15093fc_3e71_4af0_bb06_fd31cf42798c.slice/crio-5a747a467ca6888a286b6ed21291e50048d142988d620ef823d4173944bf85a0 WatchSource:0}: Error finding container 5a747a467ca6888a286b6ed21291e50048d142988d620ef823d4173944bf85a0: Status 404 returned error can't find the container with id 5a747a467ca6888a286b6ed21291e50048d142988d620ef823d4173944bf85a0 Jul 14 02:03:05.862872 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.862808 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a15093fc-3e71-4af0-bb06-fd31cf42798c","Type":"ContainerStarted","Data":"5a747a467ca6888a286b6ed21291e50048d142988d620ef823d4173944bf85a0"} Jul 14 02:03:05.924646 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:05.924589 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" podUID="b2ab7b4e-fa36-44f3-86e6-151a58a5631b" containerName="registry" containerID="cri-o://5bc2c3019de4f3c9e7e48bd49be473e0ff91d9e75eb8b860612d364192ed01ad" gracePeriod=30 Jul 14 02:03:06.348573 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:06.348507 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" Jul 14 02:03:06.387811 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:06.387785 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-trusted-ca\") pod \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\" (UID: \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\") " Jul 14 02:03:06.387976 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:06.387823 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-bound-sa-token\") pod \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\" (UID: \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\") " Jul 14 02:03:06.387976 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:06.387859 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-installation-pull-secrets\") pod \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\" (UID: \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\") " Jul 14 02:03:06.388089 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:06.387969 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-ca-trust-extracted\") pod \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\" (UID: \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\") " Jul 14 02:03:06.388089 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:06.388005 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-registry-tls\") pod \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\" (UID: \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\") " Jul 14 02:03:06.388089 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:06.388034 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-image-registry-private-configuration\") pod \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\" (UID: \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\") " Jul 14 02:03:06.388233 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:06.388173 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6j5cc\" (UniqueName: \"kubernetes.io/projected/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-kube-api-access-6j5cc\") pod \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\" (UID: \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\") " Jul 14 02:03:06.388233 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:06.388227 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-registry-certificates\") pod \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\" (UID: \"b2ab7b4e-fa36-44f3-86e6-151a58a5631b\") " Jul 14 02:03:06.388369 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:06.388245 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "b2ab7b4e-fa36-44f3-86e6-151a58a5631b" (UID: "b2ab7b4e-fa36-44f3-86e6-151a58a5631b"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 02:03:06.388534 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:06.388513 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-trusted-ca\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:03:06.388932 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:06.388900 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "b2ab7b4e-fa36-44f3-86e6-151a58a5631b" (UID: "b2ab7b4e-fa36-44f3-86e6-151a58a5631b"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 02:03:06.390506 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:06.390454 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "b2ab7b4e-fa36-44f3-86e6-151a58a5631b" (UID: "b2ab7b4e-fa36-44f3-86e6-151a58a5631b"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 02:03:06.390583 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:06.390507 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "b2ab7b4e-fa36-44f3-86e6-151a58a5631b" (UID: "b2ab7b4e-fa36-44f3-86e6-151a58a5631b"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:03:06.390583 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:06.390546 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "b2ab7b4e-fa36-44f3-86e6-151a58a5631b" (UID: "b2ab7b4e-fa36-44f3-86e6-151a58a5631b"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 02:03:06.390583 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:06.390576 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "b2ab7b4e-fa36-44f3-86e6-151a58a5631b" (UID: "b2ab7b4e-fa36-44f3-86e6-151a58a5631b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:03:06.390689 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:06.390647 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-kube-api-access-6j5cc" (OuterVolumeSpecName: "kube-api-access-6j5cc") pod "b2ab7b4e-fa36-44f3-86e6-151a58a5631b" (UID: "b2ab7b4e-fa36-44f3-86e6-151a58a5631b"). InnerVolumeSpecName "kube-api-access-6j5cc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:03:06.396566 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:06.396543 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "b2ab7b4e-fa36-44f3-86e6-151a58a5631b" (UID: "b2ab7b4e-fa36-44f3-86e6-151a58a5631b"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:03:06.489584 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:06.489506 2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-ca-trust-extracted\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:03:06.489584 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:06.489530 2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-registry-tls\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:03:06.489584 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:06.489540 2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-image-registry-private-configuration\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:03:06.489584 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:06.489550 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6j5cc\" (UniqueName: \"kubernetes.io/projected/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-kube-api-access-6j5cc\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:03:06.489584 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:06.489559 2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-registry-certificates\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:03:06.489584 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:06.489568 2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-bound-sa-token\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:03:06.489584 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:06.489577 2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b2ab7b4e-fa36-44f3-86e6-151a58a5631b-installation-pull-secrets\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:03:06.827611 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:06.827531 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:06.868238 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:06.868200 2577 generic.go:356] "Generic (PLEG): container finished" podID="b2ab7b4e-fa36-44f3-86e6-151a58a5631b" containerID="5bc2c3019de4f3c9e7e48bd49be473e0ff91d9e75eb8b860612d364192ed01ad" exitCode=0 Jul 14 02:03:06.868396 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:06.868247 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" Jul 14 02:03:06.868396 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:06.868288 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" event={"ID":"b2ab7b4e-fa36-44f3-86e6-151a58a5631b","Type":"ContainerDied","Data":"5bc2c3019de4f3c9e7e48bd49be473e0ff91d9e75eb8b860612d364192ed01ad"} Jul 14 02:03:06.868396 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:06.868336 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6497d48c68-d9xm4" event={"ID":"b2ab7b4e-fa36-44f3-86e6-151a58a5631b","Type":"ContainerDied","Data":"0b85ebd5d1986b77585304c75e8e65777aefc1a3e93d1c8254935b64d940eb58"} Jul 14 02:03:06.868939 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:06.868913 2577 scope.go:117] "RemoveContainer" containerID="5bc2c3019de4f3c9e7e48bd49be473e0ff91d9e75eb8b860612d364192ed01ad" Jul 14 02:03:06.870119 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:06.870074 2577 generic.go:356] "Generic (PLEG): container finished" podID="a15093fc-3e71-4af0-bb06-fd31cf42798c" containerID="ae0258c118bc2331427fd847b5918589273fdbb62e667414704778144862da67" exitCode=0 Jul 14 02:03:06.870119 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:06.870111 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a15093fc-3e71-4af0-bb06-fd31cf42798c","Type":"ContainerDied","Data":"ae0258c118bc2331427fd847b5918589273fdbb62e667414704778144862da67"} Jul 14 02:03:06.878230 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:06.878214 2577 scope.go:117] "RemoveContainer" containerID="5bc2c3019de4f3c9e7e48bd49be473e0ff91d9e75eb8b860612d364192ed01ad" Jul 14 02:03:06.878519 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:03:06.878498 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bc2c3019de4f3c9e7e48bd49be473e0ff91d9e75eb8b860612d364192ed01ad\": container with ID starting with 5bc2c3019de4f3c9e7e48bd49be473e0ff91d9e75eb8b860612d364192ed01ad not found: ID does not exist" containerID="5bc2c3019de4f3c9e7e48bd49be473e0ff91d9e75eb8b860612d364192ed01ad" Jul 14 02:03:06.878593 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:06.878531 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bc2c3019de4f3c9e7e48bd49be473e0ff91d9e75eb8b860612d364192ed01ad"} err="failed to get container status \"5bc2c3019de4f3c9e7e48bd49be473e0ff91d9e75eb8b860612d364192ed01ad\": rpc error: code = NotFound desc = could not find container \"5bc2c3019de4f3c9e7e48bd49be473e0ff91d9e75eb8b860612d364192ed01ad\": container with ID starting with 5bc2c3019de4f3c9e7e48bd49be473e0ff91d9e75eb8b860612d364192ed01ad not found: ID does not exist" Jul 14 02:03:06.910470 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:06.910430 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:06.919297 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:06.919253 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6497d48c68-d9xm4"] Jul 14 02:03:06.925579 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:06.925556 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6497d48c68-d9xm4"] Jul 14 02:03:08.265629 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:08.265591 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2ab7b4e-fa36-44f3-86e6-151a58a5631b" path="/var/lib/kubelet/pods/b2ab7b4e-fa36-44f3-86e6-151a58a5631b/volumes" Jul 14 02:03:08.628600 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:08.628562 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:08.729439 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:08.729365 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:08.782382 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:08.782348 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:08.920615 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:08.920513 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:09.298834 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:09.298756 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:09.359200 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:09.359164 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:09.506690 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:09.506656 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:09.611893 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:09.611860 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:09.718404 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:09.718371 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:09.821931 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:09.821896 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:10.312570 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:10.312536 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:10.525687 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:10.525600 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:10.665851 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:10.665801 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:10.892173 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:10.892143 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a15093fc-3e71-4af0-bb06-fd31cf42798c","Type":"ContainerStarted","Data":"fbf948905b0e5dc26d1c1243c707d05223790b702602a185e9651eea7f7cd448"} Jul 14 02:03:10.892371 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:10.892210 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a15093fc-3e71-4af0-bb06-fd31cf42798c","Type":"ContainerStarted","Data":"ec381b730242ff9a9ad97a2ebc7fdc85722214a72cd8c37c707dcfdaaf356e22"} Jul 14 02:03:12.151562 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:12.151529 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:12.203754 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:12.203684 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:12.903885 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:12.903848 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a15093fc-3e71-4af0-bb06-fd31cf42798c","Type":"ContainerStarted","Data":"68122e3957ab11875bb01e88e61318ae5c69697610e03c5341a7118dd931be2a"} Jul 14 02:03:12.904085 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:12.903911 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a15093fc-3e71-4af0-bb06-fd31cf42798c","Type":"ContainerStarted","Data":"14da291faf52a6dccc595ebb36b0ec6aac985cf9272545cd874399fe7be1c65f"} Jul 14 02:03:12.904085 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:12.903936 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a15093fc-3e71-4af0-bb06-fd31cf42798c","Type":"ContainerStarted","Data":"f2a2cecb3b8bd91c56f86ea58996467c9eb7885127c600af9e5f502ea610c9f6"} Jul 14 02:03:12.904085 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:12.903959 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a15093fc-3e71-4af0-bb06-fd31cf42798c","Type":"ContainerStarted","Data":"4fe1ef404db5348be10227a3b6a070c04df78107dfbed03d6ffb57664e6ae2f4"} Jul 14 02:03:12.935038 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:12.934989 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.964581083 podStartE2EDuration="8.934973354s" totalImagesPullingTime="5.947179489s" totalInitContainerRuntime="23.212782ms" isStatefulPod=false podCreationTimestamp="2026-07-14 02:03:04 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:03:12.934071191 +0000 UTC m=+95.088907849" watchObservedRunningTime="2026-07-14 02:03:12.934973354 +0000 UTC m=+95.089810001" Jul 14 02:03:15.234925 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:15.234878 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:03:16.251886 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:16.251846 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:16.304815 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:16.304785 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:16.775727 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:16.775697 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-c6wmg" Jul 14 02:03:20.546149 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:20.546111 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:24.121132 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:24.121095 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:24.172313 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:24.172248 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:26.955790 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:26.955760 2577 generic.go:356] "Generic (PLEG): container finished" podID="bbc58bc3-8649-42d2-a277-c27b1c6670c9" containerID="e42d635c9ff1ef5e3db5b1347ac9d9cebacaf5d7d749c57d8a2a3828856ebd32" exitCode=0 Jul 14 02:03:26.956186 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:26.955832 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-bkdjp" event={"ID":"bbc58bc3-8649-42d2-a277-c27b1c6670c9","Type":"ContainerDied","Data":"e42d635c9ff1ef5e3db5b1347ac9d9cebacaf5d7d749c57d8a2a3828856ebd32"} Jul 14 02:03:26.956230 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:26.956195 2577 scope.go:117] "RemoveContainer" containerID="e42d635c9ff1ef5e3db5b1347ac9d9cebacaf5d7d749c57d8a2a3828856ebd32" Jul 14 02:03:27.532780 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:27.532743 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:27.961557 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:27.961521 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-bkdjp" event={"ID":"bbc58bc3-8649-42d2-a277-c27b1c6670c9","Type":"ContainerStarted","Data":"24ace1b7c7e30570d96787738f66767afa93498f89db1739f884f136d7377e92"} Jul 14 02:03:29.329415 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:29.329378 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:29.377527 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:29.377493 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:30.729340 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:30.729295 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:30.832590 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:30.832555 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:31.238436 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:31.238406 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:31.340422 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:31.340393 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:31.702904 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:31.702872 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:31.766415 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:31.766386 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:37.539631 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:37.539595 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:37.593177 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:37.593146 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:38.615579 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:38.615546 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:38.671417 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:38.671380 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:38.810324 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:38.810266 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:46.238160 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:46.238131 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:46.305131 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:46.305098 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:48.033222 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:48.033187 2577 generic.go:356] "Generic (PLEG): container finished" podID="78343924-8733-41ae-9863-94ec59df33d5" containerID="b737076df4093cd9e8d92d2780f51d7798ddae142038214ddd8e394955409cec" exitCode=0 Jul 14 02:03:48.033675 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:48.033260 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rn5p6" event={"ID":"78343924-8733-41ae-9863-94ec59df33d5","Type":"ContainerDied","Data":"b737076df4093cd9e8d92d2780f51d7798ddae142038214ddd8e394955409cec"} Jul 14 02:03:48.033675 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:48.033611 2577 scope.go:117] "RemoveContainer" containerID="b737076df4093cd9e8d92d2780f51d7798ddae142038214ddd8e394955409cec" Jul 14 02:03:49.038553 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:49.038515 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rn5p6" event={"ID":"78343924-8733-41ae-9863-94ec59df33d5","Type":"ContainerStarted","Data":"3a55c5421b12f36d559c66ecadbe41f9554fc26977dd6615b6c303df5ea70624"} Jul 14 02:03:50.510160 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:50.510124 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:03:58.601991 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:03:58.601954 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:04:05.234800 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:05.234755 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:05.250419 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:05.250393 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:06.119363 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:06.119336 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:10.169430 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:10.169398 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:04:17.852052 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:17.852016 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:04:18.146619 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:18.146581 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:04:23.296423 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.296388 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 02:04:23.296840 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.296720 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a15093fc-3e71-4af0-bb06-fd31cf42798c" containerName="kube-rbac-proxy" containerID="cri-o://14da291faf52a6dccc595ebb36b0ec6aac985cf9272545cd874399fe7be1c65f" gracePeriod=600 Jul 14 02:04:23.296840 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.296757 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a15093fc-3e71-4af0-bb06-fd31cf42798c" containerName="kube-rbac-proxy-web" containerID="cri-o://f2a2cecb3b8bd91c56f86ea58996467c9eb7885127c600af9e5f502ea610c9f6" gracePeriod=600 Jul 14 02:04:23.296840 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.296764 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a15093fc-3e71-4af0-bb06-fd31cf42798c" containerName="config-reloader" containerID="cri-o://fbf948905b0e5dc26d1c1243c707d05223790b702602a185e9651eea7f7cd448" gracePeriod=600 Jul 14 02:04:23.296840 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.296781 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a15093fc-3e71-4af0-bb06-fd31cf42798c" containerName="kube-rbac-proxy-thanos" containerID="cri-o://68122e3957ab11875bb01e88e61318ae5c69697610e03c5341a7118dd931be2a" gracePeriod=600 Jul 14 02:04:23.296840 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.296731 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a15093fc-3e71-4af0-bb06-fd31cf42798c" containerName="prometheus" containerID="cri-o://ec381b730242ff9a9ad97a2ebc7fdc85722214a72cd8c37c707dcfdaaf356e22" gracePeriod=600 Jul 14 02:04:23.297075 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.296764 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a15093fc-3e71-4af0-bb06-fd31cf42798c" containerName="thanos-sidecar" containerID="cri-o://4fe1ef404db5348be10227a3b6a070c04df78107dfbed03d6ffb57664e6ae2f4" gracePeriod=600 Jul 14 02:04:23.526668 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.526635 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:23.650848 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.650816 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-prometheus-k8s-rulefiles-2\") pod \"a15093fc-3e71-4af0-bb06-fd31cf42798c\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " Jul 14 02:04:23.650987 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.650857 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-thanos-prometheus-http-client-file\") pod \"a15093fc-3e71-4af0-bb06-fd31cf42798c\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " Jul 14 02:04:23.650987 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.650878 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a15093fc-3e71-4af0-bb06-fd31cf42798c-tls-assets\") pod \"a15093fc-3e71-4af0-bb06-fd31cf42798c\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " Jul 14 02:04:23.650987 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.650913 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-config\") pod \"a15093fc-3e71-4af0-bb06-fd31cf42798c\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " Jul 14 02:04:23.650987 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.650946 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxvcs\" (UniqueName: \"kubernetes.io/projected/a15093fc-3e71-4af0-bb06-fd31cf42798c-kube-api-access-gxvcs\") pod \"a15093fc-3e71-4af0-bb06-fd31cf42798c\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " Jul 14 02:04:23.650987 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.650970 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-configmap-metrics-client-ca\") pod \"a15093fc-3e71-4af0-bb06-fd31cf42798c\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " Jul 14 02:04:23.651257 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.650993 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"a15093fc-3e71-4af0-bb06-fd31cf42798c\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " Jul 14 02:04:23.651257 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.651023 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-secret-metrics-client-certs\") pod \"a15093fc-3e71-4af0-bb06-fd31cf42798c\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " Jul 14 02:04:23.651257 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.651049 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-prometheus-k8s-rulefiles-0\") pod \"a15093fc-3e71-4af0-bb06-fd31cf42798c\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " Jul 14 02:04:23.651257 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.651076 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-secret-grpc-tls\") pod \"a15093fc-3e71-4af0-bb06-fd31cf42798c\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " Jul 14 02:04:23.651257 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.651100 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"a15093fc-3e71-4af0-bb06-fd31cf42798c\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " Jul 14 02:04:23.651257 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.651127 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-web-config\") pod \"a15093fc-3e71-4af0-bb06-fd31cf42798c\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " Jul 14 02:04:23.651257 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.651182 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-configmap-kubelet-serving-ca-bundle\") pod \"a15093fc-3e71-4af0-bb06-fd31cf42798c\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " Jul 14 02:04:23.651257 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.651209 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-prometheus-trusted-ca-bundle\") pod \"a15093fc-3e71-4af0-bb06-fd31cf42798c\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " Jul 14 02:04:23.651257 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.651240 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-prometheus-k8s-rulefiles-1\") pod \"a15093fc-3e71-4af0-bb06-fd31cf42798c\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " Jul 14 02:04:23.651257 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.651246 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "a15093fc-3e71-4af0-bb06-fd31cf42798c" (UID: "a15093fc-3e71-4af0-bb06-fd31cf42798c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 02:04:23.651779 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.651266 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-configmap-serving-certs-ca-bundle\") pod \"a15093fc-3e71-4af0-bb06-fd31cf42798c\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " Jul 14 02:04:23.651779 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.651444 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-secret-kube-rbac-proxy\") pod \"a15093fc-3e71-4af0-bb06-fd31cf42798c\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " Jul 14 02:04:23.651779 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.651478 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a15093fc-3e71-4af0-bb06-fd31cf42798c-prometheus-k8s-db\") pod \"a15093fc-3e71-4af0-bb06-fd31cf42798c\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " Jul 14 02:04:23.651779 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.651506 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-secret-prometheus-k8s-tls\") pod \"a15093fc-3e71-4af0-bb06-fd31cf42798c\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " Jul 14 02:04:23.651779 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.651534 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a15093fc-3e71-4af0-bb06-fd31cf42798c-config-out\") pod \"a15093fc-3e71-4af0-bb06-fd31cf42798c\" (UID: \"a15093fc-3e71-4af0-bb06-fd31cf42798c\") " Jul 14 02:04:23.651779 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.651558 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "a15093fc-3e71-4af0-bb06-fd31cf42798c" (UID: "a15093fc-3e71-4af0-bb06-fd31cf42798c"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 02:04:23.651779 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.651755 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "a15093fc-3e71-4af0-bb06-fd31cf42798c" (UID: "a15093fc-3e71-4af0-bb06-fd31cf42798c"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 02:04:23.652130 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.651800 2577 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-configmap-metrics-client-ca\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:04:23.652130 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.651819 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:04:23.652130 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.652062 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "a15093fc-3e71-4af0-bb06-fd31cf42798c" (UID: "a15093fc-3e71-4af0-bb06-fd31cf42798c"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 02:04:23.652487 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.652463 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "a15093fc-3e71-4af0-bb06-fd31cf42798c" (UID: "a15093fc-3e71-4af0-bb06-fd31cf42798c"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 02:04:23.652892 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.652752 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a15093fc-3e71-4af0-bb06-fd31cf42798c-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "a15093fc-3e71-4af0-bb06-fd31cf42798c" (UID: "a15093fc-3e71-4af0-bb06-fd31cf42798c"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:04:23.653098 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.653081 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "a15093fc-3e71-4af0-bb06-fd31cf42798c" (UID: "a15093fc-3e71-4af0-bb06-fd31cf42798c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 02:04:23.654333 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.654302 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "a15093fc-3e71-4af0-bb06-fd31cf42798c" (UID: "a15093fc-3e71-4af0-bb06-fd31cf42798c"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 02:04:23.655047 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.655017 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "a15093fc-3e71-4af0-bb06-fd31cf42798c" (UID: "a15093fc-3e71-4af0-bb06-fd31cf42798c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 02:04:23.655131 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.655102 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "a15093fc-3e71-4af0-bb06-fd31cf42798c" (UID: "a15093fc-3e71-4af0-bb06-fd31cf42798c"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 02:04:23.655131 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.655117 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a15093fc-3e71-4af0-bb06-fd31cf42798c-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "a15093fc-3e71-4af0-bb06-fd31cf42798c" (UID: "a15093fc-3e71-4af0-bb06-fd31cf42798c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:04:23.655245 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.655150 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "a15093fc-3e71-4af0-bb06-fd31cf42798c" (UID: "a15093fc-3e71-4af0-bb06-fd31cf42798c"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 02:04:23.655245 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.655179 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-config" (OuterVolumeSpecName: "config") pod "a15093fc-3e71-4af0-bb06-fd31cf42798c" (UID: "a15093fc-3e71-4af0-bb06-fd31cf42798c"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 02:04:23.655623 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.655595 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a15093fc-3e71-4af0-bb06-fd31cf42798c-kube-api-access-gxvcs" (OuterVolumeSpecName: "kube-api-access-gxvcs") pod "a15093fc-3e71-4af0-bb06-fd31cf42798c" (UID: "a15093fc-3e71-4af0-bb06-fd31cf42798c"). InnerVolumeSpecName "kube-api-access-gxvcs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:04:23.655768 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.655750 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "a15093fc-3e71-4af0-bb06-fd31cf42798c" (UID: "a15093fc-3e71-4af0-bb06-fd31cf42798c"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 02:04:23.655915 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.655895 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "a15093fc-3e71-4af0-bb06-fd31cf42798c" (UID: "a15093fc-3e71-4af0-bb06-fd31cf42798c"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 02:04:23.656063 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.656050 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a15093fc-3e71-4af0-bb06-fd31cf42798c-config-out" (OuterVolumeSpecName: "config-out") pod "a15093fc-3e71-4af0-bb06-fd31cf42798c" (UID: "a15093fc-3e71-4af0-bb06-fd31cf42798c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:04:23.656159 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.656141 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "a15093fc-3e71-4af0-bb06-fd31cf42798c" (UID: "a15093fc-3e71-4af0-bb06-fd31cf42798c"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 02:04:23.657397 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.657374 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "a15093fc-3e71-4af0-bb06-fd31cf42798c" (UID: "a15093fc-3e71-4af0-bb06-fd31cf42798c"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 02:04:23.664755 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.664735 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-web-config" (OuterVolumeSpecName: "web-config") pod "a15093fc-3e71-4af0-bb06-fd31cf42798c" (UID: "a15093fc-3e71-4af0-bb06-fd31cf42798c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 02:04:23.752512 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.752478 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxvcs\" (UniqueName: \"kubernetes.io/projected/a15093fc-3e71-4af0-bb06-fd31cf42798c-kube-api-access-gxvcs\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:04:23.752512 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.752508 2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:04:23.752512 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.752520 2577 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-secret-metrics-client-certs\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:04:23.752727 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.752529 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:04:23.752727 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.752538 2577 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-secret-grpc-tls\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:04:23.752727 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.752548 2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:04:23.752727 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.752576 2577 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-web-config\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:04:23.752727 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.752585 2577 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:04:23.752727 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.752594 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-prometheus-trusted-ca-bundle\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:04:23.752727 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.752602 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:04:23.752727 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.752610 2577 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a15093fc-3e71-4af0-bb06-fd31cf42798c-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:04:23.752727 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.752620 2577 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-secret-kube-rbac-proxy\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:04:23.752727 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.752628 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a15093fc-3e71-4af0-bb06-fd31cf42798c-prometheus-k8s-db\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:04:23.752727 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.752637 2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-secret-prometheus-k8s-tls\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:04:23.752727 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.752645 2577 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a15093fc-3e71-4af0-bb06-fd31cf42798c-config-out\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:04:23.752727 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.752654 2577 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-thanos-prometheus-http-client-file\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:04:23.752727 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.752662 2577 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a15093fc-3e71-4af0-bb06-fd31cf42798c-tls-assets\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:04:23.752727 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:23.752670 2577 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/a15093fc-3e71-4af0-bb06-fd31cf42798c-config\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:04:24.168198 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.168163 2577 generic.go:356] "Generic (PLEG): container finished" podID="a15093fc-3e71-4af0-bb06-fd31cf42798c" containerID="68122e3957ab11875bb01e88e61318ae5c69697610e03c5341a7118dd931be2a" exitCode=0 Jul 14 02:04:24.168198 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.168193 2577 generic.go:356] "Generic (PLEG): container finished" podID="a15093fc-3e71-4af0-bb06-fd31cf42798c" containerID="14da291faf52a6dccc595ebb36b0ec6aac985cf9272545cd874399fe7be1c65f" exitCode=0 Jul 14 02:04:24.168198 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.168203 2577 generic.go:356] "Generic (PLEG): container finished" podID="a15093fc-3e71-4af0-bb06-fd31cf42798c" containerID="f2a2cecb3b8bd91c56f86ea58996467c9eb7885127c600af9e5f502ea610c9f6" exitCode=0 Jul 14 02:04:24.168449 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.168212 2577 generic.go:356] "Generic (PLEG): container finished" podID="a15093fc-3e71-4af0-bb06-fd31cf42798c" containerID="4fe1ef404db5348be10227a3b6a070c04df78107dfbed03d6ffb57664e6ae2f4" exitCode=0 Jul 14 02:04:24.168449 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.168219 2577 generic.go:356] "Generic (PLEG): container finished" podID="a15093fc-3e71-4af0-bb06-fd31cf42798c" containerID="fbf948905b0e5dc26d1c1243c707d05223790b702602a185e9651eea7f7cd448" exitCode=0 Jul 14 02:04:24.168449 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.168226 2577 generic.go:356] "Generic (PLEG): container finished" podID="a15093fc-3e71-4af0-bb06-fd31cf42798c" containerID="ec381b730242ff9a9ad97a2ebc7fdc85722214a72cd8c37c707dcfdaaf356e22" exitCode=0 Jul 14 02:04:24.168449 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.168256 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.168449 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.168293 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a15093fc-3e71-4af0-bb06-fd31cf42798c","Type":"ContainerDied","Data":"68122e3957ab11875bb01e88e61318ae5c69697610e03c5341a7118dd931be2a"} Jul 14 02:04:24.168449 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.168393 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a15093fc-3e71-4af0-bb06-fd31cf42798c","Type":"ContainerDied","Data":"14da291faf52a6dccc595ebb36b0ec6aac985cf9272545cd874399fe7be1c65f"} Jul 14 02:04:24.168449 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.168425 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a15093fc-3e71-4af0-bb06-fd31cf42798c","Type":"ContainerDied","Data":"f2a2cecb3b8bd91c56f86ea58996467c9eb7885127c600af9e5f502ea610c9f6"} Jul 14 02:04:24.168664 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.168456 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a15093fc-3e71-4af0-bb06-fd31cf42798c","Type":"ContainerDied","Data":"4fe1ef404db5348be10227a3b6a070c04df78107dfbed03d6ffb57664e6ae2f4"} Jul 14 02:04:24.168664 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.168492 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a15093fc-3e71-4af0-bb06-fd31cf42798c","Type":"ContainerDied","Data":"fbf948905b0e5dc26d1c1243c707d05223790b702602a185e9651eea7f7cd448"} Jul 14 02:04:24.168664 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.168466 2577 scope.go:117] "RemoveContainer" containerID="68122e3957ab11875bb01e88e61318ae5c69697610e03c5341a7118dd931be2a" Jul 14 02:04:24.168664 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.168542 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a15093fc-3e71-4af0-bb06-fd31cf42798c","Type":"ContainerDied","Data":"ec381b730242ff9a9ad97a2ebc7fdc85722214a72cd8c37c707dcfdaaf356e22"} Jul 14 02:04:24.168664 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.168583 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a15093fc-3e71-4af0-bb06-fd31cf42798c","Type":"ContainerDied","Data":"5a747a467ca6888a286b6ed21291e50048d142988d620ef823d4173944bf85a0"} Jul 14 02:04:24.176974 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.176958 2577 scope.go:117] "RemoveContainer" containerID="14da291faf52a6dccc595ebb36b0ec6aac985cf9272545cd874399fe7be1c65f" Jul 14 02:04:24.185613 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.185593 2577 scope.go:117] "RemoveContainer" containerID="f2a2cecb3b8bd91c56f86ea58996467c9eb7885127c600af9e5f502ea610c9f6" Jul 14 02:04:24.192243 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.192221 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 02:04:24.192876 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.192863 2577 scope.go:117] "RemoveContainer" containerID="4fe1ef404db5348be10227a3b6a070c04df78107dfbed03d6ffb57664e6ae2f4" Jul 14 02:04:24.198656 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.198637 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 02:04:24.199869 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.199856 2577 scope.go:117] "RemoveContainer" containerID="fbf948905b0e5dc26d1c1243c707d05223790b702602a185e9651eea7f7cd448" Jul 14 02:04:24.206196 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.206180 2577 scope.go:117] "RemoveContainer" containerID="ec381b730242ff9a9ad97a2ebc7fdc85722214a72cd8c37c707dcfdaaf356e22" Jul 14 02:04:24.212731 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.212716 2577 scope.go:117] "RemoveContainer" containerID="ae0258c118bc2331427fd847b5918589273fdbb62e667414704778144862da67" Jul 14 02:04:24.219362 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.219344 2577 scope.go:117] "RemoveContainer" containerID="68122e3957ab11875bb01e88e61318ae5c69697610e03c5341a7118dd931be2a" Jul 14 02:04:24.219618 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:04:24.219598 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68122e3957ab11875bb01e88e61318ae5c69697610e03c5341a7118dd931be2a\": container with ID starting with 68122e3957ab11875bb01e88e61318ae5c69697610e03c5341a7118dd931be2a not found: ID does not exist" containerID="68122e3957ab11875bb01e88e61318ae5c69697610e03c5341a7118dd931be2a" Jul 14 02:04:24.219686 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.219630 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68122e3957ab11875bb01e88e61318ae5c69697610e03c5341a7118dd931be2a"} err="failed to get container status \"68122e3957ab11875bb01e88e61318ae5c69697610e03c5341a7118dd931be2a\": rpc error: code = NotFound desc = could not find container \"68122e3957ab11875bb01e88e61318ae5c69697610e03c5341a7118dd931be2a\": container with ID starting with 68122e3957ab11875bb01e88e61318ae5c69697610e03c5341a7118dd931be2a not found: ID does not exist" Jul 14 02:04:24.219686 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.219657 2577 scope.go:117] "RemoveContainer" containerID="14da291faf52a6dccc595ebb36b0ec6aac985cf9272545cd874399fe7be1c65f" Jul 14 02:04:24.219895 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:04:24.219879 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14da291faf52a6dccc595ebb36b0ec6aac985cf9272545cd874399fe7be1c65f\": container with ID starting with 14da291faf52a6dccc595ebb36b0ec6aac985cf9272545cd874399fe7be1c65f not found: ID does not exist" containerID="14da291faf52a6dccc595ebb36b0ec6aac985cf9272545cd874399fe7be1c65f" Jul 14 02:04:24.219932 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.219900 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14da291faf52a6dccc595ebb36b0ec6aac985cf9272545cd874399fe7be1c65f"} err="failed to get container status \"14da291faf52a6dccc595ebb36b0ec6aac985cf9272545cd874399fe7be1c65f\": rpc error: code = NotFound desc = could not find container \"14da291faf52a6dccc595ebb36b0ec6aac985cf9272545cd874399fe7be1c65f\": container with ID starting with 14da291faf52a6dccc595ebb36b0ec6aac985cf9272545cd874399fe7be1c65f not found: ID does not exist" Jul 14 02:04:24.219932 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.219917 2577 scope.go:117] "RemoveContainer" containerID="f2a2cecb3b8bd91c56f86ea58996467c9eb7885127c600af9e5f502ea610c9f6" Jul 14 02:04:24.220131 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:04:24.220112 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2a2cecb3b8bd91c56f86ea58996467c9eb7885127c600af9e5f502ea610c9f6\": container with ID starting with f2a2cecb3b8bd91c56f86ea58996467c9eb7885127c600af9e5f502ea610c9f6 not found: ID does not exist" containerID="f2a2cecb3b8bd91c56f86ea58996467c9eb7885127c600af9e5f502ea610c9f6" Jul 14 02:04:24.220196 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.220139 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2a2cecb3b8bd91c56f86ea58996467c9eb7885127c600af9e5f502ea610c9f6"} err="failed to get container status \"f2a2cecb3b8bd91c56f86ea58996467c9eb7885127c600af9e5f502ea610c9f6\": rpc error: code = NotFound desc = could not find container \"f2a2cecb3b8bd91c56f86ea58996467c9eb7885127c600af9e5f502ea610c9f6\": container with ID starting with f2a2cecb3b8bd91c56f86ea58996467c9eb7885127c600af9e5f502ea610c9f6 not found: ID does not exist" Jul 14 02:04:24.220196 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.220158 2577 scope.go:117] "RemoveContainer" containerID="4fe1ef404db5348be10227a3b6a070c04df78107dfbed03d6ffb57664e6ae2f4" Jul 14 02:04:24.220415 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:04:24.220399 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fe1ef404db5348be10227a3b6a070c04df78107dfbed03d6ffb57664e6ae2f4\": container with ID starting with 4fe1ef404db5348be10227a3b6a070c04df78107dfbed03d6ffb57664e6ae2f4 not found: ID does not exist" containerID="4fe1ef404db5348be10227a3b6a070c04df78107dfbed03d6ffb57664e6ae2f4" Jul 14 02:04:24.220453 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.220419 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fe1ef404db5348be10227a3b6a070c04df78107dfbed03d6ffb57664e6ae2f4"} err="failed to get container status \"4fe1ef404db5348be10227a3b6a070c04df78107dfbed03d6ffb57664e6ae2f4\": rpc error: code = NotFound desc = could not find container \"4fe1ef404db5348be10227a3b6a070c04df78107dfbed03d6ffb57664e6ae2f4\": container with ID starting with 4fe1ef404db5348be10227a3b6a070c04df78107dfbed03d6ffb57664e6ae2f4 not found: ID does not exist" Jul 14 02:04:24.220453 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.220433 2577 scope.go:117] "RemoveContainer" containerID="fbf948905b0e5dc26d1c1243c707d05223790b702602a185e9651eea7f7cd448" Jul 14 02:04:24.220675 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:04:24.220657 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbf948905b0e5dc26d1c1243c707d05223790b702602a185e9651eea7f7cd448\": container with ID starting with fbf948905b0e5dc26d1c1243c707d05223790b702602a185e9651eea7f7cd448 not found: ID does not exist" containerID="fbf948905b0e5dc26d1c1243c707d05223790b702602a185e9651eea7f7cd448" Jul 14 02:04:24.220746 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.220682 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbf948905b0e5dc26d1c1243c707d05223790b702602a185e9651eea7f7cd448"} err="failed to get container status \"fbf948905b0e5dc26d1c1243c707d05223790b702602a185e9651eea7f7cd448\": rpc error: code = NotFound desc = could not find container \"fbf948905b0e5dc26d1c1243c707d05223790b702602a185e9651eea7f7cd448\": container with ID starting with fbf948905b0e5dc26d1c1243c707d05223790b702602a185e9651eea7f7cd448 not found: ID does not exist" Jul 14 02:04:24.220746 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.220701 2577 scope.go:117] "RemoveContainer" containerID="ec381b730242ff9a9ad97a2ebc7fdc85722214a72cd8c37c707dcfdaaf356e22" Jul 14 02:04:24.220951 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:04:24.220934 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec381b730242ff9a9ad97a2ebc7fdc85722214a72cd8c37c707dcfdaaf356e22\": container with ID starting with ec381b730242ff9a9ad97a2ebc7fdc85722214a72cd8c37c707dcfdaaf356e22 not found: ID does not exist" containerID="ec381b730242ff9a9ad97a2ebc7fdc85722214a72cd8c37c707dcfdaaf356e22" Jul 14 02:04:24.220987 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.220955 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec381b730242ff9a9ad97a2ebc7fdc85722214a72cd8c37c707dcfdaaf356e22"} err="failed to get container status \"ec381b730242ff9a9ad97a2ebc7fdc85722214a72cd8c37c707dcfdaaf356e22\": rpc error: code = NotFound desc = could not find container \"ec381b730242ff9a9ad97a2ebc7fdc85722214a72cd8c37c707dcfdaaf356e22\": container with ID starting with ec381b730242ff9a9ad97a2ebc7fdc85722214a72cd8c37c707dcfdaaf356e22 not found: ID does not exist" Jul 14 02:04:24.220987 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.220971 2577 scope.go:117] "RemoveContainer" containerID="ae0258c118bc2331427fd847b5918589273fdbb62e667414704778144862da67" Jul 14 02:04:24.221177 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:04:24.221163 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae0258c118bc2331427fd847b5918589273fdbb62e667414704778144862da67\": container with ID starting with ae0258c118bc2331427fd847b5918589273fdbb62e667414704778144862da67 not found: ID does not exist" containerID="ae0258c118bc2331427fd847b5918589273fdbb62e667414704778144862da67" Jul 14 02:04:24.221220 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.221180 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae0258c118bc2331427fd847b5918589273fdbb62e667414704778144862da67"} err="failed to get container status \"ae0258c118bc2331427fd847b5918589273fdbb62e667414704778144862da67\": rpc error: code = NotFound desc = could not find container \"ae0258c118bc2331427fd847b5918589273fdbb62e667414704778144862da67\": container with ID starting with ae0258c118bc2331427fd847b5918589273fdbb62e667414704778144862da67 not found: ID does not exist" Jul 14 02:04:24.221220 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.221191 2577 scope.go:117] "RemoveContainer" containerID="68122e3957ab11875bb01e88e61318ae5c69697610e03c5341a7118dd931be2a" Jul 14 02:04:24.221478 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.221449 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68122e3957ab11875bb01e88e61318ae5c69697610e03c5341a7118dd931be2a"} err="failed to get container status \"68122e3957ab11875bb01e88e61318ae5c69697610e03c5341a7118dd931be2a\": rpc error: code = NotFound desc = could not find container \"68122e3957ab11875bb01e88e61318ae5c69697610e03c5341a7118dd931be2a\": container with ID starting with 68122e3957ab11875bb01e88e61318ae5c69697610e03c5341a7118dd931be2a not found: ID does not exist" Jul 14 02:04:24.221767 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.221483 2577 scope.go:117] "RemoveContainer" containerID="14da291faf52a6dccc595ebb36b0ec6aac985cf9272545cd874399fe7be1c65f" Jul 14 02:04:24.221871 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.221846 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14da291faf52a6dccc595ebb36b0ec6aac985cf9272545cd874399fe7be1c65f"} err="failed to get container status \"14da291faf52a6dccc595ebb36b0ec6aac985cf9272545cd874399fe7be1c65f\": rpc error: code = NotFound desc = could not find container \"14da291faf52a6dccc595ebb36b0ec6aac985cf9272545cd874399fe7be1c65f\": container with ID starting with 14da291faf52a6dccc595ebb36b0ec6aac985cf9272545cd874399fe7be1c65f not found: ID does not exist" Jul 14 02:04:24.221947 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.221874 2577 scope.go:117] "RemoveContainer" containerID="f2a2cecb3b8bd91c56f86ea58996467c9eb7885127c600af9e5f502ea610c9f6" Jul 14 02:04:24.222152 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.222129 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2a2cecb3b8bd91c56f86ea58996467c9eb7885127c600af9e5f502ea610c9f6"} err="failed to get container status \"f2a2cecb3b8bd91c56f86ea58996467c9eb7885127c600af9e5f502ea610c9f6\": rpc error: code = NotFound desc = could not find container \"f2a2cecb3b8bd91c56f86ea58996467c9eb7885127c600af9e5f502ea610c9f6\": container with ID starting with f2a2cecb3b8bd91c56f86ea58996467c9eb7885127c600af9e5f502ea610c9f6 not found: ID does not exist" Jul 14 02:04:24.222240 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.222154 2577 scope.go:117] "RemoveContainer" containerID="4fe1ef404db5348be10227a3b6a070c04df78107dfbed03d6ffb57664e6ae2f4" Jul 14 02:04:24.222466 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.222439 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fe1ef404db5348be10227a3b6a070c04df78107dfbed03d6ffb57664e6ae2f4"} err="failed to get container status \"4fe1ef404db5348be10227a3b6a070c04df78107dfbed03d6ffb57664e6ae2f4\": rpc error: code = NotFound desc = could not find container \"4fe1ef404db5348be10227a3b6a070c04df78107dfbed03d6ffb57664e6ae2f4\": container with ID starting with 4fe1ef404db5348be10227a3b6a070c04df78107dfbed03d6ffb57664e6ae2f4 not found: ID does not exist" Jul 14 02:04:24.222535 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.222469 2577 scope.go:117] "RemoveContainer" containerID="fbf948905b0e5dc26d1c1243c707d05223790b702602a185e9651eea7f7cd448" Jul 14 02:04:24.222740 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.222717 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbf948905b0e5dc26d1c1243c707d05223790b702602a185e9651eea7f7cd448"} err="failed to get container status \"fbf948905b0e5dc26d1c1243c707d05223790b702602a185e9651eea7f7cd448\": rpc error: code = NotFound desc = could not find container \"fbf948905b0e5dc26d1c1243c707d05223790b702602a185e9651eea7f7cd448\": container with ID starting with fbf948905b0e5dc26d1c1243c707d05223790b702602a185e9651eea7f7cd448 not found: ID does not exist" Jul 14 02:04:24.222740 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.222740 2577 scope.go:117] "RemoveContainer" containerID="ec381b730242ff9a9ad97a2ebc7fdc85722214a72cd8c37c707dcfdaaf356e22" Jul 14 02:04:24.222974 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.222958 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec381b730242ff9a9ad97a2ebc7fdc85722214a72cd8c37c707dcfdaaf356e22"} err="failed to get container status \"ec381b730242ff9a9ad97a2ebc7fdc85722214a72cd8c37c707dcfdaaf356e22\": rpc error: code = NotFound desc = could not find container \"ec381b730242ff9a9ad97a2ebc7fdc85722214a72cd8c37c707dcfdaaf356e22\": container with ID starting with ec381b730242ff9a9ad97a2ebc7fdc85722214a72cd8c37c707dcfdaaf356e22 not found: ID does not exist" Jul 14 02:04:24.222974 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.222974 2577 scope.go:117] "RemoveContainer" containerID="ae0258c118bc2331427fd847b5918589273fdbb62e667414704778144862da67" Jul 14 02:04:24.223167 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.223149 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae0258c118bc2331427fd847b5918589273fdbb62e667414704778144862da67"} err="failed to get container status \"ae0258c118bc2331427fd847b5918589273fdbb62e667414704778144862da67\": rpc error: code = NotFound desc = could not find container \"ae0258c118bc2331427fd847b5918589273fdbb62e667414704778144862da67\": container with ID starting with ae0258c118bc2331427fd847b5918589273fdbb62e667414704778144862da67 not found: ID does not exist" Jul 14 02:04:24.223167 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.223166 2577 scope.go:117] "RemoveContainer" containerID="68122e3957ab11875bb01e88e61318ae5c69697610e03c5341a7118dd931be2a" Jul 14 02:04:24.223431 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.223414 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68122e3957ab11875bb01e88e61318ae5c69697610e03c5341a7118dd931be2a"} err="failed to get container status \"68122e3957ab11875bb01e88e61318ae5c69697610e03c5341a7118dd931be2a\": rpc error: code = NotFound desc = could not find container \"68122e3957ab11875bb01e88e61318ae5c69697610e03c5341a7118dd931be2a\": container with ID starting with 68122e3957ab11875bb01e88e61318ae5c69697610e03c5341a7118dd931be2a not found: ID does not exist" Jul 14 02:04:24.223431 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.223425 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 02:04:24.223507 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.223432 2577 scope.go:117] "RemoveContainer" containerID="14da291faf52a6dccc595ebb36b0ec6aac985cf9272545cd874399fe7be1c65f" Jul 14 02:04:24.223722 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.223709 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2ab7b4e-fa36-44f3-86e6-151a58a5631b" containerName="registry" Jul 14 02:04:24.223766 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.223728 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2ab7b4e-fa36-44f3-86e6-151a58a5631b" containerName="registry" Jul 14 02:04:24.223766 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.223738 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a15093fc-3e71-4af0-bb06-fd31cf42798c" containerName="thanos-sidecar" Jul 14 02:04:24.223766 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.223744 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a15093fc-3e71-4af0-bb06-fd31cf42798c" containerName="thanos-sidecar" Jul 14 02:04:24.223766 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.223753 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a15093fc-3e71-4af0-bb06-fd31cf42798c" containerName="kube-rbac-proxy-thanos" Jul 14 02:04:24.223766 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.223760 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a15093fc-3e71-4af0-bb06-fd31cf42798c" containerName="kube-rbac-proxy-thanos" Jul 14 02:04:24.223925 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.223770 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a15093fc-3e71-4af0-bb06-fd31cf42798c" containerName="init-config-reloader" Jul 14 02:04:24.223925 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.223776 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a15093fc-3e71-4af0-bb06-fd31cf42798c" containerName="init-config-reloader" Jul 14 02:04:24.223925 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.223786 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a15093fc-3e71-4af0-bb06-fd31cf42798c" containerName="prometheus" Jul 14 02:04:24.223925 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.223703 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14da291faf52a6dccc595ebb36b0ec6aac985cf9272545cd874399fe7be1c65f"} err="failed to get container status \"14da291faf52a6dccc595ebb36b0ec6aac985cf9272545cd874399fe7be1c65f\": rpc error: code = NotFound desc = could not find container \"14da291faf52a6dccc595ebb36b0ec6aac985cf9272545cd874399fe7be1c65f\": container with ID starting with 14da291faf52a6dccc595ebb36b0ec6aac985cf9272545cd874399fe7be1c65f not found: ID does not exist" Jul 14 02:04:24.223925 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.223799 2577 scope.go:117] "RemoveContainer" containerID="f2a2cecb3b8bd91c56f86ea58996467c9eb7885127c600af9e5f502ea610c9f6" Jul 14 02:04:24.223925 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.223792 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a15093fc-3e71-4af0-bb06-fd31cf42798c" containerName="prometheus" Jul 14 02:04:24.223925 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.223846 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a15093fc-3e71-4af0-bb06-fd31cf42798c" containerName="kube-rbac-proxy-web" Jul 14 02:04:24.223925 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.223855 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a15093fc-3e71-4af0-bb06-fd31cf42798c" containerName="kube-rbac-proxy-web" Jul 14 02:04:24.223925 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.223869 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a15093fc-3e71-4af0-bb06-fd31cf42798c" containerName="config-reloader" Jul 14 02:04:24.223925 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.223875 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a15093fc-3e71-4af0-bb06-fd31cf42798c" containerName="config-reloader" Jul 14 02:04:24.223925 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.223884 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a15093fc-3e71-4af0-bb06-fd31cf42798c" containerName="kube-rbac-proxy" Jul 14 02:04:24.223925 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.223889 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a15093fc-3e71-4af0-bb06-fd31cf42798c" containerName="kube-rbac-proxy" Jul 14 02:04:24.224302 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.223964 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a15093fc-3e71-4af0-bb06-fd31cf42798c" containerName="kube-rbac-proxy" Jul 14 02:04:24.224302 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.223976 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a15093fc-3e71-4af0-bb06-fd31cf42798c" containerName="kube-rbac-proxy-thanos" Jul 14 02:04:24.224302 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.223985 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2ab7b4e-fa36-44f3-86e6-151a58a5631b" containerName="registry" Jul 14 02:04:24.224302 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.223995 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a15093fc-3e71-4af0-bb06-fd31cf42798c" containerName="config-reloader" Jul 14 02:04:24.224302 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.223993 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2a2cecb3b8bd91c56f86ea58996467c9eb7885127c600af9e5f502ea610c9f6"} err="failed to get container status \"f2a2cecb3b8bd91c56f86ea58996467c9eb7885127c600af9e5f502ea610c9f6\": rpc error: code = NotFound desc = could not find container \"f2a2cecb3b8bd91c56f86ea58996467c9eb7885127c600af9e5f502ea610c9f6\": container with ID starting with f2a2cecb3b8bd91c56f86ea58996467c9eb7885127c600af9e5f502ea610c9f6 not found: ID does not exist" Jul 14 02:04:24.224302 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.224012 2577 scope.go:117] "RemoveContainer" containerID="4fe1ef404db5348be10227a3b6a070c04df78107dfbed03d6ffb57664e6ae2f4" Jul 14 02:04:24.224302 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.224003 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a15093fc-3e71-4af0-bb06-fd31cf42798c" containerName="thanos-sidecar" Jul 14 02:04:24.224302 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.224054 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a15093fc-3e71-4af0-bb06-fd31cf42798c" containerName="kube-rbac-proxy-web" Jul 14 02:04:24.224302 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.224064 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a15093fc-3e71-4af0-bb06-fd31cf42798c" containerName="prometheus" Jul 14 02:04:24.224302 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.224240 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fe1ef404db5348be10227a3b6a070c04df78107dfbed03d6ffb57664e6ae2f4"} err="failed to get container status \"4fe1ef404db5348be10227a3b6a070c04df78107dfbed03d6ffb57664e6ae2f4\": rpc error: code = NotFound desc = could not find container \"4fe1ef404db5348be10227a3b6a070c04df78107dfbed03d6ffb57664e6ae2f4\": container with ID starting with 4fe1ef404db5348be10227a3b6a070c04df78107dfbed03d6ffb57664e6ae2f4 not found: ID does not exist" Jul 14 02:04:24.224302 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.224255 2577 scope.go:117] "RemoveContainer" containerID="fbf948905b0e5dc26d1c1243c707d05223790b702602a185e9651eea7f7cd448" Jul 14 02:04:24.224655 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.224491 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbf948905b0e5dc26d1c1243c707d05223790b702602a185e9651eea7f7cd448"} err="failed to get container status \"fbf948905b0e5dc26d1c1243c707d05223790b702602a185e9651eea7f7cd448\": rpc error: code = NotFound desc = could not find container \"fbf948905b0e5dc26d1c1243c707d05223790b702602a185e9651eea7f7cd448\": container with ID starting with fbf948905b0e5dc26d1c1243c707d05223790b702602a185e9651eea7f7cd448 not found: ID does not exist" Jul 14 02:04:24.224655 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.224514 2577 scope.go:117] "RemoveContainer" containerID="ec381b730242ff9a9ad97a2ebc7fdc85722214a72cd8c37c707dcfdaaf356e22" Jul 14 02:04:24.224737 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.224718 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec381b730242ff9a9ad97a2ebc7fdc85722214a72cd8c37c707dcfdaaf356e22"} err="failed to get container status \"ec381b730242ff9a9ad97a2ebc7fdc85722214a72cd8c37c707dcfdaaf356e22\": rpc error: code = NotFound desc = could not find container \"ec381b730242ff9a9ad97a2ebc7fdc85722214a72cd8c37c707dcfdaaf356e22\": container with ID starting with ec381b730242ff9a9ad97a2ebc7fdc85722214a72cd8c37c707dcfdaaf356e22 not found: ID does not exist" Jul 14 02:04:24.224737 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.224733 2577 scope.go:117] "RemoveContainer" containerID="ae0258c118bc2331427fd847b5918589273fdbb62e667414704778144862da67" Jul 14 02:04:24.224947 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.224929 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae0258c118bc2331427fd847b5918589273fdbb62e667414704778144862da67"} err="failed to get container status \"ae0258c118bc2331427fd847b5918589273fdbb62e667414704778144862da67\": rpc error: code = NotFound desc = could not find container \"ae0258c118bc2331427fd847b5918589273fdbb62e667414704778144862da67\": container with ID starting with ae0258c118bc2331427fd847b5918589273fdbb62e667414704778144862da67 not found: ID does not exist" Jul 14 02:04:24.225016 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.224949 2577 scope.go:117] "RemoveContainer" containerID="68122e3957ab11875bb01e88e61318ae5c69697610e03c5341a7118dd931be2a" Jul 14 02:04:24.225175 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.225158 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68122e3957ab11875bb01e88e61318ae5c69697610e03c5341a7118dd931be2a"} err="failed to get container status \"68122e3957ab11875bb01e88e61318ae5c69697610e03c5341a7118dd931be2a\": rpc error: code = NotFound desc = could not find container \"68122e3957ab11875bb01e88e61318ae5c69697610e03c5341a7118dd931be2a\": container with ID starting with 68122e3957ab11875bb01e88e61318ae5c69697610e03c5341a7118dd931be2a not found: ID does not exist" Jul 14 02:04:24.225236 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.225176 2577 scope.go:117] "RemoveContainer" containerID="14da291faf52a6dccc595ebb36b0ec6aac985cf9272545cd874399fe7be1c65f" Jul 14 02:04:24.225410 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.225394 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14da291faf52a6dccc595ebb36b0ec6aac985cf9272545cd874399fe7be1c65f"} err="failed to get container status \"14da291faf52a6dccc595ebb36b0ec6aac985cf9272545cd874399fe7be1c65f\": rpc error: code = NotFound desc = could not find container \"14da291faf52a6dccc595ebb36b0ec6aac985cf9272545cd874399fe7be1c65f\": container with ID starting with 14da291faf52a6dccc595ebb36b0ec6aac985cf9272545cd874399fe7be1c65f not found: ID does not exist" Jul 14 02:04:24.225483 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.225411 2577 scope.go:117] "RemoveContainer" containerID="f2a2cecb3b8bd91c56f86ea58996467c9eb7885127c600af9e5f502ea610c9f6" Jul 14 02:04:24.225630 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.225607 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2a2cecb3b8bd91c56f86ea58996467c9eb7885127c600af9e5f502ea610c9f6"} err="failed to get container status \"f2a2cecb3b8bd91c56f86ea58996467c9eb7885127c600af9e5f502ea610c9f6\": rpc error: code = NotFound desc = could not find container \"f2a2cecb3b8bd91c56f86ea58996467c9eb7885127c600af9e5f502ea610c9f6\": container with ID starting with f2a2cecb3b8bd91c56f86ea58996467c9eb7885127c600af9e5f502ea610c9f6 not found: ID does not exist" Jul 14 02:04:24.225630 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.225628 2577 scope.go:117] "RemoveContainer" containerID="4fe1ef404db5348be10227a3b6a070c04df78107dfbed03d6ffb57664e6ae2f4" Jul 14 02:04:24.225851 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.225830 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fe1ef404db5348be10227a3b6a070c04df78107dfbed03d6ffb57664e6ae2f4"} err="failed to get container status \"4fe1ef404db5348be10227a3b6a070c04df78107dfbed03d6ffb57664e6ae2f4\": rpc error: code = NotFound desc = could not find container \"4fe1ef404db5348be10227a3b6a070c04df78107dfbed03d6ffb57664e6ae2f4\": container with ID starting with 4fe1ef404db5348be10227a3b6a070c04df78107dfbed03d6ffb57664e6ae2f4 not found: ID does not exist" Jul 14 02:04:24.225899 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.225854 2577 scope.go:117] "RemoveContainer" containerID="fbf948905b0e5dc26d1c1243c707d05223790b702602a185e9651eea7f7cd448" Jul 14 02:04:24.226103 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.226086 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbf948905b0e5dc26d1c1243c707d05223790b702602a185e9651eea7f7cd448"} err="failed to get container status \"fbf948905b0e5dc26d1c1243c707d05223790b702602a185e9651eea7f7cd448\": rpc error: code = NotFound desc = could not find container \"fbf948905b0e5dc26d1c1243c707d05223790b702602a185e9651eea7f7cd448\": container with ID starting with fbf948905b0e5dc26d1c1243c707d05223790b702602a185e9651eea7f7cd448 not found: ID does not exist" Jul 14 02:04:24.226143 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.226103 2577 scope.go:117] "RemoveContainer" containerID="ec381b730242ff9a9ad97a2ebc7fdc85722214a72cd8c37c707dcfdaaf356e22" Jul 14 02:04:24.226312 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.226296 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec381b730242ff9a9ad97a2ebc7fdc85722214a72cd8c37c707dcfdaaf356e22"} err="failed to get container status \"ec381b730242ff9a9ad97a2ebc7fdc85722214a72cd8c37c707dcfdaaf356e22\": rpc error: code = NotFound desc = could not find container \"ec381b730242ff9a9ad97a2ebc7fdc85722214a72cd8c37c707dcfdaaf356e22\": container with ID starting with ec381b730242ff9a9ad97a2ebc7fdc85722214a72cd8c37c707dcfdaaf356e22 not found: ID does not exist" Jul 14 02:04:24.226312 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.226311 2577 scope.go:117] "RemoveContainer" containerID="ae0258c118bc2331427fd847b5918589273fdbb62e667414704778144862da67" Jul 14 02:04:24.226501 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.226482 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae0258c118bc2331427fd847b5918589273fdbb62e667414704778144862da67"} err="failed to get container status \"ae0258c118bc2331427fd847b5918589273fdbb62e667414704778144862da67\": rpc error: code = NotFound desc = could not find container \"ae0258c118bc2331427fd847b5918589273fdbb62e667414704778144862da67\": container with ID starting with ae0258c118bc2331427fd847b5918589273fdbb62e667414704778144862da67 not found: ID does not exist" Jul 14 02:04:24.226546 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.226501 2577 scope.go:117] "RemoveContainer" containerID="68122e3957ab11875bb01e88e61318ae5c69697610e03c5341a7118dd931be2a" Jul 14 02:04:24.226713 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.226697 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68122e3957ab11875bb01e88e61318ae5c69697610e03c5341a7118dd931be2a"} err="failed to get container status \"68122e3957ab11875bb01e88e61318ae5c69697610e03c5341a7118dd931be2a\": rpc error: code = NotFound desc = could not find container \"68122e3957ab11875bb01e88e61318ae5c69697610e03c5341a7118dd931be2a\": container with ID starting with 68122e3957ab11875bb01e88e61318ae5c69697610e03c5341a7118dd931be2a not found: ID does not exist" Jul 14 02:04:24.226713 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.226712 2577 scope.go:117] "RemoveContainer" containerID="14da291faf52a6dccc595ebb36b0ec6aac985cf9272545cd874399fe7be1c65f" Jul 14 02:04:24.226914 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.226891 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14da291faf52a6dccc595ebb36b0ec6aac985cf9272545cd874399fe7be1c65f"} err="failed to get container status \"14da291faf52a6dccc595ebb36b0ec6aac985cf9272545cd874399fe7be1c65f\": rpc error: code = NotFound desc = could not find container \"14da291faf52a6dccc595ebb36b0ec6aac985cf9272545cd874399fe7be1c65f\": container with ID starting with 14da291faf52a6dccc595ebb36b0ec6aac985cf9272545cd874399fe7be1c65f not found: ID does not exist" Jul 14 02:04:24.226955 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.226917 2577 scope.go:117] "RemoveContainer" containerID="f2a2cecb3b8bd91c56f86ea58996467c9eb7885127c600af9e5f502ea610c9f6" Jul 14 02:04:24.227103 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.227088 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2a2cecb3b8bd91c56f86ea58996467c9eb7885127c600af9e5f502ea610c9f6"} err="failed to get container status \"f2a2cecb3b8bd91c56f86ea58996467c9eb7885127c600af9e5f502ea610c9f6\": rpc error: code = NotFound desc = could not find container \"f2a2cecb3b8bd91c56f86ea58996467c9eb7885127c600af9e5f502ea610c9f6\": container with ID starting with f2a2cecb3b8bd91c56f86ea58996467c9eb7885127c600af9e5f502ea610c9f6 not found: ID does not exist" Jul 14 02:04:24.227144 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.227104 2577 scope.go:117] "RemoveContainer" containerID="4fe1ef404db5348be10227a3b6a070c04df78107dfbed03d6ffb57664e6ae2f4" Jul 14 02:04:24.227300 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.227264 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fe1ef404db5348be10227a3b6a070c04df78107dfbed03d6ffb57664e6ae2f4"} err="failed to get container status \"4fe1ef404db5348be10227a3b6a070c04df78107dfbed03d6ffb57664e6ae2f4\": rpc error: code = NotFound desc = could not find container \"4fe1ef404db5348be10227a3b6a070c04df78107dfbed03d6ffb57664e6ae2f4\": container with ID starting with 4fe1ef404db5348be10227a3b6a070c04df78107dfbed03d6ffb57664e6ae2f4 not found: ID does not exist" Jul 14 02:04:24.227346 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.227301 2577 scope.go:117] "RemoveContainer" containerID="fbf948905b0e5dc26d1c1243c707d05223790b702602a185e9651eea7f7cd448" Jul 14 02:04:24.227469 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.227453 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbf948905b0e5dc26d1c1243c707d05223790b702602a185e9651eea7f7cd448"} err="failed to get container status \"fbf948905b0e5dc26d1c1243c707d05223790b702602a185e9651eea7f7cd448\": rpc error: code = NotFound desc = could not find container \"fbf948905b0e5dc26d1c1243c707d05223790b702602a185e9651eea7f7cd448\": container with ID starting with fbf948905b0e5dc26d1c1243c707d05223790b702602a185e9651eea7f7cd448 not found: ID does not exist" Jul 14 02:04:24.227504 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.227469 2577 scope.go:117] "RemoveContainer" containerID="ec381b730242ff9a9ad97a2ebc7fdc85722214a72cd8c37c707dcfdaaf356e22" Jul 14 02:04:24.227645 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.227630 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec381b730242ff9a9ad97a2ebc7fdc85722214a72cd8c37c707dcfdaaf356e22"} err="failed to get container status \"ec381b730242ff9a9ad97a2ebc7fdc85722214a72cd8c37c707dcfdaaf356e22\": rpc error: code = NotFound desc = could not find container \"ec381b730242ff9a9ad97a2ebc7fdc85722214a72cd8c37c707dcfdaaf356e22\": container with ID starting with ec381b730242ff9a9ad97a2ebc7fdc85722214a72cd8c37c707dcfdaaf356e22 not found: ID does not exist" Jul 14 02:04:24.227682 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.227645 2577 scope.go:117] "RemoveContainer" containerID="ae0258c118bc2331427fd847b5918589273fdbb62e667414704778144862da67" Jul 14 02:04:24.227881 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.227867 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae0258c118bc2331427fd847b5918589273fdbb62e667414704778144862da67"} err="failed to get container status \"ae0258c118bc2331427fd847b5918589273fdbb62e667414704778144862da67\": rpc error: code = NotFound desc = could not find container \"ae0258c118bc2331427fd847b5918589273fdbb62e667414704778144862da67\": container with ID starting with ae0258c118bc2331427fd847b5918589273fdbb62e667414704778144862da67 not found: ID does not exist" Jul 14 02:04:24.227919 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.227883 2577 scope.go:117] "RemoveContainer" containerID="68122e3957ab11875bb01e88e61318ae5c69697610e03c5341a7118dd931be2a" Jul 14 02:04:24.228074 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.228058 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68122e3957ab11875bb01e88e61318ae5c69697610e03c5341a7118dd931be2a"} err="failed to get container status \"68122e3957ab11875bb01e88e61318ae5c69697610e03c5341a7118dd931be2a\": rpc error: code = NotFound desc = could not find container \"68122e3957ab11875bb01e88e61318ae5c69697610e03c5341a7118dd931be2a\": container with ID starting with 68122e3957ab11875bb01e88e61318ae5c69697610e03c5341a7118dd931be2a not found: ID does not exist" Jul 14 02:04:24.228074 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.228073 2577 scope.go:117] "RemoveContainer" containerID="14da291faf52a6dccc595ebb36b0ec6aac985cf9272545cd874399fe7be1c65f" Jul 14 02:04:24.228256 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.228240 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14da291faf52a6dccc595ebb36b0ec6aac985cf9272545cd874399fe7be1c65f"} err="failed to get container status \"14da291faf52a6dccc595ebb36b0ec6aac985cf9272545cd874399fe7be1c65f\": rpc error: code = NotFound desc = could not find container \"14da291faf52a6dccc595ebb36b0ec6aac985cf9272545cd874399fe7be1c65f\": container with ID starting with 14da291faf52a6dccc595ebb36b0ec6aac985cf9272545cd874399fe7be1c65f not found: ID does not exist" Jul 14 02:04:24.228256 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.228255 2577 scope.go:117] "RemoveContainer" containerID="f2a2cecb3b8bd91c56f86ea58996467c9eb7885127c600af9e5f502ea610c9f6" Jul 14 02:04:24.228437 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.228423 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2a2cecb3b8bd91c56f86ea58996467c9eb7885127c600af9e5f502ea610c9f6"} err="failed to get container status \"f2a2cecb3b8bd91c56f86ea58996467c9eb7885127c600af9e5f502ea610c9f6\": rpc error: code = NotFound desc = could not find container \"f2a2cecb3b8bd91c56f86ea58996467c9eb7885127c600af9e5f502ea610c9f6\": container with ID starting with f2a2cecb3b8bd91c56f86ea58996467c9eb7885127c600af9e5f502ea610c9f6 not found: ID does not exist" Jul 14 02:04:24.228478 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.228437 2577 scope.go:117] "RemoveContainer" containerID="4fe1ef404db5348be10227a3b6a070c04df78107dfbed03d6ffb57664e6ae2f4" Jul 14 02:04:24.228622 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.228607 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fe1ef404db5348be10227a3b6a070c04df78107dfbed03d6ffb57664e6ae2f4"} err="failed to get container status \"4fe1ef404db5348be10227a3b6a070c04df78107dfbed03d6ffb57664e6ae2f4\": rpc error: code = NotFound desc = could not find container \"4fe1ef404db5348be10227a3b6a070c04df78107dfbed03d6ffb57664e6ae2f4\": container with ID starting with 4fe1ef404db5348be10227a3b6a070c04df78107dfbed03d6ffb57664e6ae2f4 not found: ID does not exist" Jul 14 02:04:24.228674 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.228622 2577 scope.go:117] "RemoveContainer" containerID="fbf948905b0e5dc26d1c1243c707d05223790b702602a185e9651eea7f7cd448" Jul 14 02:04:24.228845 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.228827 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbf948905b0e5dc26d1c1243c707d05223790b702602a185e9651eea7f7cd448"} err="failed to get container status \"fbf948905b0e5dc26d1c1243c707d05223790b702602a185e9651eea7f7cd448\": rpc error: code = NotFound desc = could not find container \"fbf948905b0e5dc26d1c1243c707d05223790b702602a185e9651eea7f7cd448\": container with ID starting with fbf948905b0e5dc26d1c1243c707d05223790b702602a185e9651eea7f7cd448 not found: ID does not exist" Jul 14 02:04:24.228898 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.228846 2577 scope.go:117] "RemoveContainer" containerID="ec381b730242ff9a9ad97a2ebc7fdc85722214a72cd8c37c707dcfdaaf356e22" Jul 14 02:04:24.229039 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.229021 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec381b730242ff9a9ad97a2ebc7fdc85722214a72cd8c37c707dcfdaaf356e22"} err="failed to get container status \"ec381b730242ff9a9ad97a2ebc7fdc85722214a72cd8c37c707dcfdaaf356e22\": rpc error: code = NotFound desc = could not find container \"ec381b730242ff9a9ad97a2ebc7fdc85722214a72cd8c37c707dcfdaaf356e22\": container with ID starting with ec381b730242ff9a9ad97a2ebc7fdc85722214a72cd8c37c707dcfdaaf356e22 not found: ID does not exist" Jul 14 02:04:24.229093 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.229039 2577 scope.go:117] "RemoveContainer" containerID="ae0258c118bc2331427fd847b5918589273fdbb62e667414704778144862da67" Jul 14 02:04:24.229231 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.229208 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae0258c118bc2331427fd847b5918589273fdbb62e667414704778144862da67"} err="failed to get container status \"ae0258c118bc2331427fd847b5918589273fdbb62e667414704778144862da67\": rpc error: code = NotFound desc = could not find container \"ae0258c118bc2331427fd847b5918589273fdbb62e667414704778144862da67\": container with ID starting with ae0258c118bc2331427fd847b5918589273fdbb62e667414704778144862da67 not found: ID does not exist" Jul 14 02:04:24.229419 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.229403 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.232252 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.232117 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 14 02:04:24.232252 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.232128 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 14 02:04:24.232252 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.232191 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 14 02:04:24.232252 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.232215 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 14 02:04:24.232252 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.232251 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 14 02:04:24.232588 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.232288 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 14 02:04:24.232588 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.232499 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 14 02:04:24.232588 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.232520 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 14 02:04:24.232588 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.232543 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 14 02:04:24.232588 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.232562 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-hp76r\"" Jul 14 02:04:24.232588 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.232579 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 14 02:04:24.233096 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.233080 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 14 02:04:24.233159 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.233145 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5rr5kl436dgm0\"" Jul 14 02:04:24.233216 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.233171 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 14 02:04:24.233216 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.233128 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 14 02:04:24.235616 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.235587 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 14 02:04:24.238196 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.238178 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 14 02:04:24.241805 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.241787 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 02:04:24.262866 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.262835 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a15093fc-3e71-4af0-bb06-fd31cf42798c" path="/var/lib/kubelet/pods/a15093fc-3e71-4af0-bb06-fd31cf42798c/volumes" Jul 14 02:04:24.359620 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.359582 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5ee55730-71d9-410e-9d34-6858e719419f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.359620 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.359620 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5ee55730-71d9-410e-9d34-6858e719419f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.360116 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.359640 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5ee55730-71d9-410e-9d34-6858e719419f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.360116 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.359658 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5ee55730-71d9-410e-9d34-6858e719419f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.360116 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.359723 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5ee55730-71d9-410e-9d34-6858e719419f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.360116 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.359755 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5ee55730-71d9-410e-9d34-6858e719419f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.360116 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.359777 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ee55730-71d9-410e-9d34-6858e719419f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.360116 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.359793 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ee55730-71d9-410e-9d34-6858e719419f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.360116 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.359811 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5ee55730-71d9-410e-9d34-6858e719419f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.360116 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.359904 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5ee55730-71d9-410e-9d34-6858e719419f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.360116 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.359942 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5ee55730-71d9-410e-9d34-6858e719419f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.360116 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.359965 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5ee55730-71d9-410e-9d34-6858e719419f-config-out\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.360116 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.359984 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5ee55730-71d9-410e-9d34-6858e719419f-web-config\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.360116 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.360006 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5ee55730-71d9-410e-9d34-6858e719419f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.360116 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.360026 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5ee55730-71d9-410e-9d34-6858e719419f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.360554 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.360046 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6rmg\" (UniqueName: \"kubernetes.io/projected/5ee55730-71d9-410e-9d34-6858e719419f-kube-api-access-b6rmg\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.360554 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.360079 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5ee55730-71d9-410e-9d34-6858e719419f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.360554 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.360149 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ee55730-71d9-410e-9d34-6858e719419f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.360554 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.360171 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ee55730-71d9-410e-9d34-6858e719419f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.360554 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.360189 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5ee55730-71d9-410e-9d34-6858e719419f-config\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.461127 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.461036 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ee55730-71d9-410e-9d34-6858e719419f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.461127 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.461079 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ee55730-71d9-410e-9d34-6858e719419f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.461127 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.461105 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5ee55730-71d9-410e-9d34-6858e719419f-config\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.461432 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.461143 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5ee55730-71d9-410e-9d34-6858e719419f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.461432 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.461173 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5ee55730-71d9-410e-9d34-6858e719419f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.461432 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.461196 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5ee55730-71d9-410e-9d34-6858e719419f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.461432 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.461221 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5ee55730-71d9-410e-9d34-6858e719419f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.461606 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.461264 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5ee55730-71d9-410e-9d34-6858e719419f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.461606 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.461600 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5ee55730-71d9-410e-9d34-6858e719419f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.461715 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.461628 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ee55730-71d9-410e-9d34-6858e719419f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.461715 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.461652 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ee55730-71d9-410e-9d34-6858e719419f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.461715 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.461710 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5ee55730-71d9-410e-9d34-6858e719419f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.461858 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.461776 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5ee55730-71d9-410e-9d34-6858e719419f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.461858 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.461806 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5ee55730-71d9-410e-9d34-6858e719419f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.461858 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.461829 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5ee55730-71d9-410e-9d34-6858e719419f-config-out\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.461858 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.461853 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5ee55730-71d9-410e-9d34-6858e719419f-web-config\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.462050 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.461879 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5ee55730-71d9-410e-9d34-6858e719419f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.462050 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.461903 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5ee55730-71d9-410e-9d34-6858e719419f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.462050 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.461927 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6rmg\" (UniqueName: \"kubernetes.io/projected/5ee55730-71d9-410e-9d34-6858e719419f-kube-api-access-b6rmg\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.462050 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.461947 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ee55730-71d9-410e-9d34-6858e719419f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.462050 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.461972 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5ee55730-71d9-410e-9d34-6858e719419f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.462720 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.462696 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5ee55730-71d9-410e-9d34-6858e719419f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.462880 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.462855 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ee55730-71d9-410e-9d34-6858e719419f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.462957 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.462893 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ee55730-71d9-410e-9d34-6858e719419f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.463331 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.463307 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5ee55730-71d9-410e-9d34-6858e719419f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.463507 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.463482 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ee55730-71d9-410e-9d34-6858e719419f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.464322 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.464299 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5ee55730-71d9-410e-9d34-6858e719419f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.464416 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.464339 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5ee55730-71d9-410e-9d34-6858e719419f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.464693 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.464669 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5ee55730-71d9-410e-9d34-6858e719419f-config\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.464693 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.464685 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5ee55730-71d9-410e-9d34-6858e719419f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.465500 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.465476 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5ee55730-71d9-410e-9d34-6858e719419f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.466071 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.466025 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5ee55730-71d9-410e-9d34-6858e719419f-config-out\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.466251 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.466233 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5ee55730-71d9-410e-9d34-6858e719419f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.466351 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.466232 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5ee55730-71d9-410e-9d34-6858e719419f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.466729 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.466707 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5ee55730-71d9-410e-9d34-6858e719419f-web-config\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.466844 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.466826 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5ee55730-71d9-410e-9d34-6858e719419f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.466907 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.466863 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5ee55730-71d9-410e-9d34-6858e719419f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.467542 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.467523 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5ee55730-71d9-410e-9d34-6858e719419f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.467966 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.467950 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5ee55730-71d9-410e-9d34-6858e719419f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.470394 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.470371 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6rmg\" (UniqueName: \"kubernetes.io/projected/5ee55730-71d9-410e-9d34-6858e719419f-kube-api-access-b6rmg\") pod \"prometheus-k8s-0\" (UID: \"5ee55730-71d9-410e-9d34-6858e719419f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.539435 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.539397 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:24.673376 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:24.673358 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 02:04:24.675255 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:04:24.675229 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ee55730_71d9_410e_9d34_6858e719419f.slice/crio-a126840aa887d2778d00d2b75cf75aac89438821efee9c2615e7cb2cd42c0c04 WatchSource:0}: Error finding container a126840aa887d2778d00d2b75cf75aac89438821efee9c2615e7cb2cd42c0c04: Status 404 returned error can't find the container with id a126840aa887d2778d00d2b75cf75aac89438821efee9c2615e7cb2cd42c0c04 Jul 14 02:04:25.174149 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:25.174111 2577 generic.go:356] "Generic (PLEG): container finished" podID="5ee55730-71d9-410e-9d34-6858e719419f" containerID="57731a6a77519baa68b17bfe3170658355adf0eea3460bde74d611a53c2def26" exitCode=0 Jul 14 02:04:25.174334 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:25.174177 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5ee55730-71d9-410e-9d34-6858e719419f","Type":"ContainerDied","Data":"57731a6a77519baa68b17bfe3170658355adf0eea3460bde74d611a53c2def26"} Jul 14 02:04:25.174334 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:25.174245 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5ee55730-71d9-410e-9d34-6858e719419f","Type":"ContainerStarted","Data":"a126840aa887d2778d00d2b75cf75aac89438821efee9c2615e7cb2cd42c0c04"} Jul 14 02:04:26.180937 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:26.180901 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5ee55730-71d9-410e-9d34-6858e719419f","Type":"ContainerStarted","Data":"afe9d27437e31cca982fd6ea5746a0f854c7cf34e468a90747ea4cfc0cfb3a0e"} Jul 14 02:04:26.181350 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:26.180973 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5ee55730-71d9-410e-9d34-6858e719419f","Type":"ContainerStarted","Data":"1c0c55a2d7afa644a1dbf136431e9333046667a5ff6c80e1145cf9727a387e7f"} Jul 14 02:04:26.181350 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:26.181001 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5ee55730-71d9-410e-9d34-6858e719419f","Type":"ContainerStarted","Data":"05b22620158470846d62b8041da1312ea99f226ab05c55e5c60191556a1d4138"} Jul 14 02:04:26.181350 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:26.181037 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5ee55730-71d9-410e-9d34-6858e719419f","Type":"ContainerStarted","Data":"128e163aa71e534254b0941b9b45cc03d1f310332c17f419b40e21add2e38c14"} Jul 14 02:04:26.181350 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:26.181065 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5ee55730-71d9-410e-9d34-6858e719419f","Type":"ContainerStarted","Data":"2d794ab3ebd7245a8031973fa38b1de2bf22d1be22737e958da45544de390a03"} Jul 14 02:04:26.181350 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:26.181087 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5ee55730-71d9-410e-9d34-6858e719419f","Type":"ContainerStarted","Data":"f76aff226626fdfe92c86a706244c44bd7423dc85acf0090df46b0fcbe7003b8"} Jul 14 02:04:26.207848 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:26.207796 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.188417819 podStartE2EDuration="2.207778633s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.360814ms" isStatefulPod=false podCreationTimestamp="2026-07-14 02:04:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:04:26.206096511 +0000 UTC m=+168.360933159" watchObservedRunningTime="2026-07-14 02:04:26.207778633 +0000 UTC m=+168.362615280" Jul 14 02:04:27.996942 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:27.996911 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:04:28.097464 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:28.097437 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:04:28.197594 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:28.197560 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:04:28.747935 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:28.747895 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:04:28.800844 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:28.800817 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:04:29.540093 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:29.540060 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:04:39.071258 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:39.071225 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:04:39.176067 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:39.176023 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:04:39.229445 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:39.229413 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:04:57.993454 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:57.993421 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:04:58.095134 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:04:58.095102 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:05:18.117730 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:05:18.117694 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:05:24.540028 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:05:24.539996 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:05:24.555200 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:05:24.555174 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:05:25.415844 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:05:25.415820 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 02:05:33.100086 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:05:33.100053 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:05:33.152825 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:05:33.152789 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:05:44.994440 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:05:44.994390 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:05:45.046110 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:05:45.046078 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:06:01.251204 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:06:01.251116 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:06:01.351477 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:06:01.351448 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:06:08.713502 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:06:08.713467 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:06:08.766230 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:06:08.766201 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:06:19.497665 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:06:19.497629 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:06:19.546852 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:06:19.546824 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:06:19.600025 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:06:19.599998 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:06:29.691895 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:06:29.691863 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:06:29.789969 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:06:29.789932 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:06:38.162524 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:06:38.162493 2577 kubelet.go:1648] "Image garbage collection succeeded" Jul 14 02:07:01.162045 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:01.162010 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:07:02.525739 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:02.525705 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:07:04.782082 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:04.782041 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:07:04.880615 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:04.880581 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:07:06.655222 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:06.655192 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:07:06.755323 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:06.755292 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:07:06.931570 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:06.931477 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:07:07.027074 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:07.027048 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:07:09.113926 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:09.113896 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:07:09.523773 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:09.523702 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:07:11.759759 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:11.759733 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:07:11.820722 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:11.820695 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:07:15.800158 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:15.800126 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:07:15.901249 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:15.901214 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:07:16.926643 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:16.926603 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:07:23.580231 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:23.580196 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:07:23.678433 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:23.678398 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:07:25.598610 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:25.598576 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v88wb"] Jul 14 02:07:25.601722 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:25.601705 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v88wb" Jul 14 02:07:25.603648 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:25.603628 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 14 02:07:25.603760 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:25.603643 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 14 02:07:25.604138 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:25.604124 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-dbxh4\"" Jul 14 02:07:25.609118 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:25.609097 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v88wb"] Jul 14 02:07:25.753450 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:25.753415 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b990867-e76a-497f-ab31-2bd3e55b51c5-tmp\") pod \"openshift-lws-operator-fd8ccff4c-v88wb\" (UID: \"4b990867-e76a-497f-ab31-2bd3e55b51c5\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v88wb" Jul 14 02:07:25.753629 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:25.753472 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8j9ch\" (UniqueName: \"kubernetes.io/projected/4b990867-e76a-497f-ab31-2bd3e55b51c5-kube-api-access-8j9ch\") pod \"openshift-lws-operator-fd8ccff4c-v88wb\" (UID: \"4b990867-e76a-497f-ab31-2bd3e55b51c5\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v88wb" Jul 14 02:07:25.854967 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:25.854875 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b990867-e76a-497f-ab31-2bd3e55b51c5-tmp\") pod \"openshift-lws-operator-fd8ccff4c-v88wb\" (UID: \"4b990867-e76a-497f-ab31-2bd3e55b51c5\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v88wb" Jul 14 02:07:25.854967 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:25.854942 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8j9ch\" (UniqueName: \"kubernetes.io/projected/4b990867-e76a-497f-ab31-2bd3e55b51c5-kube-api-access-8j9ch\") pod \"openshift-lws-operator-fd8ccff4c-v88wb\" (UID: \"4b990867-e76a-497f-ab31-2bd3e55b51c5\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v88wb" Jul 14 02:07:25.855311 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:25.855262 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b990867-e76a-497f-ab31-2bd3e55b51c5-tmp\") pod \"openshift-lws-operator-fd8ccff4c-v88wb\" (UID: \"4b990867-e76a-497f-ab31-2bd3e55b51c5\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v88wb" Jul 14 02:07:25.862560 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:25.862543 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8j9ch\" (UniqueName: \"kubernetes.io/projected/4b990867-e76a-497f-ab31-2bd3e55b51c5-kube-api-access-8j9ch\") pod \"openshift-lws-operator-fd8ccff4c-v88wb\" (UID: \"4b990867-e76a-497f-ab31-2bd3e55b51c5\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v88wb" Jul 14 02:07:25.918442 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:25.918403 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v88wb" Jul 14 02:07:26.050144 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:26.050117 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v88wb"] Jul 14 02:07:26.052967 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:07:26.052935 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b990867_e76a_497f_ab31_2bd3e55b51c5.slice/crio-f787943f05edd9a0990029a4ed9b7ec9f748723c35c27e8b8c8117afa312fa10 WatchSource:0}: Error finding container f787943f05edd9a0990029a4ed9b7ec9f748723c35c27e8b8c8117afa312fa10: Status 404 returned error can't find the container with id f787943f05edd9a0990029a4ed9b7ec9f748723c35c27e8b8c8117afa312fa10 Jul 14 02:07:26.054595 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:26.054574 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 02:07:26.837398 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:26.837361 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v88wb" event={"ID":"4b990867-e76a-497f-ab31-2bd3e55b51c5","Type":"ContainerStarted","Data":"f787943f05edd9a0990029a4ed9b7ec9f748723c35c27e8b8c8117afa312fa10"} Jul 14 02:07:28.847481 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:28.847437 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v88wb" event={"ID":"4b990867-e76a-497f-ab31-2bd3e55b51c5","Type":"ContainerStarted","Data":"60b1c2bcbf30471897b7e455d8a92e08b9bddf53b8e34029900b683f436ba4a0"} Jul 14 02:07:28.864178 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:28.864126 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v88wb" podStartSLOduration=1.271176224 podStartE2EDuration="3.864108321s" totalImagesPullingTime="2.592932097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:07:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:07:28.862632266 +0000 UTC m=+351.017468918" watchObservedRunningTime="2026-07-14 02:07:28.864108321 +0000 UTC m=+351.018944968" Jul 14 02:07:29.557146 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:29.557112 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:07:29.658907 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:29.658877 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:07:31.657087 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:31.657056 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:07:41.881376 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:41.881341 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:07:42.010533 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:42.010495 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7cbf965c66-v7xzk"] Jul 14 02:07:42.029328 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:42.029292 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7cbf965c66-v7xzk"] Jul 14 02:07:42.029328 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:42.029324 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:07:42.029565 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:42.029379 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7cbf965c66-v7xzk" Jul 14 02:07:42.031523 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:42.031502 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 14 02:07:42.032027 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:42.032010 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 14 02:07:42.032109 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:42.032067 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 14 02:07:42.032201 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:42.032181 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-gxlns\"" Jul 14 02:07:42.082799 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:42.082763 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9426ae77-c6f7-4e20-b782-97af2aef5277-metrics-cert\") pod \"lws-controller-manager-7cbf965c66-v7xzk\" (UID: \"9426ae77-c6f7-4e20-b782-97af2aef5277\") " pod="openshift-lws-operator/lws-controller-manager-7cbf965c66-v7xzk" Jul 14 02:07:42.082969 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:42.082821 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9426ae77-c6f7-4e20-b782-97af2aef5277-cert\") pod \"lws-controller-manager-7cbf965c66-v7xzk\" (UID: \"9426ae77-c6f7-4e20-b782-97af2aef5277\") " pod="openshift-lws-operator/lws-controller-manager-7cbf965c66-v7xzk" Jul 14 02:07:42.082969 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:42.082897 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9426ae77-c6f7-4e20-b782-97af2aef5277-manager-config\") pod \"lws-controller-manager-7cbf965c66-v7xzk\" (UID: \"9426ae77-c6f7-4e20-b782-97af2aef5277\") " pod="openshift-lws-operator/lws-controller-manager-7cbf965c66-v7xzk" Jul 14 02:07:42.082969 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:42.082940 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdj88\" (UniqueName: \"kubernetes.io/projected/9426ae77-c6f7-4e20-b782-97af2aef5277-kube-api-access-mdj88\") pod \"lws-controller-manager-7cbf965c66-v7xzk\" (UID: \"9426ae77-c6f7-4e20-b782-97af2aef5277\") " pod="openshift-lws-operator/lws-controller-manager-7cbf965c66-v7xzk" Jul 14 02:07:42.184044 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:42.183955 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdj88\" (UniqueName: \"kubernetes.io/projected/9426ae77-c6f7-4e20-b782-97af2aef5277-kube-api-access-mdj88\") pod \"lws-controller-manager-7cbf965c66-v7xzk\" (UID: \"9426ae77-c6f7-4e20-b782-97af2aef5277\") " pod="openshift-lws-operator/lws-controller-manager-7cbf965c66-v7xzk" Jul 14 02:07:42.184044 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:42.184019 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9426ae77-c6f7-4e20-b782-97af2aef5277-metrics-cert\") pod \"lws-controller-manager-7cbf965c66-v7xzk\" (UID: \"9426ae77-c6f7-4e20-b782-97af2aef5277\") " pod="openshift-lws-operator/lws-controller-manager-7cbf965c66-v7xzk" Jul 14 02:07:42.184238 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:42.184050 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9426ae77-c6f7-4e20-b782-97af2aef5277-cert\") pod \"lws-controller-manager-7cbf965c66-v7xzk\" (UID: \"9426ae77-c6f7-4e20-b782-97af2aef5277\") " pod="openshift-lws-operator/lws-controller-manager-7cbf965c66-v7xzk" Jul 14 02:07:42.184238 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:42.184075 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9426ae77-c6f7-4e20-b782-97af2aef5277-manager-config\") pod \"lws-controller-manager-7cbf965c66-v7xzk\" (UID: \"9426ae77-c6f7-4e20-b782-97af2aef5277\") " pod="openshift-lws-operator/lws-controller-manager-7cbf965c66-v7xzk" Jul 14 02:07:42.184715 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:42.184690 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9426ae77-c6f7-4e20-b782-97af2aef5277-manager-config\") pod \"lws-controller-manager-7cbf965c66-v7xzk\" (UID: \"9426ae77-c6f7-4e20-b782-97af2aef5277\") " pod="openshift-lws-operator/lws-controller-manager-7cbf965c66-v7xzk" Jul 14 02:07:42.186602 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:42.186572 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9426ae77-c6f7-4e20-b782-97af2aef5277-metrics-cert\") pod \"lws-controller-manager-7cbf965c66-v7xzk\" (UID: \"9426ae77-c6f7-4e20-b782-97af2aef5277\") " pod="openshift-lws-operator/lws-controller-manager-7cbf965c66-v7xzk" Jul 14 02:07:42.186725 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:42.186711 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9426ae77-c6f7-4e20-b782-97af2aef5277-cert\") pod \"lws-controller-manager-7cbf965c66-v7xzk\" (UID: \"9426ae77-c6f7-4e20-b782-97af2aef5277\") " pod="openshift-lws-operator/lws-controller-manager-7cbf965c66-v7xzk" Jul 14 02:07:42.191842 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:42.191821 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdj88\" (UniqueName: \"kubernetes.io/projected/9426ae77-c6f7-4e20-b782-97af2aef5277-kube-api-access-mdj88\") pod \"lws-controller-manager-7cbf965c66-v7xzk\" (UID: \"9426ae77-c6f7-4e20-b782-97af2aef5277\") " pod="openshift-lws-operator/lws-controller-manager-7cbf965c66-v7xzk" Jul 14 02:07:42.342615 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:42.342582 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7cbf965c66-v7xzk" Jul 14 02:07:42.476748 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:42.476721 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7cbf965c66-v7xzk"] Jul 14 02:07:42.479211 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:07:42.479183 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9426ae77_c6f7_4e20_b782_97af2aef5277.slice/crio-6ba79473e2ac56c7e47e17d07bc78fb51b7f5d189a5d2dd9771d7340960c24d9 WatchSource:0}: Error finding container 6ba79473e2ac56c7e47e17d07bc78fb51b7f5d189a5d2dd9771d7340960c24d9: Status 404 returned error can't find the container with id 6ba79473e2ac56c7e47e17d07bc78fb51b7f5d189a5d2dd9771d7340960c24d9 Jul 14 02:07:42.901220 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:42.901185 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7cbf965c66-v7xzk" event={"ID":"9426ae77-c6f7-4e20-b782-97af2aef5277","Type":"ContainerStarted","Data":"6ba79473e2ac56c7e47e17d07bc78fb51b7f5d189a5d2dd9771d7340960c24d9"} Jul 14 02:07:44.386872 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:44.386839 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:07:44.484838 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:44.484808 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:07:44.910698 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:44.910660 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7cbf965c66-v7xzk" event={"ID":"9426ae77-c6f7-4e20-b782-97af2aef5277","Type":"ContainerStarted","Data":"732435abf8ae2028a80fa9f3fef37649e70e49860f1b4445e0f4598e3e431384"} Jul 14 02:07:44.910873 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:44.910770 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7cbf965c66-v7xzk" Jul 14 02:07:44.925160 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:44.925106 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7cbf965c66-v7xzk" podStartSLOduration=2.424964185 podStartE2EDuration="3.925086156s" totalImagesPullingTime="1.500121971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:07:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:07:44.924855075 +0000 UTC m=+367.079691724" watchObservedRunningTime="2026-07-14 02:07:44.925086156 +0000 UTC m=+367.079922805" Jul 14 02:07:44.995001 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:44.994968 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:07:45.195330 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:45.195206 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:07:45.304033 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:45.303997 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:07:45.410238 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:45.410203 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5756559977-b9bqv"] Jul 14 02:07:45.413594 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:45.413568 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5756559977-b9bqv" Jul 14 02:07:45.418086 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:45.418060 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 14 02:07:45.418217 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:45.418113 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 14 02:07:45.418593 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:45.418573 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 14 02:07:45.418708 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:45.418579 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 14 02:07:45.418777 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:45.418724 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-wpb5w\"" Jul 14 02:07:45.457070 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:45.456981 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5756559977-b9bqv"] Jul 14 02:07:45.510824 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:45.510786 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1d26deb9-d556-46a9-8c88-43fb9390a06b-manifests\") pod \"opendatahub-operator-controller-manager-5756559977-b9bqv\" (UID: \"1d26deb9-d556-46a9-8c88-43fb9390a06b\") " pod="opendatahub/opendatahub-operator-controller-manager-5756559977-b9bqv" Jul 14 02:07:45.510824 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:45.510828 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1d26deb9-d556-46a9-8c88-43fb9390a06b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5756559977-b9bqv\" (UID: \"1d26deb9-d556-46a9-8c88-43fb9390a06b\") " pod="opendatahub/opendatahub-operator-controller-manager-5756559977-b9bqv" Jul 14 02:07:45.511079 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:45.510846 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c949c\" (UniqueName: \"kubernetes.io/projected/1d26deb9-d556-46a9-8c88-43fb9390a06b-kube-api-access-c949c\") pod \"opendatahub-operator-controller-manager-5756559977-b9bqv\" (UID: \"1d26deb9-d556-46a9-8c88-43fb9390a06b\") " pod="opendatahub/opendatahub-operator-controller-manager-5756559977-b9bqv" Jul 14 02:07:45.511079 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:45.510938 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d26deb9-d556-46a9-8c88-43fb9390a06b-webhook-cert\") pod \"opendatahub-operator-controller-manager-5756559977-b9bqv\" (UID: \"1d26deb9-d556-46a9-8c88-43fb9390a06b\") " pod="opendatahub/opendatahub-operator-controller-manager-5756559977-b9bqv" Jul 14 02:07:45.511079 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:45.510991 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d26deb9-d556-46a9-8c88-43fb9390a06b-tmp\") pod \"opendatahub-operator-controller-manager-5756559977-b9bqv\" (UID: \"1d26deb9-d556-46a9-8c88-43fb9390a06b\") " pod="opendatahub/opendatahub-operator-controller-manager-5756559977-b9bqv" Jul 14 02:07:45.566182 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:45.566151 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:07:45.612210 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:45.612170 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d26deb9-d556-46a9-8c88-43fb9390a06b-tmp\") pod \"opendatahub-operator-controller-manager-5756559977-b9bqv\" (UID: \"1d26deb9-d556-46a9-8c88-43fb9390a06b\") " pod="opendatahub/opendatahub-operator-controller-manager-5756559977-b9bqv" Jul 14 02:07:45.612210 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:45.612214 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1d26deb9-d556-46a9-8c88-43fb9390a06b-manifests\") pod \"opendatahub-operator-controller-manager-5756559977-b9bqv\" (UID: \"1d26deb9-d556-46a9-8c88-43fb9390a06b\") " pod="opendatahub/opendatahub-operator-controller-manager-5756559977-b9bqv" Jul 14 02:07:45.612455 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:45.612248 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1d26deb9-d556-46a9-8c88-43fb9390a06b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5756559977-b9bqv\" (UID: \"1d26deb9-d556-46a9-8c88-43fb9390a06b\") " pod="opendatahub/opendatahub-operator-controller-manager-5756559977-b9bqv" Jul 14 02:07:45.612455 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:45.612296 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c949c\" (UniqueName: \"kubernetes.io/projected/1d26deb9-d556-46a9-8c88-43fb9390a06b-kube-api-access-c949c\") pod \"opendatahub-operator-controller-manager-5756559977-b9bqv\" (UID: \"1d26deb9-d556-46a9-8c88-43fb9390a06b\") " pod="opendatahub/opendatahub-operator-controller-manager-5756559977-b9bqv" Jul 14 02:07:45.612455 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:45.612373 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d26deb9-d556-46a9-8c88-43fb9390a06b-webhook-cert\") pod \"opendatahub-operator-controller-manager-5756559977-b9bqv\" (UID: \"1d26deb9-d556-46a9-8c88-43fb9390a06b\") " pod="opendatahub/opendatahub-operator-controller-manager-5756559977-b9bqv" Jul 14 02:07:45.612684 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:45.612660 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1d26deb9-d556-46a9-8c88-43fb9390a06b-manifests\") pod \"opendatahub-operator-controller-manager-5756559977-b9bqv\" (UID: \"1d26deb9-d556-46a9-8c88-43fb9390a06b\") " pod="opendatahub/opendatahub-operator-controller-manager-5756559977-b9bqv" Jul 14 02:07:45.614792 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:45.614765 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d26deb9-d556-46a9-8c88-43fb9390a06b-tmp\") pod \"opendatahub-operator-controller-manager-5756559977-b9bqv\" (UID: \"1d26deb9-d556-46a9-8c88-43fb9390a06b\") " pod="opendatahub/opendatahub-operator-controller-manager-5756559977-b9bqv" Jul 14 02:07:45.614917 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:45.614862 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1d26deb9-d556-46a9-8c88-43fb9390a06b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5756559977-b9bqv\" (UID: \"1d26deb9-d556-46a9-8c88-43fb9390a06b\") " pod="opendatahub/opendatahub-operator-controller-manager-5756559977-b9bqv" Jul 14 02:07:45.614917 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:45.614886 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d26deb9-d556-46a9-8c88-43fb9390a06b-webhook-cert\") pod \"opendatahub-operator-controller-manager-5756559977-b9bqv\" (UID: \"1d26deb9-d556-46a9-8c88-43fb9390a06b\") " pod="opendatahub/opendatahub-operator-controller-manager-5756559977-b9bqv" Jul 14 02:07:45.619400 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:45.619373 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c949c\" (UniqueName: \"kubernetes.io/projected/1d26deb9-d556-46a9-8c88-43fb9390a06b-kube-api-access-c949c\") pod \"opendatahub-operator-controller-manager-5756559977-b9bqv\" (UID: \"1d26deb9-d556-46a9-8c88-43fb9390a06b\") " pod="opendatahub/opendatahub-operator-controller-manager-5756559977-b9bqv" Jul 14 02:07:45.669943 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:45.669906 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:07:45.724413 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:45.724321 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5756559977-b9bqv" Jul 14 02:07:45.778291 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:45.778213 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:07:45.872139 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:45.872106 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:07:45.872393 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:45.872378 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5756559977-b9bqv"] Jul 14 02:07:45.875366 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:07:45.875344 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d26deb9_d556_46a9_8c88_43fb9390a06b.slice/crio-7acd3854a96eb7780ec86e91a9a33f64d21fde4a0a498e257405c9719a038354 WatchSource:0}: Error finding container 7acd3854a96eb7780ec86e91a9a33f64d21fde4a0a498e257405c9719a038354: Status 404 returned error can't find the container with id 7acd3854a96eb7780ec86e91a9a33f64d21fde4a0a498e257405c9719a038354 Jul 14 02:07:45.915884 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:45.915853 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5756559977-b9bqv" event={"ID":"1d26deb9-d556-46a9-8c88-43fb9390a06b","Type":"ContainerStarted","Data":"7acd3854a96eb7780ec86e91a9a33f64d21fde4a0a498e257405c9719a038354"} Jul 14 02:07:45.975551 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:45.975462 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:07:46.071354 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:46.071321 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:07:49.934692 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:49.934651 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5756559977-b9bqv" event={"ID":"1d26deb9-d556-46a9-8c88-43fb9390a06b","Type":"ContainerStarted","Data":"97711f7b6591fe8989d44ec9d8e51ce98e98139acc4395ddd8e931555b847b10"} Jul 14 02:07:50.940360 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:50.940329 2577 generic.go:356] "Generic (PLEG): container finished" podID="1d26deb9-d556-46a9-8c88-43fb9390a06b" containerID="97711f7b6591fe8989d44ec9d8e51ce98e98139acc4395ddd8e931555b847b10" exitCode=0 Jul 14 02:07:50.940747 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:50.940418 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5756559977-b9bqv" event={"ID":"1d26deb9-d556-46a9-8c88-43fb9390a06b","Type":"ContainerDied","Data":"97711f7b6591fe8989d44ec9d8e51ce98e98139acc4395ddd8e931555b847b10"} Jul 14 02:07:51.946000 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:51.945961 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5756559977-b9bqv" event={"ID":"1d26deb9-d556-46a9-8c88-43fb9390a06b","Type":"ContainerStarted","Data":"9f292f1195d0e11016deca83c76f2865dd785bf182bc6a960f83b05a1727eed4"} Jul 14 02:07:51.946385 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:51.946094 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5756559977-b9bqv" Jul 14 02:07:51.964654 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:51.964606 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5756559977-b9bqv" podStartSLOduration=2.515760782 podStartE2EDuration="6.96458826s" totalImagesPullingTime="3.368792073s" totalInitContainerRuntime="1.080035405s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:07:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:07:51.96312587 +0000 UTC m=+374.117962518" watchObservedRunningTime="2026-07-14 02:07:51.96458826 +0000 UTC m=+374.119424908" Jul 14 02:07:55.919007 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:55.918976 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7cbf965c66-v7xzk" Jul 14 02:07:56.558738 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:56.558699 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:07:56.660890 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:07:56.660844 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:08:02.952872 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:02.952843 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5756559977-b9bqv" Jul 14 02:08:04.479246 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:04.479206 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:08:04.575857 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:04.575781 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:08:15.272112 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:15.272078 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:08:15.490760 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:15.490730 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:08:15.559864 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:15.559785 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:08:15.661263 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:15.661210 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:08:16.260282 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:16.260239 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:08:16.461849 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:16.461804 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:08:16.529063 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:16.528981 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:08:17.317929 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:17.317896 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:08:22.134556 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:22.134520 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:08:22.203697 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:22.203659 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:08:22.762689 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:22.762653 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt"] Jul 14 02:08:22.772857 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:22.772836 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt" Jul 14 02:08:22.778163 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:22.778135 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 14 02:08:22.778338 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:22.778320 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-t9gvq\"" Jul 14 02:08:22.805914 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:22.805887 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt"] Jul 14 02:08:22.812449 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:22.812426 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:08:22.928607 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:22.928573 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83887e08-ddc3-4ae6-a9cd-126107e2dde0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf447zmt\" (UID: \"83887e08-ddc3-4ae6-a9cd-126107e2dde0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt" Jul 14 02:08:22.928771 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:22.928615 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83887e08-ddc3-4ae6-a9cd-126107e2dde0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf447zmt\" (UID: \"83887e08-ddc3-4ae6-a9cd-126107e2dde0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt" Jul 14 02:08:22.928771 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:22.928658 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83887e08-ddc3-4ae6-a9cd-126107e2dde0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf447zmt\" (UID: \"83887e08-ddc3-4ae6-a9cd-126107e2dde0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt" Jul 14 02:08:22.928771 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:22.928692 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83887e08-ddc3-4ae6-a9cd-126107e2dde0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf447zmt\" (UID: \"83887e08-ddc3-4ae6-a9cd-126107e2dde0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt" Jul 14 02:08:22.928771 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:22.928714 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83887e08-ddc3-4ae6-a9cd-126107e2dde0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf447zmt\" (UID: \"83887e08-ddc3-4ae6-a9cd-126107e2dde0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt" Jul 14 02:08:22.928771 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:22.928742 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83887e08-ddc3-4ae6-a9cd-126107e2dde0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf447zmt\" (UID: \"83887e08-ddc3-4ae6-a9cd-126107e2dde0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt" Jul 14 02:08:22.928771 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:22.928764 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlsqj\" (UniqueName: \"kubernetes.io/projected/83887e08-ddc3-4ae6-a9cd-126107e2dde0-kube-api-access-vlsqj\") pod \"data-science-gateway-data-science-gateway-class-6576859cf447zmt\" (UID: \"83887e08-ddc3-4ae6-a9cd-126107e2dde0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt" Jul 14 02:08:22.929027 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:22.928847 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83887e08-ddc3-4ae6-a9cd-126107e2dde0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf447zmt\" (UID: \"83887e08-ddc3-4ae6-a9cd-126107e2dde0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt" Jul 14 02:08:22.929027 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:22.928888 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83887e08-ddc3-4ae6-a9cd-126107e2dde0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf447zmt\" (UID: \"83887e08-ddc3-4ae6-a9cd-126107e2dde0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt" Jul 14 02:08:22.975037 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:22.974987 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:08:23.030224 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:23.030154 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83887e08-ddc3-4ae6-a9cd-126107e2dde0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf447zmt\" (UID: \"83887e08-ddc3-4ae6-a9cd-126107e2dde0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt" Jul 14 02:08:23.030224 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:23.030187 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83887e08-ddc3-4ae6-a9cd-126107e2dde0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf447zmt\" (UID: \"83887e08-ddc3-4ae6-a9cd-126107e2dde0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt" Jul 14 02:08:23.030224 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:23.030213 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83887e08-ddc3-4ae6-a9cd-126107e2dde0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf447zmt\" (UID: \"83887e08-ddc3-4ae6-a9cd-126107e2dde0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt" Jul 14 02:08:23.030521 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:23.030236 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83887e08-ddc3-4ae6-a9cd-126107e2dde0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf447zmt\" (UID: \"83887e08-ddc3-4ae6-a9cd-126107e2dde0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt" Jul 14 02:08:23.030521 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:23.030419 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83887e08-ddc3-4ae6-a9cd-126107e2dde0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf447zmt\" (UID: \"83887e08-ddc3-4ae6-a9cd-126107e2dde0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt" Jul 14 02:08:23.030521 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:23.030476 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83887e08-ddc3-4ae6-a9cd-126107e2dde0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf447zmt\" (UID: \"83887e08-ddc3-4ae6-a9cd-126107e2dde0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt" Jul 14 02:08:23.030674 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:23.030520 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83887e08-ddc3-4ae6-a9cd-126107e2dde0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf447zmt\" (UID: \"83887e08-ddc3-4ae6-a9cd-126107e2dde0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt" Jul 14 02:08:23.030674 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:23.030582 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83887e08-ddc3-4ae6-a9cd-126107e2dde0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf447zmt\" (UID: \"83887e08-ddc3-4ae6-a9cd-126107e2dde0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt" Jul 14 02:08:23.030674 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:23.030612 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlsqj\" (UniqueName: \"kubernetes.io/projected/83887e08-ddc3-4ae6-a9cd-126107e2dde0-kube-api-access-vlsqj\") pod \"data-science-gateway-data-science-gateway-class-6576859cf447zmt\" (UID: \"83887e08-ddc3-4ae6-a9cd-126107e2dde0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt" Jul 14 02:08:23.030825 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:23.030808 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83887e08-ddc3-4ae6-a9cd-126107e2dde0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf447zmt\" (UID: \"83887e08-ddc3-4ae6-a9cd-126107e2dde0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt" Jul 14 02:08:23.030876 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:23.030834 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83887e08-ddc3-4ae6-a9cd-126107e2dde0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf447zmt\" (UID: \"83887e08-ddc3-4ae6-a9cd-126107e2dde0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt" Jul 14 02:08:23.030951 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:23.030933 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83887e08-ddc3-4ae6-a9cd-126107e2dde0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf447zmt\" (UID: \"83887e08-ddc3-4ae6-a9cd-126107e2dde0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt" Jul 14 02:08:23.031262 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:23.031240 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83887e08-ddc3-4ae6-a9cd-126107e2dde0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf447zmt\" (UID: \"83887e08-ddc3-4ae6-a9cd-126107e2dde0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt" Jul 14 02:08:23.031346 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:23.031332 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83887e08-ddc3-4ae6-a9cd-126107e2dde0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf447zmt\" (UID: \"83887e08-ddc3-4ae6-a9cd-126107e2dde0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt" Jul 14 02:08:23.032427 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:23.032407 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83887e08-ddc3-4ae6-a9cd-126107e2dde0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf447zmt\" (UID: \"83887e08-ddc3-4ae6-a9cd-126107e2dde0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt" Jul 14 02:08:23.032990 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:23.032969 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83887e08-ddc3-4ae6-a9cd-126107e2dde0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf447zmt\" (UID: \"83887e08-ddc3-4ae6-a9cd-126107e2dde0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt" Jul 14 02:08:23.040536 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:23.040512 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83887e08-ddc3-4ae6-a9cd-126107e2dde0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf447zmt\" (UID: \"83887e08-ddc3-4ae6-a9cd-126107e2dde0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt" Jul 14 02:08:23.041036 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:23.041016 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlsqj\" (UniqueName: \"kubernetes.io/projected/83887e08-ddc3-4ae6-a9cd-126107e2dde0-kube-api-access-vlsqj\") pod \"data-science-gateway-data-science-gateway-class-6576859cf447zmt\" (UID: \"83887e08-ddc3-4ae6-a9cd-126107e2dde0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt" Jul 14 02:08:23.078939 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:23.078919 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:08:23.082903 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:23.082887 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt" Jul 14 02:08:23.211761 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:23.211729 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt"] Jul 14 02:08:23.215132 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:08:23.215107 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83887e08_ddc3_4ae6_a9cd_126107e2dde0.slice/crio-2041e2c9066b08764d026498dbb730752273b4ad426b6240a5903e6e1ef92574 WatchSource:0}: Error finding container 2041e2c9066b08764d026498dbb730752273b4ad426b6240a5903e6e1ef92574: Status 404 returned error can't find the container with id 2041e2c9066b08764d026498dbb730752273b4ad426b6240a5903e6e1ef92574 Jul 14 02:08:24.080471 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:24.080403 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt" event={"ID":"83887e08-ddc3-4ae6-a9cd-126107e2dde0","Type":"ContainerStarted","Data":"2041e2c9066b08764d026498dbb730752273b4ad426b6240a5903e6e1ef92574"} Jul 14 02:08:26.023382 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:26.023344 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:08:26.023626 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:26.023422 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:08:26.023626 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:26.023452 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:08:26.096068 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:26.096028 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt" event={"ID":"83887e08-ddc3-4ae6-a9cd-126107e2dde0","Type":"ContainerStarted","Data":"f037cb64add93aa6f9dedf23b9adad450f638e74b97e7b67d59c459c349a3e1f"} Jul 14 02:08:26.118889 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:26.118812 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt" podStartSLOduration=1.31268603 podStartE2EDuration="4.118794756s" totalImagesPullingTime="2.806108726s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:08:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:08:26.118011129 +0000 UTC m=+408.272847779" watchObservedRunningTime="2026-07-14 02:08:26.118794756 +0000 UTC m=+408.273631402" Jul 14 02:08:26.622346 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:26.622257 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:08:26.695803 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:26.695770 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-k4497"] Jul 14 02:08:26.699401 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:26.699381 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-k4497" Jul 14 02:08:26.701599 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:26.701571 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 14 02:08:26.701722 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:26.701585 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-2p5rd\"" Jul 14 02:08:26.710385 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:26.710361 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zf54c"] Jul 14 02:08:26.713590 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:26.713570 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-k4497"] Jul 14 02:08:26.713680 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:26.713649 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zf54c" Jul 14 02:08:26.715594 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:26.715561 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 14 02:08:26.715888 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:26.715872 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-4vz66\"" Jul 14 02:08:26.720859 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:26.720838 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:08:26.729420 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:26.729397 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zf54c"] Jul 14 02:08:26.866868 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:26.866822 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xknqb\" (UniqueName: \"kubernetes.io/projected/44a4b3f6-f9fb-40d2-ad21-7e63a6a34269-kube-api-access-xknqb\") pod \"kserve-controller-manager-d6f4b65b8-k4497\" (UID: \"44a4b3f6-f9fb-40d2-ad21-7e63a6a34269\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-k4497" Jul 14 02:08:26.866868 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:26.866860 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/44a4b3f6-f9fb-40d2-ad21-7e63a6a34269-cert\") pod \"kserve-controller-manager-d6f4b65b8-k4497\" (UID: \"44a4b3f6-f9fb-40d2-ad21-7e63a6a34269\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-k4497" Jul 14 02:08:26.867062 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:26.866897 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4lqk\" (UniqueName: \"kubernetes.io/projected/bc75f235-9d07-4c00-8e8e-59043d8f108e-kube-api-access-p4lqk\") pod \"llmisvc-controller-manager-777c6bd7d5-zf54c\" (UID: \"bc75f235-9d07-4c00-8e8e-59043d8f108e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zf54c" Jul 14 02:08:26.867062 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:26.866964 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bc75f235-9d07-4c00-8e8e-59043d8f108e-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zf54c\" (UID: \"bc75f235-9d07-4c00-8e8e-59043d8f108e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zf54c" Jul 14 02:08:26.968323 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:26.968211 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xknqb\" (UniqueName: \"kubernetes.io/projected/44a4b3f6-f9fb-40d2-ad21-7e63a6a34269-kube-api-access-xknqb\") pod \"kserve-controller-manager-d6f4b65b8-k4497\" (UID: \"44a4b3f6-f9fb-40d2-ad21-7e63a6a34269\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-k4497" Jul 14 02:08:26.968323 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:26.968248 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/44a4b3f6-f9fb-40d2-ad21-7e63a6a34269-cert\") pod \"kserve-controller-manager-d6f4b65b8-k4497\" (UID: \"44a4b3f6-f9fb-40d2-ad21-7e63a6a34269\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-k4497" Jul 14 02:08:26.968323 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:26.968309 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4lqk\" (UniqueName: \"kubernetes.io/projected/bc75f235-9d07-4c00-8e8e-59043d8f108e-kube-api-access-p4lqk\") pod \"llmisvc-controller-manager-777c6bd7d5-zf54c\" (UID: \"bc75f235-9d07-4c00-8e8e-59043d8f108e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zf54c" Jul 14 02:08:26.968622 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:26.968330 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bc75f235-9d07-4c00-8e8e-59043d8f108e-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zf54c\" (UID: \"bc75f235-9d07-4c00-8e8e-59043d8f108e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zf54c" Jul 14 02:08:26.968622 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:08:26.968449 2577 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 14 02:08:26.968622 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:08:26.968528 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44a4b3f6-f9fb-40d2-ad21-7e63a6a34269-cert podName:44a4b3f6-f9fb-40d2-ad21-7e63a6a34269 nodeName:}" failed. No retries permitted until 2026-07-14 02:08:27.468508141 +0000 UTC m=+409.623344766 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/44a4b3f6-f9fb-40d2-ad21-7e63a6a34269-cert") pod "kserve-controller-manager-d6f4b65b8-k4497" (UID: "44a4b3f6-f9fb-40d2-ad21-7e63a6a34269") : secret "kserve-webhook-server-cert" not found Jul 14 02:08:26.970790 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:26.970769 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bc75f235-9d07-4c00-8e8e-59043d8f108e-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zf54c\" (UID: \"bc75f235-9d07-4c00-8e8e-59043d8f108e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zf54c" Jul 14 02:08:26.983879 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:26.983849 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xknqb\" (UniqueName: \"kubernetes.io/projected/44a4b3f6-f9fb-40d2-ad21-7e63a6a34269-kube-api-access-xknqb\") pod \"kserve-controller-manager-d6f4b65b8-k4497\" (UID: \"44a4b3f6-f9fb-40d2-ad21-7e63a6a34269\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-k4497" Jul 14 02:08:26.991601 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:26.991577 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4lqk\" (UniqueName: \"kubernetes.io/projected/bc75f235-9d07-4c00-8e8e-59043d8f108e-kube-api-access-p4lqk\") pod \"llmisvc-controller-manager-777c6bd7d5-zf54c\" (UID: \"bc75f235-9d07-4c00-8e8e-59043d8f108e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zf54c" Jul 14 02:08:26.999767 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:26.999744 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:08:27.024030 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:27.024002 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zf54c" Jul 14 02:08:27.083636 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:27.083598 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt" Jul 14 02:08:27.089613 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:27.089484 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt" Jul 14 02:08:27.101330 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:27.101233 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt" Jul 14 02:08:27.102302 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:27.102261 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf447zmt" Jul 14 02:08:27.161028 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:27.161005 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zf54c"] Jul 14 02:08:27.163142 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:08:27.163110 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podbc75f235_9d07_4c00_8e8e_59043d8f108e.slice/crio-78e6e3162c5083d48c142569c2b14c7553e3793d328d0848ae0beec7d48f7757 WatchSource:0}: Error finding container 78e6e3162c5083d48c142569c2b14c7553e3793d328d0848ae0beec7d48f7757: Status 404 returned error can't find the container with id 78e6e3162c5083d48c142569c2b14c7553e3793d328d0848ae0beec7d48f7757 Jul 14 02:08:27.239215 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:27.239104 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:08:27.472470 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:27.472439 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/44a4b3f6-f9fb-40d2-ad21-7e63a6a34269-cert\") pod \"kserve-controller-manager-d6f4b65b8-k4497\" (UID: \"44a4b3f6-f9fb-40d2-ad21-7e63a6a34269\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-k4497" Jul 14 02:08:27.474748 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:27.474720 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/44a4b3f6-f9fb-40d2-ad21-7e63a6a34269-cert\") pod \"kserve-controller-manager-d6f4b65b8-k4497\" (UID: \"44a4b3f6-f9fb-40d2-ad21-7e63a6a34269\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-k4497" Jul 14 02:08:27.609538 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:27.609459 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-k4497" Jul 14 02:08:27.747559 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:27.747531 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-k4497"] Jul 14 02:08:27.749263 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:08:27.749211 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44a4b3f6_f9fb_40d2_ad21_7e63a6a34269.slice/crio-7276e965465e54b17240a33d7fc369bd06874d6a728c1394eb3af760c086ad72 WatchSource:0}: Error finding container 7276e965465e54b17240a33d7fc369bd06874d6a728c1394eb3af760c086ad72: Status 404 returned error can't find the container with id 7276e965465e54b17240a33d7fc369bd06874d6a728c1394eb3af760c086ad72 Jul 14 02:08:27.902427 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:27.902397 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:08:28.018026 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:28.017998 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:08:28.107685 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:28.107654 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-k4497" event={"ID":"44a4b3f6-f9fb-40d2-ad21-7e63a6a34269","Type":"ContainerStarted","Data":"7276e965465e54b17240a33d7fc369bd06874d6a728c1394eb3af760c086ad72"} Jul 14 02:08:28.108872 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:28.108850 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zf54c" event={"ID":"bc75f235-9d07-4c00-8e8e-59043d8f108e","Type":"ContainerStarted","Data":"78e6e3162c5083d48c142569c2b14c7553e3793d328d0848ae0beec7d48f7757"} Jul 14 02:08:31.123215 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:31.123185 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-k4497" event={"ID":"44a4b3f6-f9fb-40d2-ad21-7e63a6a34269","Type":"ContainerStarted","Data":"262318e7bb3ff6d80725fd0f787c8aa567cd30db75ceb045fc673e26ab939268"} Jul 14 02:08:31.123683 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:31.123255 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-k4497" Jul 14 02:08:31.124626 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:31.124606 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zf54c" event={"ID":"bc75f235-9d07-4c00-8e8e-59043d8f108e","Type":"ContainerStarted","Data":"8dacdd2c742462d18c62bf8dff6e35894fa1b28823ae3e2b5e5af6b48297b695"} Jul 14 02:08:31.124734 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:31.124724 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zf54c" Jul 14 02:08:31.139819 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:31.139745 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-k4497" podStartSLOduration=2.007043246 podStartE2EDuration="5.139732799s" totalImagesPullingTime="3.132689553s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:08:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:08:31.138126509 +0000 UTC m=+413.292963157" watchObservedRunningTime="2026-07-14 02:08:31.139732799 +0000 UTC m=+413.294569444" Jul 14 02:08:31.154653 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:08:31.154612 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zf54c" podStartSLOduration=1.43747113 podStartE2EDuration="5.154599366s" totalImagesPullingTime="3.717128236s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:08:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:08:31.153596985 +0000 UTC m=+413.308433633" watchObservedRunningTime="2026-07-14 02:08:31.154599366 +0000 UTC m=+413.309436013" Jul 14 02:09:02.130935 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:09:02.130854 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zf54c" Jul 14 02:09:02.133844 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:09:02.133825 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-k4497" Jul 14 02:09:02.604183 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:09:02.604057 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:09:02.705097 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:09:02.705065 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:09:05.952792 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:09:05.952734 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:09:06.049016 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:09:06.048981 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:09:06.130331 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:09:06.130297 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:09:06.884112 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:09:06.884076 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:09:07.104177 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:09:07.104104 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:09:28.779785 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:09:28.779738 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:09:28.832767 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:09:28.832710 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:09:39.550905 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:09:39.550866 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:09:39.650396 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:09:39.650356 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:09:39.706115 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:09:39.706085 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:09:50.207307 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:09:50.207247 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:09:50.307002 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:09:50.306961 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:09:52.158160 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:09:52.158099 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:09:52.260306 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:09:52.260256 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:09:52.362926 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:09:52.362889 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:09:55.752446 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:09:55.752406 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:09:56.566547 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:09:56.566515 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:09:59.787446 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:09:59.787413 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:10:00.155924 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:00.155875 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:10:01.772743 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:01.772693 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:10:02.412311 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:02.412260 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-gkhww"] Jul 14 02:10:02.415656 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:02.415634 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gkhww" Jul 14 02:10:02.418455 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:02.418426 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 14 02:10:02.419071 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:02.419051 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-bjjkj\"" Jul 14 02:10:02.419143 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:02.419092 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 14 02:10:02.419143 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:02.419131 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 14 02:10:02.434257 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:02.434228 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-gkhww"] Jul 14 02:10:02.492329 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:02.492295 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6d7h6\" (UniqueName: \"kubernetes.io/projected/d823fbe9-7068-4f4d-af9d-8fa935a9f485-kube-api-access-6d7h6\") pod \"dns-operator-controller-manager-864dd78c4c-gkhww\" (UID: \"d823fbe9-7068-4f4d-af9d-8fa935a9f485\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gkhww" Jul 14 02:10:02.593086 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:02.593044 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6d7h6\" (UniqueName: \"kubernetes.io/projected/d823fbe9-7068-4f4d-af9d-8fa935a9f485-kube-api-access-6d7h6\") pod \"dns-operator-controller-manager-864dd78c4c-gkhww\" (UID: \"d823fbe9-7068-4f4d-af9d-8fa935a9f485\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gkhww" Jul 14 02:10:02.601296 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:02.601255 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6d7h6\" (UniqueName: \"kubernetes.io/projected/d823fbe9-7068-4f4d-af9d-8fa935a9f485-kube-api-access-6d7h6\") pod \"dns-operator-controller-manager-864dd78c4c-gkhww\" (UID: \"d823fbe9-7068-4f4d-af9d-8fa935a9f485\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gkhww" Jul 14 02:10:02.725993 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:02.725892 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gkhww" Jul 14 02:10:02.861579 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:02.861552 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-gkhww"] Jul 14 02:10:02.864240 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:10:02.864217 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd823fbe9_7068_4f4d_af9d_8fa935a9f485.slice/crio-4a98799e7b0f2cba0923d169a4222bb48101d8f8a60a96585f6b0aa13e1f4578 WatchSource:0}: Error finding container 4a98799e7b0f2cba0923d169a4222bb48101d8f8a60a96585f6b0aa13e1f4578: Status 404 returned error can't find the container with id 4a98799e7b0f2cba0923d169a4222bb48101d8f8a60a96585f6b0aa13e1f4578 Jul 14 02:10:02.953395 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:02.953360 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:10:03.050304 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:03.050203 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:10:03.505605 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:03.505575 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gkhww" event={"ID":"d823fbe9-7068-4f4d-af9d-8fa935a9f485","Type":"ContainerStarted","Data":"4a98799e7b0f2cba0923d169a4222bb48101d8f8a60a96585f6b0aa13e1f4578"} Jul 14 02:10:04.784208 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:04.784171 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:10:05.516502 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:05.516469 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gkhww" event={"ID":"d823fbe9-7068-4f4d-af9d-8fa935a9f485","Type":"ContainerStarted","Data":"c95f8b34f4d4afe852f8d228348e06983643c7274ce667ad8bab68a276fc19b0"} Jul 14 02:10:05.516682 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:05.516567 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gkhww" Jul 14 02:10:05.531691 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:05.531630 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gkhww" podStartSLOduration=1.438629978 podStartE2EDuration="3.531610289s" totalImagesPullingTime="2.092980311s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:10:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:10:05.531130247 +0000 UTC m=+507.685966896" watchObservedRunningTime="2026-07-14 02:10:05.531610289 +0000 UTC m=+507.686446938" Jul 14 02:10:06.838686 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:06.838651 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h6xzg"] Jul 14 02:10:06.842009 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:06.841985 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h6xzg" Jul 14 02:10:06.844093 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:06.844072 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-st7t7\"" Jul 14 02:10:06.857882 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:06.857852 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h6xzg"] Jul 14 02:10:06.932340 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:06.932300 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c07fb5ca-238b-4b03-bebb-903ae4d81e19-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-h6xzg\" (UID: \"c07fb5ca-238b-4b03-bebb-903ae4d81e19\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h6xzg" Jul 14 02:10:06.932526 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:06.932354 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gn6g\" (UniqueName: \"kubernetes.io/projected/c07fb5ca-238b-4b03-bebb-903ae4d81e19-kube-api-access-7gn6g\") pod \"kuadrant-operator-controller-manager-8684c9ff49-h6xzg\" (UID: \"c07fb5ca-238b-4b03-bebb-903ae4d81e19\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h6xzg" Jul 14 02:10:07.033531 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:07.033495 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c07fb5ca-238b-4b03-bebb-903ae4d81e19-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-h6xzg\" (UID: \"c07fb5ca-238b-4b03-bebb-903ae4d81e19\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h6xzg" Jul 14 02:10:07.033717 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:07.033551 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gn6g\" (UniqueName: \"kubernetes.io/projected/c07fb5ca-238b-4b03-bebb-903ae4d81e19-kube-api-access-7gn6g\") pod \"kuadrant-operator-controller-manager-8684c9ff49-h6xzg\" (UID: \"c07fb5ca-238b-4b03-bebb-903ae4d81e19\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h6xzg" Jul 14 02:10:07.033916 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:07.033895 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c07fb5ca-238b-4b03-bebb-903ae4d81e19-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-h6xzg\" (UID: \"c07fb5ca-238b-4b03-bebb-903ae4d81e19\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h6xzg" Jul 14 02:10:07.041232 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:07.041208 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gn6g\" (UniqueName: \"kubernetes.io/projected/c07fb5ca-238b-4b03-bebb-903ae4d81e19-kube-api-access-7gn6g\") pod \"kuadrant-operator-controller-manager-8684c9ff49-h6xzg\" (UID: \"c07fb5ca-238b-4b03-bebb-903ae4d81e19\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h6xzg" Jul 14 02:10:07.152342 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:07.152300 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h6xzg" Jul 14 02:10:07.291912 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:07.291887 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h6xzg"] Jul 14 02:10:07.294867 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:10:07.294841 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc07fb5ca_238b_4b03_bebb_903ae4d81e19.slice/crio-261d08c22ae6e0ef2149383a85ccf28df525de066bec5a3f6c563a65e56ae4df WatchSource:0}: Error finding container 261d08c22ae6e0ef2149383a85ccf28df525de066bec5a3f6c563a65e56ae4df: Status 404 returned error can't find the container with id 261d08c22ae6e0ef2149383a85ccf28df525de066bec5a3f6c563a65e56ae4df Jul 14 02:10:07.526803 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:07.526707 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h6xzg" event={"ID":"c07fb5ca-238b-4b03-bebb-903ae4d81e19","Type":"ContainerStarted","Data":"261d08c22ae6e0ef2149383a85ccf28df525de066bec5a3f6c563a65e56ae4df"} Jul 14 02:10:07.951531 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:07.951495 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:10:08.054458 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:08.054426 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:10:13.390736 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:13.390698 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:10:13.481614 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:13.481577 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:10:13.758284 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:13.758162 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:10:13.858950 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:13.858899 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:10:14.565161 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:14.565128 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h6xzg" event={"ID":"c07fb5ca-238b-4b03-bebb-903ae4d81e19","Type":"ContainerStarted","Data":"342e6cc5a06c817234e0834a60adbba56f914aebf3868d28342a236eafb8c053"} Jul 14 02:10:14.565556 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:14.565262 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h6xzg" Jul 14 02:10:14.587262 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:14.587208 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h6xzg" podStartSLOduration=2.326333932 podStartE2EDuration="8.587191173s" totalImagesPullingTime="6.260857241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:10:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:10:14.585319793 +0000 UTC m=+516.740156435" watchObservedRunningTime="2026-07-14 02:10:14.587191173 +0000 UTC m=+516.742027820" Jul 14 02:10:16.284184 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:16.284154 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:10:16.396915 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:16.396884 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:10:16.523534 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:16.523507 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gkhww" Jul 14 02:10:16.774931 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:16.774896 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:10:16.878769 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:16.878738 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:10:20.974559 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:20.974523 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:10:21.073253 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:21.073221 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:10:25.571369 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:25.571338 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h6xzg" Jul 14 02:10:27.257881 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:27.257847 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h6xzg"] Jul 14 02:10:27.258399 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:27.258081 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h6xzg" podUID="c07fb5ca-238b-4b03-bebb-903ae4d81e19" containerName="manager" containerID="cri-o://342e6cc5a06c817234e0834a60adbba56f914aebf3868d28342a236eafb8c053" gracePeriod=2 Jul 14 02:10:27.271804 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:27.271336 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h6xzg"] Jul 14 02:10:27.285328 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:10:27.285291 2577 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"c07fb5ca-238b-4b03-bebb-903ae4d81e19\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-h6xzg\": pods \"kuadrant-operator-controller-manager-8684c9ff49-h6xzg\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h6xzg" Jul 14 02:10:27.288348 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:27.288321 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rr4qq"] Jul 14 02:10:27.288707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:27.288694 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c07fb5ca-238b-4b03-bebb-903ae4d81e19" containerName="manager" Jul 14 02:10:27.288752 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:27.288709 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c07fb5ca-238b-4b03-bebb-903ae4d81e19" containerName="manager" Jul 14 02:10:27.288785 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:27.288761 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c07fb5ca-238b-4b03-bebb-903ae4d81e19" containerName="manager" Jul 14 02:10:27.292506 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:27.292480 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rr4qq" Jul 14 02:10:27.294441 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:10:27.294411 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-h6xzg\" is forbidden: User \"system:node:ip-10-0-131-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-25.ec2.internal' and this object" podUID="c07fb5ca-238b-4b03-bebb-903ae4d81e19" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h6xzg" Jul 14 02:10:27.308071 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:27.308039 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rr4qq"] Jul 14 02:10:27.412929 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:27.412891 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/72cb7106-3133-4e0d-8b21-3ad6fdd63c53-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rr4qq\" (UID: \"72cb7106-3133-4e0d-8b21-3ad6fdd63c53\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rr4qq" Jul 14 02:10:27.413108 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:27.412958 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pg25\" (UniqueName: \"kubernetes.io/projected/72cb7106-3133-4e0d-8b21-3ad6fdd63c53-kube-api-access-9pg25\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rr4qq\" (UID: \"72cb7106-3133-4e0d-8b21-3ad6fdd63c53\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rr4qq" Jul 14 02:10:27.504295 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:27.504248 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h6xzg" Jul 14 02:10:27.506457 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:10:27.506432 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-h6xzg\" is forbidden: User \"system:node:ip-10-0-131-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-25.ec2.internal' and this object" podUID="c07fb5ca-238b-4b03-bebb-903ae4d81e19" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h6xzg" Jul 14 02:10:27.513878 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:27.513820 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/72cb7106-3133-4e0d-8b21-3ad6fdd63c53-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rr4qq\" (UID: \"72cb7106-3133-4e0d-8b21-3ad6fdd63c53\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rr4qq" Jul 14 02:10:27.513955 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:27.513884 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pg25\" (UniqueName: \"kubernetes.io/projected/72cb7106-3133-4e0d-8b21-3ad6fdd63c53-kube-api-access-9pg25\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rr4qq\" (UID: \"72cb7106-3133-4e0d-8b21-3ad6fdd63c53\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rr4qq" Jul 14 02:10:27.514282 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:27.514249 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/72cb7106-3133-4e0d-8b21-3ad6fdd63c53-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rr4qq\" (UID: \"72cb7106-3133-4e0d-8b21-3ad6fdd63c53\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rr4qq" Jul 14 02:10:27.522846 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:27.522815 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pg25\" (UniqueName: \"kubernetes.io/projected/72cb7106-3133-4e0d-8b21-3ad6fdd63c53-kube-api-access-9pg25\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rr4qq\" (UID: \"72cb7106-3133-4e0d-8b21-3ad6fdd63c53\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rr4qq" Jul 14 02:10:27.614742 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:27.614712 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c07fb5ca-238b-4b03-bebb-903ae4d81e19-extensions-socket-volume\") pod \"c07fb5ca-238b-4b03-bebb-903ae4d81e19\" (UID: \"c07fb5ca-238b-4b03-bebb-903ae4d81e19\") " Jul 14 02:10:27.614905 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:27.614768 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7gn6g\" (UniqueName: \"kubernetes.io/projected/c07fb5ca-238b-4b03-bebb-903ae4d81e19-kube-api-access-7gn6g\") pod \"c07fb5ca-238b-4b03-bebb-903ae4d81e19\" (UID: \"c07fb5ca-238b-4b03-bebb-903ae4d81e19\") " Jul 14 02:10:27.615149 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:27.615125 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c07fb5ca-238b-4b03-bebb-903ae4d81e19-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c07fb5ca-238b-4b03-bebb-903ae4d81e19" (UID: "c07fb5ca-238b-4b03-bebb-903ae4d81e19"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:10:27.617034 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:27.617004 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c07fb5ca-238b-4b03-bebb-903ae4d81e19-kube-api-access-7gn6g" (OuterVolumeSpecName: "kube-api-access-7gn6g") pod "c07fb5ca-238b-4b03-bebb-903ae4d81e19" (UID: "c07fb5ca-238b-4b03-bebb-903ae4d81e19"). InnerVolumeSpecName "kube-api-access-7gn6g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:10:27.625706 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:27.625681 2577 generic.go:356] "Generic (PLEG): container finished" podID="c07fb5ca-238b-4b03-bebb-903ae4d81e19" containerID="342e6cc5a06c817234e0834a60adbba56f914aebf3868d28342a236eafb8c053" exitCode=0 Jul 14 02:10:27.625820 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:27.625721 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h6xzg" Jul 14 02:10:27.625820 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:27.625744 2577 scope.go:117] "RemoveContainer" containerID="342e6cc5a06c817234e0834a60adbba56f914aebf3868d28342a236eafb8c053" Jul 14 02:10:27.628015 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:10:27.627986 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-h6xzg\" is forbidden: User \"system:node:ip-10-0-131-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-25.ec2.internal' and this object" podUID="c07fb5ca-238b-4b03-bebb-903ae4d81e19" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h6xzg" Jul 14 02:10:27.634497 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:27.634481 2577 scope.go:117] "RemoveContainer" containerID="342e6cc5a06c817234e0834a60adbba56f914aebf3868d28342a236eafb8c053" Jul 14 02:10:27.634552 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:27.634525 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rr4qq" Jul 14 02:10:27.634769 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:10:27.634753 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"342e6cc5a06c817234e0834a60adbba56f914aebf3868d28342a236eafb8c053\": container with ID starting with 342e6cc5a06c817234e0834a60adbba56f914aebf3868d28342a236eafb8c053 not found: ID does not exist" containerID="342e6cc5a06c817234e0834a60adbba56f914aebf3868d28342a236eafb8c053" Jul 14 02:10:27.634811 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:27.634776 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"342e6cc5a06c817234e0834a60adbba56f914aebf3868d28342a236eafb8c053"} err="failed to get container status \"342e6cc5a06c817234e0834a60adbba56f914aebf3868d28342a236eafb8c053\": rpc error: code = NotFound desc = could not find container \"342e6cc5a06c817234e0834a60adbba56f914aebf3868d28342a236eafb8c053\": container with ID starting with 342e6cc5a06c817234e0834a60adbba56f914aebf3868d28342a236eafb8c053 not found: ID does not exist" Jul 14 02:10:27.635501 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:10:27.635478 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-h6xzg\" is forbidden: User \"system:node:ip-10-0-131-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-25.ec2.internal' and this object" podUID="c07fb5ca-238b-4b03-bebb-903ae4d81e19" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h6xzg" Jul 14 02:10:27.641941 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:27.641922 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:10:27.715619 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:27.715589 2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c07fb5ca-238b-4b03-bebb-903ae4d81e19-extensions-socket-volume\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:10:27.715619 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:27.715615 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7gn6g\" (UniqueName: \"kubernetes.io/projected/c07fb5ca-238b-4b03-bebb-903ae4d81e19-kube-api-access-7gn6g\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:10:27.734800 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:27.734771 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:10:27.781891 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:27.781861 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rr4qq"] Jul 14 02:10:27.783629 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:10:27.783602 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72cb7106_3133_4e0d_8b21_3ad6fdd63c53.slice/crio-030c606097dce92bf930a9e0ac000dcee4c94a9b08964a4a23a07c0a85f08f8f WatchSource:0}: Error finding container 030c606097dce92bf930a9e0ac000dcee4c94a9b08964a4a23a07c0a85f08f8f: Status 404 returned error can't find the container with id 030c606097dce92bf930a9e0ac000dcee4c94a9b08964a4a23a07c0a85f08f8f Jul 14 02:10:28.262449 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:10:28.262408 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-h6xzg\" is forbidden: User \"system:node:ip-10-0-131-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-25.ec2.internal' and this object" podUID="c07fb5ca-238b-4b03-bebb-903ae4d81e19" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h6xzg" Jul 14 02:10:28.264691 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:28.264670 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c07fb5ca-238b-4b03-bebb-903ae4d81e19" path="/var/lib/kubelet/pods/c07fb5ca-238b-4b03-bebb-903ae4d81e19/volumes" Jul 14 02:10:28.632166 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:28.632122 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rr4qq" event={"ID":"72cb7106-3133-4e0d-8b21-3ad6fdd63c53","Type":"ContainerStarted","Data":"132b19f2c9f6809348ed8b0617f181f05963ad76201a620e35a6d6bd7290cbbe"} Jul 14 02:10:28.632379 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:28.632193 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rr4qq" event={"ID":"72cb7106-3133-4e0d-8b21-3ad6fdd63c53","Type":"ContainerStarted","Data":"030c606097dce92bf930a9e0ac000dcee4c94a9b08964a4a23a07c0a85f08f8f"} Jul 14 02:10:28.632379 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:28.632317 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rr4qq" Jul 14 02:10:28.652104 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:28.652052 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rr4qq" podStartSLOduration=1.6520349909999998 podStartE2EDuration="1.652034991s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:10:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:10:28.650927145 +0000 UTC m=+530.805763796" watchObservedRunningTime="2026-07-14 02:10:28.652034991 +0000 UTC m=+530.806871639" Jul 14 02:10:29.727649 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:29.727613 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:10:31.401495 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:31.401457 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:10:31.501469 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:31.501434 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:10:31.604226 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:31.604192 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:10:31.702616 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:31.702527 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:10:35.199523 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:35.199486 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:10:35.302849 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:35.302813 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:10:39.639869 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:39.639842 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rr4qq" Jul 14 02:10:39.763238 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:39.763207 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:10:39.861912 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:39.861879 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:10:44.103475 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:44.103441 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rr4qq"] Jul 14 02:10:44.103910 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:44.103620 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rr4qq" podUID="72cb7106-3133-4e0d-8b21-3ad6fdd63c53" containerName="manager" containerID="cri-o://132b19f2c9f6809348ed8b0617f181f05963ad76201a620e35a6d6bd7290cbbe" gracePeriod=10 Jul 14 02:10:44.307382 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:44.307348 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phnk9"] Jul 14 02:10:44.310828 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:44.310806 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phnk9" Jul 14 02:10:44.327715 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:44.327683 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phnk9"] Jul 14 02:10:44.358600 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:44.358538 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rr4qq" Jul 14 02:10:44.470487 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:44.470452 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9pg25\" (UniqueName: \"kubernetes.io/projected/72cb7106-3133-4e0d-8b21-3ad6fdd63c53-kube-api-access-9pg25\") pod \"72cb7106-3133-4e0d-8b21-3ad6fdd63c53\" (UID: \"72cb7106-3133-4e0d-8b21-3ad6fdd63c53\") " Jul 14 02:10:44.470637 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:44.470525 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/72cb7106-3133-4e0d-8b21-3ad6fdd63c53-extensions-socket-volume\") pod \"72cb7106-3133-4e0d-8b21-3ad6fdd63c53\" (UID: \"72cb7106-3133-4e0d-8b21-3ad6fdd63c53\") " Jul 14 02:10:44.470697 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:44.470678 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fcc04ee9-b14a-472a-81d0-b307a428b1c2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-phnk9\" (UID: \"fcc04ee9-b14a-472a-81d0-b307a428b1c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phnk9" Jul 14 02:10:44.470876 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:44.470855 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72cb7106-3133-4e0d-8b21-3ad6fdd63c53-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "72cb7106-3133-4e0d-8b21-3ad6fdd63c53" (UID: "72cb7106-3133-4e0d-8b21-3ad6fdd63c53"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:10:44.470927 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:44.470861 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxxvx\" (UniqueName: \"kubernetes.io/projected/fcc04ee9-b14a-472a-81d0-b307a428b1c2-kube-api-access-hxxvx\") pod \"kuadrant-operator-controller-manager-86878c769f-phnk9\" (UID: \"fcc04ee9-b14a-472a-81d0-b307a428b1c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phnk9" Jul 14 02:10:44.470927 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:44.470919 2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/72cb7106-3133-4e0d-8b21-3ad6fdd63c53-extensions-socket-volume\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:10:44.472638 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:44.472615 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72cb7106-3133-4e0d-8b21-3ad6fdd63c53-kube-api-access-9pg25" (OuterVolumeSpecName: "kube-api-access-9pg25") pod "72cb7106-3133-4e0d-8b21-3ad6fdd63c53" (UID: "72cb7106-3133-4e0d-8b21-3ad6fdd63c53"). InnerVolumeSpecName "kube-api-access-9pg25". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:10:44.572104 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:44.572069 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxxvx\" (UniqueName: \"kubernetes.io/projected/fcc04ee9-b14a-472a-81d0-b307a428b1c2-kube-api-access-hxxvx\") pod \"kuadrant-operator-controller-manager-86878c769f-phnk9\" (UID: \"fcc04ee9-b14a-472a-81d0-b307a428b1c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phnk9" Jul 14 02:10:44.572104 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:44.572108 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fcc04ee9-b14a-472a-81d0-b307a428b1c2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-phnk9\" (UID: \"fcc04ee9-b14a-472a-81d0-b307a428b1c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phnk9" Jul 14 02:10:44.572330 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:44.572193 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9pg25\" (UniqueName: \"kubernetes.io/projected/72cb7106-3133-4e0d-8b21-3ad6fdd63c53-kube-api-access-9pg25\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:10:44.572533 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:44.572515 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fcc04ee9-b14a-472a-81d0-b307a428b1c2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-phnk9\" (UID: \"fcc04ee9-b14a-472a-81d0-b307a428b1c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phnk9" Jul 14 02:10:44.581618 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:44.581593 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxxvx\" (UniqueName: \"kubernetes.io/projected/fcc04ee9-b14a-472a-81d0-b307a428b1c2-kube-api-access-hxxvx\") pod \"kuadrant-operator-controller-manager-86878c769f-phnk9\" (UID: \"fcc04ee9-b14a-472a-81d0-b307a428b1c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phnk9" Jul 14 02:10:44.620659 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:44.620628 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phnk9" Jul 14 02:10:44.708200 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:44.708169 2577 generic.go:356] "Generic (PLEG): container finished" podID="72cb7106-3133-4e0d-8b21-3ad6fdd63c53" containerID="132b19f2c9f6809348ed8b0617f181f05963ad76201a620e35a6d6bd7290cbbe" exitCode=0 Jul 14 02:10:44.708366 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:44.708297 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rr4qq" Jul 14 02:10:44.708366 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:44.708306 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rr4qq" event={"ID":"72cb7106-3133-4e0d-8b21-3ad6fdd63c53","Type":"ContainerDied","Data":"132b19f2c9f6809348ed8b0617f181f05963ad76201a620e35a6d6bd7290cbbe"} Jul 14 02:10:44.708366 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:44.708360 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rr4qq" event={"ID":"72cb7106-3133-4e0d-8b21-3ad6fdd63c53","Type":"ContainerDied","Data":"030c606097dce92bf930a9e0ac000dcee4c94a9b08964a4a23a07c0a85f08f8f"} Jul 14 02:10:44.708526 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:44.708386 2577 scope.go:117] "RemoveContainer" containerID="132b19f2c9f6809348ed8b0617f181f05963ad76201a620e35a6d6bd7290cbbe" Jul 14 02:10:44.724755 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:44.724659 2577 scope.go:117] "RemoveContainer" containerID="132b19f2c9f6809348ed8b0617f181f05963ad76201a620e35a6d6bd7290cbbe" Jul 14 02:10:44.725424 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:10:44.725211 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"132b19f2c9f6809348ed8b0617f181f05963ad76201a620e35a6d6bd7290cbbe\": container with ID starting with 132b19f2c9f6809348ed8b0617f181f05963ad76201a620e35a6d6bd7290cbbe not found: ID does not exist" containerID="132b19f2c9f6809348ed8b0617f181f05963ad76201a620e35a6d6bd7290cbbe" Jul 14 02:10:44.725525 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:44.725453 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"132b19f2c9f6809348ed8b0617f181f05963ad76201a620e35a6d6bd7290cbbe"} err="failed to get container status \"132b19f2c9f6809348ed8b0617f181f05963ad76201a620e35a6d6bd7290cbbe\": rpc error: code = NotFound desc = could not find container \"132b19f2c9f6809348ed8b0617f181f05963ad76201a620e35a6d6bd7290cbbe\": container with ID starting with 132b19f2c9f6809348ed8b0617f181f05963ad76201a620e35a6d6bd7290cbbe not found: ID does not exist" Jul 14 02:10:44.737319 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:44.737286 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rr4qq"] Jul 14 02:10:44.741410 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:44.741379 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rr4qq"] Jul 14 02:10:44.761447 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:44.761421 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phnk9"] Jul 14 02:10:44.764430 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:10:44.764401 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcc04ee9_b14a_472a_81d0_b307a428b1c2.slice/crio-46006add1301a099939287434c0980f09e1fd5075bac19f1c9ab723a319e6c9a WatchSource:0}: Error finding container 46006add1301a099939287434c0980f09e1fd5075bac19f1c9ab723a319e6c9a: Status 404 returned error can't find the container with id 46006add1301a099939287434c0980f09e1fd5075bac19f1c9ab723a319e6c9a Jul 14 02:10:45.714223 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:45.714182 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phnk9" event={"ID":"fcc04ee9-b14a-472a-81d0-b307a428b1c2","Type":"ContainerStarted","Data":"4484c4a1fe858068a18953942a7f081f8001e582a8e85e6dd18854efc2d002ae"} Jul 14 02:10:45.714713 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:45.714312 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phnk9" Jul 14 02:10:45.714713 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:45.714349 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phnk9" event={"ID":"fcc04ee9-b14a-472a-81d0-b307a428b1c2","Type":"ContainerStarted","Data":"46006add1301a099939287434c0980f09e1fd5075bac19f1c9ab723a319e6c9a"} Jul 14 02:10:45.750939 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:45.750889 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phnk9" podStartSLOduration=1.750872426 podStartE2EDuration="1.750872426s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:10:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:10:45.750297171 +0000 UTC m=+547.905133819" watchObservedRunningTime="2026-07-14 02:10:45.750872426 +0000 UTC m=+547.905709074" Jul 14 02:10:46.264818 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:46.264779 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72cb7106-3133-4e0d-8b21-3ad6fdd63c53" path="/var/lib/kubelet/pods/72cb7106-3133-4e0d-8b21-3ad6fdd63c53/volumes" Jul 14 02:10:56.722852 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:10:56.722820 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phnk9" Jul 14 02:11:00.429730 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.429697 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7"] Jul 14 02:11:00.430093 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.430045 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72cb7106-3133-4e0d-8b21-3ad6fdd63c53" containerName="manager" Jul 14 02:11:00.430093 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.430057 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="72cb7106-3133-4e0d-8b21-3ad6fdd63c53" containerName="manager" Jul 14 02:11:00.430162 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.430114 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="72cb7106-3133-4e0d-8b21-3ad6fdd63c53" containerName="manager" Jul 14 02:11:00.433453 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.433436 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" Jul 14 02:11:00.435579 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.435553 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-6qx6n\"" Jul 14 02:11:00.444924 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.444900 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7"] Jul 14 02:11:00.454539 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.454507 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:11:00.530233 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.530186 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:11:00.615693 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.615655 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a993e36-54b0-48f9-876d-170e53d0794a-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-fdsj7\" (UID: \"8a993e36-54b0-48f9-876d-170e53d0794a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" Jul 14 02:11:00.615896 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.615703 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a993e36-54b0-48f9-876d-170e53d0794a-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-fdsj7\" (UID: \"8a993e36-54b0-48f9-876d-170e53d0794a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" Jul 14 02:11:00.615896 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.615792 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a993e36-54b0-48f9-876d-170e53d0794a-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-fdsj7\" (UID: \"8a993e36-54b0-48f9-876d-170e53d0794a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" Jul 14 02:11:00.615896 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.615835 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a993e36-54b0-48f9-876d-170e53d0794a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-fdsj7\" (UID: \"8a993e36-54b0-48f9-876d-170e53d0794a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" Jul 14 02:11:00.615896 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.615862 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tczrp\" (UniqueName: \"kubernetes.io/projected/8a993e36-54b0-48f9-876d-170e53d0794a-kube-api-access-tczrp\") pod \"maas-default-gateway-openshift-default-56654dcf7-fdsj7\" (UID: \"8a993e36-54b0-48f9-876d-170e53d0794a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" Jul 14 02:11:00.616099 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.615907 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a993e36-54b0-48f9-876d-170e53d0794a-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-fdsj7\" (UID: \"8a993e36-54b0-48f9-876d-170e53d0794a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" Jul 14 02:11:00.616099 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.615968 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a993e36-54b0-48f9-876d-170e53d0794a-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-fdsj7\" (UID: \"8a993e36-54b0-48f9-876d-170e53d0794a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" Jul 14 02:11:00.616099 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.615991 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a993e36-54b0-48f9-876d-170e53d0794a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-fdsj7\" (UID: \"8a993e36-54b0-48f9-876d-170e53d0794a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" Jul 14 02:11:00.616099 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.616041 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a993e36-54b0-48f9-876d-170e53d0794a-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-fdsj7\" (UID: \"8a993e36-54b0-48f9-876d-170e53d0794a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" Jul 14 02:11:00.716810 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.716711 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a993e36-54b0-48f9-876d-170e53d0794a-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-fdsj7\" (UID: \"8a993e36-54b0-48f9-876d-170e53d0794a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" Jul 14 02:11:00.716810 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.716760 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a993e36-54b0-48f9-876d-170e53d0794a-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-fdsj7\" (UID: \"8a993e36-54b0-48f9-876d-170e53d0794a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" Jul 14 02:11:00.716810 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.716778 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a993e36-54b0-48f9-876d-170e53d0794a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-fdsj7\" (UID: \"8a993e36-54b0-48f9-876d-170e53d0794a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" Jul 14 02:11:00.716810 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.716803 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tczrp\" (UniqueName: \"kubernetes.io/projected/8a993e36-54b0-48f9-876d-170e53d0794a-kube-api-access-tczrp\") pod \"maas-default-gateway-openshift-default-56654dcf7-fdsj7\" (UID: \"8a993e36-54b0-48f9-876d-170e53d0794a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" Jul 14 02:11:00.717146 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.716985 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a993e36-54b0-48f9-876d-170e53d0794a-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-fdsj7\" (UID: \"8a993e36-54b0-48f9-876d-170e53d0794a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" Jul 14 02:11:00.717146 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.717035 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a993e36-54b0-48f9-876d-170e53d0794a-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-fdsj7\" (UID: \"8a993e36-54b0-48f9-876d-170e53d0794a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" Jul 14 02:11:00.717146 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.717064 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a993e36-54b0-48f9-876d-170e53d0794a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-fdsj7\" (UID: \"8a993e36-54b0-48f9-876d-170e53d0794a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" Jul 14 02:11:00.717146 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.717105 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a993e36-54b0-48f9-876d-170e53d0794a-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-fdsj7\" (UID: \"8a993e36-54b0-48f9-876d-170e53d0794a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" Jul 14 02:11:00.717146 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.717123 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a993e36-54b0-48f9-876d-170e53d0794a-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-fdsj7\" (UID: \"8a993e36-54b0-48f9-876d-170e53d0794a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" Jul 14 02:11:00.717415 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.717182 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a993e36-54b0-48f9-876d-170e53d0794a-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-fdsj7\" (UID: \"8a993e36-54b0-48f9-876d-170e53d0794a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" Jul 14 02:11:00.717415 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.717184 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a993e36-54b0-48f9-876d-170e53d0794a-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-fdsj7\" (UID: \"8a993e36-54b0-48f9-876d-170e53d0794a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" Jul 14 02:11:00.717415 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.717374 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a993e36-54b0-48f9-876d-170e53d0794a-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-fdsj7\" (UID: \"8a993e36-54b0-48f9-876d-170e53d0794a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" Jul 14 02:11:00.717687 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.717666 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a993e36-54b0-48f9-876d-170e53d0794a-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-fdsj7\" (UID: \"8a993e36-54b0-48f9-876d-170e53d0794a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" Jul 14 02:11:00.717792 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.717775 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a993e36-54b0-48f9-876d-170e53d0794a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-fdsj7\" (UID: \"8a993e36-54b0-48f9-876d-170e53d0794a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" Jul 14 02:11:00.719121 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.719099 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a993e36-54b0-48f9-876d-170e53d0794a-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-fdsj7\" (UID: \"8a993e36-54b0-48f9-876d-170e53d0794a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" Jul 14 02:11:00.719246 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.719231 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a993e36-54b0-48f9-876d-170e53d0794a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-fdsj7\" (UID: \"8a993e36-54b0-48f9-876d-170e53d0794a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" Jul 14 02:11:00.724528 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.724504 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a993e36-54b0-48f9-876d-170e53d0794a-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-fdsj7\" (UID: \"8a993e36-54b0-48f9-876d-170e53d0794a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" Jul 14 02:11:00.724629 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.724572 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tczrp\" (UniqueName: \"kubernetes.io/projected/8a993e36-54b0-48f9-876d-170e53d0794a-kube-api-access-tczrp\") pod \"maas-default-gateway-openshift-default-56654dcf7-fdsj7\" (UID: \"8a993e36-54b0-48f9-876d-170e53d0794a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" Jul 14 02:11:00.744518 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.744487 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" Jul 14 02:11:00.879187 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.879158 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7"] Jul 14 02:11:00.881172 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:11:00.881146 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a993e36_54b0_48f9_876d_170e53d0794a.slice/crio-851169440b0fd86667a3055ede105a476b07e08c2525d9964bc337a4a6c32956 WatchSource:0}: Error finding container 851169440b0fd86667a3055ede105a476b07e08c2525d9964bc337a4a6c32956: Status 404 returned error can't find the container with id 851169440b0fd86667a3055ede105a476b07e08c2525d9964bc337a4a6c32956 Jul 14 02:11:00.883466 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.883435 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:11:00.883555 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.883504 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:11:00.883555 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:00.883531 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:11:01.791578 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:01.791543 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" event={"ID":"8a993e36-54b0-48f9-876d-170e53d0794a","Type":"ContainerStarted","Data":"cadb90a06ea520220915bce57aeec1a3e728e2b2da2bbae4b442f94791dc7200"} Jul 14 02:11:01.792063 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:01.791591 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" event={"ID":"8a993e36-54b0-48f9-876d-170e53d0794a","Type":"ContainerStarted","Data":"851169440b0fd86667a3055ede105a476b07e08c2525d9964bc337a4a6c32956"} Jul 14 02:11:01.811203 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:01.811152 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" podStartSLOduration=1.8111344489999999 podStartE2EDuration="1.811134449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:11:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:11:01.809608606 +0000 UTC m=+563.964445255" watchObservedRunningTime="2026-07-14 02:11:01.811134449 +0000 UTC m=+563.965971154" Jul 14 02:11:02.745675 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:02.745638 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" Jul 14 02:11:02.750660 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:02.750624 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" Jul 14 02:11:02.796493 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:02.796466 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" Jul 14 02:11:02.797597 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:02.797577 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" Jul 14 02:11:08.620093 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:08.620062 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:11:15.386346 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:15.386312 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:11:15.496852 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:15.496815 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:11:15.551323 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:15.551293 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:11:16.173239 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:16.173203 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-vp22p"] Jul 14 02:11:16.176949 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:16.176932 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vp22p" Jul 14 02:11:16.179059 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:16.179043 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-5cd7c\"" Jul 14 02:11:16.182027 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:16.182009 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-vp22p"] Jul 14 02:11:16.197203 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:16.197157 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:11:16.260745 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:16.260715 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwcrg\" (UniqueName: \"kubernetes.io/projected/370dcc55-feea-4dc4-8145-f46a3d08e959-kube-api-access-kwcrg\") pod \"authorino-58cd955675-vp22p\" (UID: \"370dcc55-feea-4dc4-8145-f46a3d08e959\") " pod="kuadrant-system/authorino-58cd955675-vp22p" Jul 14 02:11:16.298431 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:16.298388 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:11:16.362263 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:16.362224 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwcrg\" (UniqueName: \"kubernetes.io/projected/370dcc55-feea-4dc4-8145-f46a3d08e959-kube-api-access-kwcrg\") pod \"authorino-58cd955675-vp22p\" (UID: \"370dcc55-feea-4dc4-8145-f46a3d08e959\") " pod="kuadrant-system/authorino-58cd955675-vp22p" Jul 14 02:11:16.369607 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:16.369580 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwcrg\" (UniqueName: \"kubernetes.io/projected/370dcc55-feea-4dc4-8145-f46a3d08e959-kube-api-access-kwcrg\") pod \"authorino-58cd955675-vp22p\" (UID: \"370dcc55-feea-4dc4-8145-f46a3d08e959\") " pod="kuadrant-system/authorino-58cd955675-vp22p" Jul 14 02:11:16.490697 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:16.490605 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vp22p" Jul 14 02:11:16.622198 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:16.622172 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-vp22p"] Jul 14 02:11:16.624652 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:11:16.624626 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod370dcc55_feea_4dc4_8145_f46a3d08e959.slice/crio-6318835de479f6188c9cfdd2686fc7c70358951fc424efa718626951b18d8721 WatchSource:0}: Error finding container 6318835de479f6188c9cfdd2686fc7c70358951fc424efa718626951b18d8721: Status 404 returned error can't find the container with id 6318835de479f6188c9cfdd2686fc7c70358951fc424efa718626951b18d8721 Jul 14 02:11:16.862649 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:16.862613 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vp22p" event={"ID":"370dcc55-feea-4dc4-8145-f46a3d08e959","Type":"ContainerStarted","Data":"6318835de479f6188c9cfdd2686fc7c70358951fc424efa718626951b18d8721"} Jul 14 02:11:19.878359 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:19.878324 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vp22p" event={"ID":"370dcc55-feea-4dc4-8145-f46a3d08e959","Type":"ContainerStarted","Data":"f073a489de2b7a80e39365780feb9b33f93eeff5bc717529703cd850cf1d32de"} Jul 14 02:11:19.892263 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:19.892216 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-vp22p" podStartSLOduration=0.85161269 podStartE2EDuration="3.892199923s" totalImagesPullingTime="3.040587233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:11:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:11:19.890860369 +0000 UTC m=+582.045697016" watchObservedRunningTime="2026-07-14 02:11:19.892199923 +0000 UTC m=+582.047036571" Jul 14 02:11:21.082333 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:21.082297 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-vp22p"] Jul 14 02:11:21.893897 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:21.893838 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-vp22p" podUID="370dcc55-feea-4dc4-8145-f46a3d08e959" containerName="authorino" containerID="cri-o://f073a489de2b7a80e39365780feb9b33f93eeff5bc717529703cd850cf1d32de" gracePeriod=30 Jul 14 02:11:22.143590 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:22.143565 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vp22p" Jul 14 02:11:22.217763 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:22.217672 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kwcrg\" (UniqueName: \"kubernetes.io/projected/370dcc55-feea-4dc4-8145-f46a3d08e959-kube-api-access-kwcrg\") pod \"370dcc55-feea-4dc4-8145-f46a3d08e959\" (UID: \"370dcc55-feea-4dc4-8145-f46a3d08e959\") " Jul 14 02:11:22.219825 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:22.219795 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/370dcc55-feea-4dc4-8145-f46a3d08e959-kube-api-access-kwcrg" (OuterVolumeSpecName: "kube-api-access-kwcrg") pod "370dcc55-feea-4dc4-8145-f46a3d08e959" (UID: "370dcc55-feea-4dc4-8145-f46a3d08e959"). InnerVolumeSpecName "kube-api-access-kwcrg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:11:22.319039 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:22.319001 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwcrg\" (UniqueName: \"kubernetes.io/projected/370dcc55-feea-4dc4-8145-f46a3d08e959-kube-api-access-kwcrg\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:11:22.899217 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:22.899181 2577 generic.go:356] "Generic (PLEG): container finished" podID="370dcc55-feea-4dc4-8145-f46a3d08e959" containerID="f073a489de2b7a80e39365780feb9b33f93eeff5bc717529703cd850cf1d32de" exitCode=0 Jul 14 02:11:22.899217 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:22.899221 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vp22p" Jul 14 02:11:22.899468 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:22.899239 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vp22p" event={"ID":"370dcc55-feea-4dc4-8145-f46a3d08e959","Type":"ContainerDied","Data":"f073a489de2b7a80e39365780feb9b33f93eeff5bc717529703cd850cf1d32de"} Jul 14 02:11:22.899468 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:22.899300 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vp22p" event={"ID":"370dcc55-feea-4dc4-8145-f46a3d08e959","Type":"ContainerDied","Data":"6318835de479f6188c9cfdd2686fc7c70358951fc424efa718626951b18d8721"} Jul 14 02:11:22.899468 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:22.899327 2577 scope.go:117] "RemoveContainer" containerID="f073a489de2b7a80e39365780feb9b33f93eeff5bc717529703cd850cf1d32de" Jul 14 02:11:22.907906 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:22.907886 2577 scope.go:117] "RemoveContainer" containerID="f073a489de2b7a80e39365780feb9b33f93eeff5bc717529703cd850cf1d32de" Jul 14 02:11:22.908181 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:11:22.908154 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f073a489de2b7a80e39365780feb9b33f93eeff5bc717529703cd850cf1d32de\": container with ID starting with f073a489de2b7a80e39365780feb9b33f93eeff5bc717529703cd850cf1d32de not found: ID does not exist" containerID="f073a489de2b7a80e39365780feb9b33f93eeff5bc717529703cd850cf1d32de" Jul 14 02:11:22.908237 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:22.908193 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f073a489de2b7a80e39365780feb9b33f93eeff5bc717529703cd850cf1d32de"} err="failed to get container status \"f073a489de2b7a80e39365780feb9b33f93eeff5bc717529703cd850cf1d32de\": rpc error: code = NotFound desc = could not find container \"f073a489de2b7a80e39365780feb9b33f93eeff5bc717529703cd850cf1d32de\": container with ID starting with f073a489de2b7a80e39365780feb9b33f93eeff5bc717529703cd850cf1d32de not found: ID does not exist" Jul 14 02:11:22.913851 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:22.913827 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-vp22p"] Jul 14 02:11:22.917085 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:22.917066 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-vp22p"] Jul 14 02:11:24.264806 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:24.264775 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="370dcc55-feea-4dc4-8145-f46a3d08e959" path="/var/lib/kubelet/pods/370dcc55-feea-4dc4-8145-f46a3d08e959/volumes" Jul 14 02:11:34.280566 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:34.280533 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:11:34.374451 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:34.374416 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:11:34.774198 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:34.774155 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:11:47.697441 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:47.697406 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:11:47.934606 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:47.934569 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-z85r5"] Jul 14 02:11:47.934932 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:47.934919 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="370dcc55-feea-4dc4-8145-f46a3d08e959" containerName="authorino" Jul 14 02:11:47.934983 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:47.934933 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="370dcc55-feea-4dc4-8145-f46a3d08e959" containerName="authorino" Jul 14 02:11:47.934983 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:47.934982 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="370dcc55-feea-4dc4-8145-f46a3d08e959" containerName="authorino" Jul 14 02:11:47.939443 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:47.939418 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-z85r5" Jul 14 02:11:47.941576 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:47.941555 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-5cd7c\"" Jul 14 02:11:47.944523 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:47.944499 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-z85r5"] Jul 14 02:11:47.945367 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:47.945346 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58c77\" (UniqueName: \"kubernetes.io/projected/ac121d0b-bc68-4907-bd3b-71c4fc517d41-kube-api-access-58c77\") pod \"authorino-5d4b67c446-z85r5\" (UID: \"ac121d0b-bc68-4907-bd3b-71c4fc517d41\") " pod="kuadrant-system/authorino-5d4b67c446-z85r5" Jul 14 02:11:47.950199 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:47.950135 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:11:48.045744 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:48.045711 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58c77\" (UniqueName: \"kubernetes.io/projected/ac121d0b-bc68-4907-bd3b-71c4fc517d41-kube-api-access-58c77\") pod \"authorino-5d4b67c446-z85r5\" (UID: \"ac121d0b-bc68-4907-bd3b-71c4fc517d41\") " pod="kuadrant-system/authorino-5d4b67c446-z85r5" Jul 14 02:11:48.053087 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:48.053057 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58c77\" (UniqueName: \"kubernetes.io/projected/ac121d0b-bc68-4907-bd3b-71c4fc517d41-kube-api-access-58c77\") pod \"authorino-5d4b67c446-z85r5\" (UID: \"ac121d0b-bc68-4907-bd3b-71c4fc517d41\") " pod="kuadrant-system/authorino-5d4b67c446-z85r5" Jul 14 02:11:48.165786 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:48.165752 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-z85r5"] Jul 14 02:11:48.166025 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:48.166013 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-z85r5" Jul 14 02:11:48.194032 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:48.194000 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-fb55988d7-mdz54"] Jul 14 02:11:48.197546 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:48.197524 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-fb55988d7-mdz54" Jul 14 02:11:48.202912 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:48.202853 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-fb55988d7-mdz54"] Jul 14 02:11:48.247267 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:48.247112 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbvj4\" (UniqueName: \"kubernetes.io/projected/f553ea91-9ea6-41fc-9114-a8785660967f-kube-api-access-mbvj4\") pod \"authorino-fb55988d7-mdz54\" (UID: \"f553ea91-9ea6-41fc-9114-a8785660967f\") " pod="kuadrant-system/authorino-fb55988d7-mdz54" Jul 14 02:11:48.304562 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:48.304533 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-z85r5"] Jul 14 02:11:48.306308 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:11:48.306260 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podac121d0b_bc68_4907_bd3b_71c4fc517d41.slice/crio-762413998eea3d77fa2ada5e606272ea30919741c8b8dc714362951bda77f2ff WatchSource:0}: Error finding container 762413998eea3d77fa2ada5e606272ea30919741c8b8dc714362951bda77f2ff: Status 404 returned error can't find the container with id 762413998eea3d77fa2ada5e606272ea30919741c8b8dc714362951bda77f2ff Jul 14 02:11:48.348172 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:48.348136 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbvj4\" (UniqueName: \"kubernetes.io/projected/f553ea91-9ea6-41fc-9114-a8785660967f-kube-api-access-mbvj4\") pod \"authorino-fb55988d7-mdz54\" (UID: \"f553ea91-9ea6-41fc-9114-a8785660967f\") " pod="kuadrant-system/authorino-fb55988d7-mdz54" Jul 14 02:11:48.356919 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:48.356892 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbvj4\" (UniqueName: \"kubernetes.io/projected/f553ea91-9ea6-41fc-9114-a8785660967f-kube-api-access-mbvj4\") pod \"authorino-fb55988d7-mdz54\" (UID: \"f553ea91-9ea6-41fc-9114-a8785660967f\") " pod="kuadrant-system/authorino-fb55988d7-mdz54" Jul 14 02:11:48.465862 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:48.465769 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-fb55988d7-mdz54"] Jul 14 02:11:48.466073 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:48.466060 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-fb55988d7-mdz54" Jul 14 02:11:48.489876 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:48.489848 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-669564f77f-r4xbn"] Jul 14 02:11:48.493296 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:48.493261 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-669564f77f-r4xbn" Jul 14 02:11:48.495317 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:48.495294 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 14 02:11:48.499011 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:48.498988 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-669564f77f-r4xbn"] Jul 14 02:11:48.550458 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:48.550415 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rk2bn\" (UniqueName: \"kubernetes.io/projected/c3f9a65c-c505-446a-b0db-9ced2378feb2-kube-api-access-rk2bn\") pod \"authorino-669564f77f-r4xbn\" (UID: \"c3f9a65c-c505-446a-b0db-9ced2378feb2\") " pod="kuadrant-system/authorino-669564f77f-r4xbn" Jul 14 02:11:48.550622 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:48.550464 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c3f9a65c-c505-446a-b0db-9ced2378feb2-tls-cert\") pod \"authorino-669564f77f-r4xbn\" (UID: \"c3f9a65c-c505-446a-b0db-9ced2378feb2\") " pod="kuadrant-system/authorino-669564f77f-r4xbn" Jul 14 02:11:48.597630 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:48.597472 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-fb55988d7-mdz54"] Jul 14 02:11:48.599836 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:11:48.599809 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf553ea91_9ea6_41fc_9114_a8785660967f.slice/crio-8074b08359ba3863e392d8f5b54badda03eea02668146100ccea8a75bd79af51 WatchSource:0}: Error finding container 8074b08359ba3863e392d8f5b54badda03eea02668146100ccea8a75bd79af51: Status 404 returned error can't find the container with id 8074b08359ba3863e392d8f5b54badda03eea02668146100ccea8a75bd79af51 Jul 14 02:11:48.651041 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:48.651008 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk2bn\" (UniqueName: \"kubernetes.io/projected/c3f9a65c-c505-446a-b0db-9ced2378feb2-kube-api-access-rk2bn\") pod \"authorino-669564f77f-r4xbn\" (UID: \"c3f9a65c-c505-446a-b0db-9ced2378feb2\") " pod="kuadrant-system/authorino-669564f77f-r4xbn" Jul 14 02:11:48.651041 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:48.651047 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c3f9a65c-c505-446a-b0db-9ced2378feb2-tls-cert\") pod \"authorino-669564f77f-r4xbn\" (UID: \"c3f9a65c-c505-446a-b0db-9ced2378feb2\") " pod="kuadrant-system/authorino-669564f77f-r4xbn" Jul 14 02:11:48.653561 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:48.653537 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c3f9a65c-c505-446a-b0db-9ced2378feb2-tls-cert\") pod \"authorino-669564f77f-r4xbn\" (UID: \"c3f9a65c-c505-446a-b0db-9ced2378feb2\") " pod="kuadrant-system/authorino-669564f77f-r4xbn" Jul 14 02:11:48.658466 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:48.658434 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rk2bn\" (UniqueName: \"kubernetes.io/projected/c3f9a65c-c505-446a-b0db-9ced2378feb2-kube-api-access-rk2bn\") pod \"authorino-669564f77f-r4xbn\" (UID: \"c3f9a65c-c505-446a-b0db-9ced2378feb2\") " pod="kuadrant-system/authorino-669564f77f-r4xbn" Jul 14 02:11:48.805452 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:48.805366 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-669564f77f-r4xbn" Jul 14 02:11:48.938088 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:48.937915 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-669564f77f-r4xbn"] Jul 14 02:11:48.940536 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:11:48.940507 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc3f9a65c_c505_446a_b0db_9ced2378feb2.slice/crio-61d751c8904bbcb1a4fb72e01373384640961f1c2d4bd4aef9fc0aba77bfbacc WatchSource:0}: Error finding container 61d751c8904bbcb1a4fb72e01373384640961f1c2d4bd4aef9fc0aba77bfbacc: Status 404 returned error can't find the container with id 61d751c8904bbcb1a4fb72e01373384640961f1c2d4bd4aef9fc0aba77bfbacc Jul 14 02:11:49.026942 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:49.026907 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-fb55988d7-mdz54" event={"ID":"f553ea91-9ea6-41fc-9114-a8785660967f","Type":"ContainerStarted","Data":"de172a0bf057ec89b780cd5a1623bcb6a8412cdb67215d29abc6b4b2d74c8170"} Jul 14 02:11:49.026942 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:49.026923 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-fb55988d7-mdz54" podUID="f553ea91-9ea6-41fc-9114-a8785660967f" containerName="authorino" containerID="cri-o://de172a0bf057ec89b780cd5a1623bcb6a8412cdb67215d29abc6b4b2d74c8170" gracePeriod=30 Jul 14 02:11:49.026942 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:49.026954 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-fb55988d7-mdz54" event={"ID":"f553ea91-9ea6-41fc-9114-a8785660967f","Type":"ContainerStarted","Data":"8074b08359ba3863e392d8f5b54badda03eea02668146100ccea8a75bd79af51"} Jul 14 02:11:49.028755 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:49.028600 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-z85r5" event={"ID":"ac121d0b-bc68-4907-bd3b-71c4fc517d41","Type":"ContainerStarted","Data":"3f59785cb97a3e68a25d0631b2ef1743a921fd0c86ba3e84d71c7be6ee54d451"} Jul 14 02:11:49.028755 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:49.028629 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-z85r5" podUID="ac121d0b-bc68-4907-bd3b-71c4fc517d41" containerName="authorino" containerID="cri-o://3f59785cb97a3e68a25d0631b2ef1743a921fd0c86ba3e84d71c7be6ee54d451" gracePeriod=30 Jul 14 02:11:49.028755 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:49.028648 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-z85r5" event={"ID":"ac121d0b-bc68-4907-bd3b-71c4fc517d41","Type":"ContainerStarted","Data":"762413998eea3d77fa2ada5e606272ea30919741c8b8dc714362951bda77f2ff"} Jul 14 02:11:49.030101 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:49.030072 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-669564f77f-r4xbn" event={"ID":"c3f9a65c-c505-446a-b0db-9ced2378feb2","Type":"ContainerStarted","Data":"61d751c8904bbcb1a4fb72e01373384640961f1c2d4bd4aef9fc0aba77bfbacc"} Jul 14 02:11:49.042236 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:49.042192 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-fb55988d7-mdz54" podStartSLOduration=0.719122513 podStartE2EDuration="1.042178542s" totalImagesPullingTime="323.056029ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:11:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:11:49.04047449 +0000 UTC m=+611.195311140" watchObservedRunningTime="2026-07-14 02:11:49.042178542 +0000 UTC m=+611.197015190" Jul 14 02:11:49.054067 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:49.054024 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-z85r5" podStartSLOduration=1.701673137 podStartE2EDuration="2.054009635s" totalImagesPullingTime="352.336498ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:11:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:11:49.052810994 +0000 UTC m=+611.207647643" watchObservedRunningTime="2026-07-14 02:11:49.054009635 +0000 UTC m=+611.208846282" Jul 14 02:11:49.320453 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:49.320429 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-z85r5" Jul 14 02:11:49.323682 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:49.323660 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-fb55988d7-mdz54" Jul 14 02:11:49.456676 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:49.456644 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-58c77\" (UniqueName: \"kubernetes.io/projected/ac121d0b-bc68-4907-bd3b-71c4fc517d41-kube-api-access-58c77\") pod \"ac121d0b-bc68-4907-bd3b-71c4fc517d41\" (UID: \"ac121d0b-bc68-4907-bd3b-71c4fc517d41\") " Jul 14 02:11:49.456861 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:49.456724 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbvj4\" (UniqueName: \"kubernetes.io/projected/f553ea91-9ea6-41fc-9114-a8785660967f-kube-api-access-mbvj4\") pod \"f553ea91-9ea6-41fc-9114-a8785660967f\" (UID: \"f553ea91-9ea6-41fc-9114-a8785660967f\") " Jul 14 02:11:49.458687 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:49.458656 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac121d0b-bc68-4907-bd3b-71c4fc517d41-kube-api-access-58c77" (OuterVolumeSpecName: "kube-api-access-58c77") pod "ac121d0b-bc68-4907-bd3b-71c4fc517d41" (UID: "ac121d0b-bc68-4907-bd3b-71c4fc517d41"). InnerVolumeSpecName "kube-api-access-58c77". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:11:49.458833 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:49.458752 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f553ea91-9ea6-41fc-9114-a8785660967f-kube-api-access-mbvj4" (OuterVolumeSpecName: "kube-api-access-mbvj4") pod "f553ea91-9ea6-41fc-9114-a8785660967f" (UID: "f553ea91-9ea6-41fc-9114-a8785660967f"). InnerVolumeSpecName "kube-api-access-mbvj4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:11:49.557569 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:49.557540 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbvj4\" (UniqueName: \"kubernetes.io/projected/f553ea91-9ea6-41fc-9114-a8785660967f-kube-api-access-mbvj4\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:11:49.557569 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:49.557566 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58c77\" (UniqueName: \"kubernetes.io/projected/ac121d0b-bc68-4907-bd3b-71c4fc517d41-kube-api-access-58c77\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:11:50.036457 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:50.036421 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-669564f77f-r4xbn" event={"ID":"c3f9a65c-c505-446a-b0db-9ced2378feb2","Type":"ContainerStarted","Data":"d057aa42ddbba328b25bc62520029800681a63247f8a15a053c1f6dcd0528d29"} Jul 14 02:11:50.037612 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:50.037586 2577 generic.go:356] "Generic (PLEG): container finished" podID="f553ea91-9ea6-41fc-9114-a8785660967f" containerID="de172a0bf057ec89b780cd5a1623bcb6a8412cdb67215d29abc6b4b2d74c8170" exitCode=0 Jul 14 02:11:50.037734 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:50.037639 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-fb55988d7-mdz54" Jul 14 02:11:50.037734 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:50.037670 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-fb55988d7-mdz54" event={"ID":"f553ea91-9ea6-41fc-9114-a8785660967f","Type":"ContainerDied","Data":"de172a0bf057ec89b780cd5a1623bcb6a8412cdb67215d29abc6b4b2d74c8170"} Jul 14 02:11:50.037734 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:50.037711 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-fb55988d7-mdz54" event={"ID":"f553ea91-9ea6-41fc-9114-a8785660967f","Type":"ContainerDied","Data":"8074b08359ba3863e392d8f5b54badda03eea02668146100ccea8a75bd79af51"} Jul 14 02:11:50.037734 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:50.037733 2577 scope.go:117] "RemoveContainer" containerID="de172a0bf057ec89b780cd5a1623bcb6a8412cdb67215d29abc6b4b2d74c8170" Jul 14 02:11:50.038881 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:50.038864 2577 generic.go:356] "Generic (PLEG): container finished" podID="ac121d0b-bc68-4907-bd3b-71c4fc517d41" containerID="3f59785cb97a3e68a25d0631b2ef1743a921fd0c86ba3e84d71c7be6ee54d451" exitCode=0 Jul 14 02:11:50.038953 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:50.038902 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-z85r5" event={"ID":"ac121d0b-bc68-4907-bd3b-71c4fc517d41","Type":"ContainerDied","Data":"3f59785cb97a3e68a25d0631b2ef1743a921fd0c86ba3e84d71c7be6ee54d451"} Jul 14 02:11:50.038953 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:50.038919 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-z85r5" Jul 14 02:11:50.039059 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:50.038924 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-z85r5" event={"ID":"ac121d0b-bc68-4907-bd3b-71c4fc517d41","Type":"ContainerDied","Data":"762413998eea3d77fa2ada5e606272ea30919741c8b8dc714362951bda77f2ff"} Jul 14 02:11:50.047383 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:50.047166 2577 scope.go:117] "RemoveContainer" containerID="de172a0bf057ec89b780cd5a1623bcb6a8412cdb67215d29abc6b4b2d74c8170" Jul 14 02:11:50.047617 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:11:50.047588 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de172a0bf057ec89b780cd5a1623bcb6a8412cdb67215d29abc6b4b2d74c8170\": container with ID starting with de172a0bf057ec89b780cd5a1623bcb6a8412cdb67215d29abc6b4b2d74c8170 not found: ID does not exist" containerID="de172a0bf057ec89b780cd5a1623bcb6a8412cdb67215d29abc6b4b2d74c8170" Jul 14 02:11:50.047710 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:50.047627 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de172a0bf057ec89b780cd5a1623bcb6a8412cdb67215d29abc6b4b2d74c8170"} err="failed to get container status \"de172a0bf057ec89b780cd5a1623bcb6a8412cdb67215d29abc6b4b2d74c8170\": rpc error: code = NotFound desc = could not find container \"de172a0bf057ec89b780cd5a1623bcb6a8412cdb67215d29abc6b4b2d74c8170\": container with ID starting with de172a0bf057ec89b780cd5a1623bcb6a8412cdb67215d29abc6b4b2d74c8170 not found: ID does not exist" Jul 14 02:11:50.047710 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:50.047655 2577 scope.go:117] "RemoveContainer" containerID="3f59785cb97a3e68a25d0631b2ef1743a921fd0c86ba3e84d71c7be6ee54d451" Jul 14 02:11:50.051111 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:50.051072 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-669564f77f-r4xbn" podStartSLOduration=1.667280398 podStartE2EDuration="2.051060501s" totalImagesPullingTime="383.780103ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:11:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:11:50.049666135 +0000 UTC m=+612.204502784" watchObservedRunningTime="2026-07-14 02:11:50.051060501 +0000 UTC m=+612.205897149" Jul 14 02:11:50.055921 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:50.055904 2577 scope.go:117] "RemoveContainer" containerID="3f59785cb97a3e68a25d0631b2ef1743a921fd0c86ba3e84d71c7be6ee54d451" Jul 14 02:11:50.056178 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:11:50.056163 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f59785cb97a3e68a25d0631b2ef1743a921fd0c86ba3e84d71c7be6ee54d451\": container with ID starting with 3f59785cb97a3e68a25d0631b2ef1743a921fd0c86ba3e84d71c7be6ee54d451 not found: ID does not exist" containerID="3f59785cb97a3e68a25d0631b2ef1743a921fd0c86ba3e84d71c7be6ee54d451" Jul 14 02:11:50.056228 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:50.056184 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f59785cb97a3e68a25d0631b2ef1743a921fd0c86ba3e84d71c7be6ee54d451"} err="failed to get container status \"3f59785cb97a3e68a25d0631b2ef1743a921fd0c86ba3e84d71c7be6ee54d451\": rpc error: code = NotFound desc = could not find container \"3f59785cb97a3e68a25d0631b2ef1743a921fd0c86ba3e84d71c7be6ee54d451\": container with ID starting with 3f59785cb97a3e68a25d0631b2ef1743a921fd0c86ba3e84d71c7be6ee54d451 not found: ID does not exist" Jul 14 02:11:50.069210 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:50.069180 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-fb55988d7-mdz54"] Jul 14 02:11:50.078542 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:50.078516 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-fb55988d7-mdz54"] Jul 14 02:11:50.091189 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:50.091160 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-z85r5"] Jul 14 02:11:50.096594 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:50.096570 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-z85r5"] Jul 14 02:11:50.265087 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:50.265046 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac121d0b-bc68-4907-bd3b-71c4fc517d41" path="/var/lib/kubelet/pods/ac121d0b-bc68-4907-bd3b-71c4fc517d41/volumes" Jul 14 02:11:50.265481 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:50.265460 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f553ea91-9ea6-41fc-9114-a8785660967f" path="/var/lib/kubelet/pods/f553ea91-9ea6-41fc-9114-a8785660967f/volumes" Jul 14 02:11:54.686061 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:54.686027 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:11:54.759569 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:54.759536 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:11:54.815732 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:54.815699 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:11:54.939072 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:54.938992 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:11:57.552682 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:57.552646 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:11:57.644148 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:57.644116 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:11:57.868292 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:11:57.868247 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:12:06.651788 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:12:06.651749 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:12:11.281449 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:12:11.281392 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:12:11.337519 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:12:11.337489 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:12:18.675494 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:12:18.675459 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:12:29.700573 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:12:29.700539 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:12:30.162894 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:12:30.162843 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:12:30.269191 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:12:30.269163 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:12:30.623211 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:12:30.623182 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:12:31.370502 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:12:31.370470 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:12:31.470711 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:12:31.470679 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:12:39.277983 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:12:39.277948 2577 generic.go:356] "Generic (PLEG): container finished" podID="8a993e36-54b0-48f9-876d-170e53d0794a" containerID="cadb90a06ea520220915bce57aeec1a3e728e2b2da2bbae4b442f94791dc7200" exitCode=0 Jul 14 02:12:39.278410 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:12:39.278020 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" event={"ID":"8a993e36-54b0-48f9-876d-170e53d0794a","Type":"ContainerDied","Data":"cadb90a06ea520220915bce57aeec1a3e728e2b2da2bbae4b442f94791dc7200"} Jul 14 02:12:39.278456 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:12:39.278415 2577 scope.go:117] "RemoveContainer" containerID="cadb90a06ea520220915bce57aeec1a3e728e2b2da2bbae4b442f94791dc7200" Jul 14 02:12:39.279434 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:12:39.279408 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:12:39.279523 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:12:39.279475 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:12:39.279523 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:12:39.279511 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:12:40.284531 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:12:40.284496 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" event={"ID":"8a993e36-54b0-48f9-876d-170e53d0794a","Type":"ContainerStarted","Data":"bca7c131c86dceb01a45b32d8f23cec18c070bd3451098ea070e93a37c7f9199"} Jul 14 02:12:40.745729 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:12:40.745225 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" Jul 14 02:12:40.745729 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:12:40.745393 2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-fdsj7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Jul 14 02:12:40.745729 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:12:40.745453 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" podUID="8a993e36-54b0-48f9-876d-170e53d0794a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Jul 14 02:12:41.585807 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:12:41.585766 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:12:41.687643 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:12:41.687610 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:12:41.745607 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:12:41.745562 2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-fdsj7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Jul 14 02:12:41.745820 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:12:41.745670 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" podUID="8a993e36-54b0-48f9-876d-170e53d0794a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Jul 14 02:12:42.746224 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:12:42.746184 2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-fdsj7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Jul 14 02:12:42.746843 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:12:42.746805 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" podUID="8a993e36-54b0-48f9-876d-170e53d0794a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Jul 14 02:12:43.745717 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:12:43.745668 2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-fdsj7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Jul 14 02:12:43.745945 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:12:43.745742 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" podUID="8a993e36-54b0-48f9-876d-170e53d0794a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Jul 14 02:12:44.749361 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:12:44.749322 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" Jul 14 02:12:44.749808 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:12:44.749640 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" Jul 14 02:12:44.751285 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:12:44.751256 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fdsj7" Jul 14 02:12:53.291093 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:12:53.291058 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:13:08.195743 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:08.195712 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-5fgjt"] Jul 14 02:13:08.196119 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:08.196097 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f553ea91-9ea6-41fc-9114-a8785660967f" containerName="authorino" Jul 14 02:13:08.196119 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:08.196109 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f553ea91-9ea6-41fc-9114-a8785660967f" containerName="authorino" Jul 14 02:13:08.196190 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:08.196126 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac121d0b-bc68-4907-bd3b-71c4fc517d41" containerName="authorino" Jul 14 02:13:08.196190 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:08.196133 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac121d0b-bc68-4907-bd3b-71c4fc517d41" containerName="authorino" Jul 14 02:13:08.196293 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:08.196191 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f553ea91-9ea6-41fc-9114-a8785660967f" containerName="authorino" Jul 14 02:13:08.196293 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:08.196205 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac121d0b-bc68-4907-bd3b-71c4fc517d41" containerName="authorino" Jul 14 02:13:08.199126 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:08.199107 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5fgjt" Jul 14 02:13:08.201921 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:08.201900 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 14 02:13:08.202051 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:08.201899 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 14 02:13:08.202051 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:08.201900 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-bxqsh\"" Jul 14 02:13:08.202051 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:08.201949 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 14 02:13:08.207392 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:08.207370 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-5fgjt"] Jul 14 02:13:08.222453 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:08.222419 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:13:08.279961 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:08.279926 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f365efbc-4564-439a-80a3-e4a30903bce1-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5fgjt\" (UID: \"f365efbc-4564-439a-80a3-e4a30903bce1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5fgjt" Jul 14 02:13:08.279961 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:08.279965 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f365efbc-4564-439a-80a3-e4a30903bce1-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5fgjt\" (UID: \"f365efbc-4564-439a-80a3-e4a30903bce1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5fgjt" Jul 14 02:13:08.280225 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:08.279984 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f365efbc-4564-439a-80a3-e4a30903bce1-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5fgjt\" (UID: \"f365efbc-4564-439a-80a3-e4a30903bce1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5fgjt" Jul 14 02:13:08.280225 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:08.280060 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpzf9\" (UniqueName: \"kubernetes.io/projected/f365efbc-4564-439a-80a3-e4a30903bce1-kube-api-access-xpzf9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5fgjt\" (UID: \"f365efbc-4564-439a-80a3-e4a30903bce1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5fgjt" Jul 14 02:13:08.280225 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:08.280174 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f365efbc-4564-439a-80a3-e4a30903bce1-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5fgjt\" (UID: \"f365efbc-4564-439a-80a3-e4a30903bce1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5fgjt" Jul 14 02:13:08.280225 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:08.280222 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f365efbc-4564-439a-80a3-e4a30903bce1-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5fgjt\" (UID: \"f365efbc-4564-439a-80a3-e4a30903bce1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5fgjt" Jul 14 02:13:08.380768 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:08.380736 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f365efbc-4564-439a-80a3-e4a30903bce1-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5fgjt\" (UID: \"f365efbc-4564-439a-80a3-e4a30903bce1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5fgjt" Jul 14 02:13:08.380768 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:08.380785 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f365efbc-4564-439a-80a3-e4a30903bce1-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5fgjt\" (UID: \"f365efbc-4564-439a-80a3-e4a30903bce1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5fgjt" Jul 14 02:13:08.381056 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:08.381029 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f365efbc-4564-439a-80a3-e4a30903bce1-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5fgjt\" (UID: \"f365efbc-4564-439a-80a3-e4a30903bce1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5fgjt" Jul 14 02:13:08.381125 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:08.381073 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f365efbc-4564-439a-80a3-e4a30903bce1-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5fgjt\" (UID: \"f365efbc-4564-439a-80a3-e4a30903bce1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5fgjt" Jul 14 02:13:08.381125 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:08.381098 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f365efbc-4564-439a-80a3-e4a30903bce1-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5fgjt\" (UID: \"f365efbc-4564-439a-80a3-e4a30903bce1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5fgjt" Jul 14 02:13:08.381243 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:08.381162 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpzf9\" (UniqueName: \"kubernetes.io/projected/f365efbc-4564-439a-80a3-e4a30903bce1-kube-api-access-xpzf9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5fgjt\" (UID: \"f365efbc-4564-439a-80a3-e4a30903bce1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5fgjt" Jul 14 02:13:08.381568 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:08.381543 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f365efbc-4564-439a-80a3-e4a30903bce1-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5fgjt\" (UID: \"f365efbc-4564-439a-80a3-e4a30903bce1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5fgjt" Jul 14 02:13:08.381804 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:08.381780 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f365efbc-4564-439a-80a3-e4a30903bce1-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5fgjt\" (UID: \"f365efbc-4564-439a-80a3-e4a30903bce1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5fgjt" Jul 14 02:13:08.381925 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:08.381822 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f365efbc-4564-439a-80a3-e4a30903bce1-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5fgjt\" (UID: \"f365efbc-4564-439a-80a3-e4a30903bce1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5fgjt" Jul 14 02:13:08.383418 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:08.383394 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f365efbc-4564-439a-80a3-e4a30903bce1-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5fgjt\" (UID: \"f365efbc-4564-439a-80a3-e4a30903bce1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5fgjt" Jul 14 02:13:08.383689 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:08.383668 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f365efbc-4564-439a-80a3-e4a30903bce1-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5fgjt\" (UID: \"f365efbc-4564-439a-80a3-e4a30903bce1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5fgjt" Jul 14 02:13:08.389493 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:08.389467 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpzf9\" (UniqueName: \"kubernetes.io/projected/f365efbc-4564-439a-80a3-e4a30903bce1-kube-api-access-xpzf9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5fgjt\" (UID: \"f365efbc-4564-439a-80a3-e4a30903bce1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5fgjt" Jul 14 02:13:08.510565 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:08.510469 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5fgjt" Jul 14 02:13:08.651450 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:08.651423 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-5fgjt"] Jul 14 02:13:08.653145 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:13:08.653116 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf365efbc_4564_439a_80a3_e4a30903bce1.slice/crio-576c4adb8ddf8d9b67fa661e172c18975e1f80bc0b363a32061a6615f6cc2997 WatchSource:0}: Error finding container 576c4adb8ddf8d9b67fa661e172c18975e1f80bc0b363a32061a6615f6cc2997: Status 404 returned error can't find the container with id 576c4adb8ddf8d9b67fa661e172c18975e1f80bc0b363a32061a6615f6cc2997 Jul 14 02:13:08.654770 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:08.654748 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 02:13:09.424148 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:09.424103 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5fgjt" event={"ID":"f365efbc-4564-439a-80a3-e4a30903bce1","Type":"ContainerStarted","Data":"576c4adb8ddf8d9b67fa661e172c18975e1f80bc0b363a32061a6615f6cc2997"} Jul 14 02:13:10.432231 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:10.432197 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5fgjt" event={"ID":"f365efbc-4564-439a-80a3-e4a30903bce1","Type":"ContainerStarted","Data":"343c16506ededccfa6caddcaed1a99c45aa415ff00ac827787d8a1c75a42e44b"} Jul 14 02:13:10.450994 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:10.450941 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5fgjt" podStartSLOduration=1.365624065 podStartE2EDuration="2.450923068s" totalImagesPullingTime="1.085299003s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:13:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:13:10.449431055 +0000 UTC m=+692.604267716" watchObservedRunningTime="2026-07-14 02:13:10.450923068 +0000 UTC m=+692.605759716" Jul 14 02:13:16.185118 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:16.185073 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2vvsp"] Jul 14 02:13:16.189094 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:16.189067 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2vvsp" Jul 14 02:13:16.191200 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:16.191172 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 14 02:13:16.196847 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:16.196824 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2vvsp"] Jul 14 02:13:16.214284 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:16.214238 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:13:16.358745 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:16.358711 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33de58ad-f50a-45c9-9475-661288f6233d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2vvsp\" (UID: \"33de58ad-f50a-45c9-9475-661288f6233d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2vvsp" Jul 14 02:13:16.358938 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:16.358786 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33de58ad-f50a-45c9-9475-661288f6233d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2vvsp\" (UID: \"33de58ad-f50a-45c9-9475-661288f6233d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2vvsp" Jul 14 02:13:16.358938 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:16.358865 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33de58ad-f50a-45c9-9475-661288f6233d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2vvsp\" (UID: \"33de58ad-f50a-45c9-9475-661288f6233d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2vvsp" Jul 14 02:13:16.359039 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:16.358965 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9x6t\" (UniqueName: \"kubernetes.io/projected/33de58ad-f50a-45c9-9475-661288f6233d-kube-api-access-k9x6t\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2vvsp\" (UID: \"33de58ad-f50a-45c9-9475-661288f6233d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2vvsp" Jul 14 02:13:16.359039 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:16.359005 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33de58ad-f50a-45c9-9475-661288f6233d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2vvsp\" (UID: \"33de58ad-f50a-45c9-9475-661288f6233d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2vvsp" Jul 14 02:13:16.359106 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:16.359077 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33de58ad-f50a-45c9-9475-661288f6233d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2vvsp\" (UID: \"33de58ad-f50a-45c9-9475-661288f6233d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2vvsp" Jul 14 02:13:16.459637 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:16.459553 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9x6t\" (UniqueName: \"kubernetes.io/projected/33de58ad-f50a-45c9-9475-661288f6233d-kube-api-access-k9x6t\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2vvsp\" (UID: \"33de58ad-f50a-45c9-9475-661288f6233d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2vvsp" Jul 14 02:13:16.459637 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:16.459586 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33de58ad-f50a-45c9-9475-661288f6233d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2vvsp\" (UID: \"33de58ad-f50a-45c9-9475-661288f6233d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2vvsp" Jul 14 02:13:16.459637 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:16.459625 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33de58ad-f50a-45c9-9475-661288f6233d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2vvsp\" (UID: \"33de58ad-f50a-45c9-9475-661288f6233d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2vvsp" Jul 14 02:13:16.459913 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:16.459660 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33de58ad-f50a-45c9-9475-661288f6233d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2vvsp\" (UID: \"33de58ad-f50a-45c9-9475-661288f6233d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2vvsp" Jul 14 02:13:16.459913 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:16.459733 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33de58ad-f50a-45c9-9475-661288f6233d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2vvsp\" (UID: \"33de58ad-f50a-45c9-9475-661288f6233d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2vvsp" Jul 14 02:13:16.459913 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:16.459764 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33de58ad-f50a-45c9-9475-661288f6233d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2vvsp\" (UID: \"33de58ad-f50a-45c9-9475-661288f6233d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2vvsp" Jul 14 02:13:16.460086 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:16.460054 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33de58ad-f50a-45c9-9475-661288f6233d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2vvsp\" (UID: \"33de58ad-f50a-45c9-9475-661288f6233d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2vvsp" Jul 14 02:13:16.460086 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:16.460077 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33de58ad-f50a-45c9-9475-661288f6233d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2vvsp\" (UID: \"33de58ad-f50a-45c9-9475-661288f6233d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2vvsp" Jul 14 02:13:16.460209 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:16.460185 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33de58ad-f50a-45c9-9475-661288f6233d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2vvsp\" (UID: \"33de58ad-f50a-45c9-9475-661288f6233d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2vvsp" Jul 14 02:13:16.462299 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:16.462261 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33de58ad-f50a-45c9-9475-661288f6233d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2vvsp\" (UID: \"33de58ad-f50a-45c9-9475-661288f6233d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2vvsp" Jul 14 02:13:16.462408 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:16.462374 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33de58ad-f50a-45c9-9475-661288f6233d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2vvsp\" (UID: \"33de58ad-f50a-45c9-9475-661288f6233d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2vvsp" Jul 14 02:13:16.467086 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:16.467059 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9x6t\" (UniqueName: \"kubernetes.io/projected/33de58ad-f50a-45c9-9475-661288f6233d-kube-api-access-k9x6t\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2vvsp\" (UID: \"33de58ad-f50a-45c9-9475-661288f6233d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2vvsp" Jul 14 02:13:16.501190 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:16.501150 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2vvsp" Jul 14 02:13:16.637472 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:16.637441 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2vvsp"] Jul 14 02:13:16.639192 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:13:16.639161 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33de58ad_f50a_45c9_9475_661288f6233d.slice/crio-027a330d609642af8d8345bc014c3b52303ab682ad93014b56637e91d8d7be2f WatchSource:0}: Error finding container 027a330d609642af8d8345bc014c3b52303ab682ad93014b56637e91d8d7be2f: Status 404 returned error can't find the container with id 027a330d609642af8d8345bc014c3b52303ab682ad93014b56637e91d8d7be2f Jul 14 02:13:17.468709 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:17.468677 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2vvsp" event={"ID":"33de58ad-f50a-45c9-9475-661288f6233d","Type":"ContainerStarted","Data":"d3b64251b1e48cec2e55231baab1b83cf0c52208cd2e251f71e2a9a8e2accb5c"} Jul 14 02:13:17.469083 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:17.468730 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2vvsp" event={"ID":"33de58ad-f50a-45c9-9475-661288f6233d","Type":"ContainerStarted","Data":"027a330d609642af8d8345bc014c3b52303ab682ad93014b56637e91d8d7be2f"} Jul 14 02:13:17.484218 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:17.484164 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2vvsp" podStartSLOduration=1.2810008210000001 podStartE2EDuration="1.484146086s" totalImagesPullingTime="203.145265ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:13:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:13:17.483484985 +0000 UTC m=+699.638321634" watchObservedRunningTime="2026-07-14 02:13:17.484146086 +0000 UTC m=+699.638982767" Jul 14 02:13:18.510606 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:18.510570 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5fgjt" Jul 14 02:13:18.511066 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:18.510640 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5fgjt" Jul 14 02:13:18.523362 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:18.523339 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5fgjt" Jul 14 02:13:19.489961 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:19.489932 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5fgjt" Jul 14 02:13:24.700531 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:24.700490 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:13:24.802454 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:24.802422 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:13:26.501669 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:26.501626 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2vvsp" Jul 14 02:13:26.502096 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:26.501763 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2vvsp" Jul 14 02:13:26.514210 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:26.514185 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2vvsp" Jul 14 02:13:27.526731 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:27.526697 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2vvsp" Jul 14 02:13:44.923832 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:44.923778 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:13:57.686639 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:57.686606 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6vh2d"] Jul 14 02:13:57.697865 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:57.697828 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6vh2d" Jul 14 02:13:57.700458 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:57.700423 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6vh2d"] Jul 14 02:13:57.700606 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:57.700563 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 14 02:13:57.719188 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:57.719160 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:13:57.847829 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:57.847788 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c950ba9-5f49-4d76-9608-b9c16356f4a7-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6vh2d\" (UID: \"2c950ba9-5f49-4d76-9608-b9c16356f4a7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6vh2d" Jul 14 02:13:57.848045 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:57.847849 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crv9h\" (UniqueName: \"kubernetes.io/projected/2c950ba9-5f49-4d76-9608-b9c16356f4a7-kube-api-access-crv9h\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6vh2d\" (UID: \"2c950ba9-5f49-4d76-9608-b9c16356f4a7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6vh2d" Jul 14 02:13:57.848045 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:57.847873 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c950ba9-5f49-4d76-9608-b9c16356f4a7-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6vh2d\" (UID: \"2c950ba9-5f49-4d76-9608-b9c16356f4a7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6vh2d" Jul 14 02:13:57.848045 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:57.847947 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c950ba9-5f49-4d76-9608-b9c16356f4a7-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6vh2d\" (UID: \"2c950ba9-5f49-4d76-9608-b9c16356f4a7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6vh2d" Jul 14 02:13:57.848045 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:57.847992 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c950ba9-5f49-4d76-9608-b9c16356f4a7-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6vh2d\" (UID: \"2c950ba9-5f49-4d76-9608-b9c16356f4a7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6vh2d" Jul 14 02:13:57.848045 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:57.848013 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c950ba9-5f49-4d76-9608-b9c16356f4a7-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6vh2d\" (UID: \"2c950ba9-5f49-4d76-9608-b9c16356f4a7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6vh2d" Jul 14 02:13:57.948909 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:57.948810 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crv9h\" (UniqueName: \"kubernetes.io/projected/2c950ba9-5f49-4d76-9608-b9c16356f4a7-kube-api-access-crv9h\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6vh2d\" (UID: \"2c950ba9-5f49-4d76-9608-b9c16356f4a7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6vh2d" Jul 14 02:13:57.948909 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:57.948849 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c950ba9-5f49-4d76-9608-b9c16356f4a7-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6vh2d\" (UID: \"2c950ba9-5f49-4d76-9608-b9c16356f4a7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6vh2d" Jul 14 02:13:57.948909 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:57.948894 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c950ba9-5f49-4d76-9608-b9c16356f4a7-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6vh2d\" (UID: \"2c950ba9-5f49-4d76-9608-b9c16356f4a7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6vh2d" Jul 14 02:13:57.949163 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:57.948921 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c950ba9-5f49-4d76-9608-b9c16356f4a7-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6vh2d\" (UID: \"2c950ba9-5f49-4d76-9608-b9c16356f4a7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6vh2d" Jul 14 02:13:57.949163 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:57.948940 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c950ba9-5f49-4d76-9608-b9c16356f4a7-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6vh2d\" (UID: \"2c950ba9-5f49-4d76-9608-b9c16356f4a7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6vh2d" Jul 14 02:13:57.949163 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:57.949120 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c950ba9-5f49-4d76-9608-b9c16356f4a7-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6vh2d\" (UID: \"2c950ba9-5f49-4d76-9608-b9c16356f4a7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6vh2d" Jul 14 02:13:57.949395 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:57.949373 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c950ba9-5f49-4d76-9608-b9c16356f4a7-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6vh2d\" (UID: \"2c950ba9-5f49-4d76-9608-b9c16356f4a7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6vh2d" Jul 14 02:13:57.949465 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:57.949442 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c950ba9-5f49-4d76-9608-b9c16356f4a7-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6vh2d\" (UID: \"2c950ba9-5f49-4d76-9608-b9c16356f4a7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6vh2d" Jul 14 02:13:57.949577 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:57.949559 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c950ba9-5f49-4d76-9608-b9c16356f4a7-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6vh2d\" (UID: \"2c950ba9-5f49-4d76-9608-b9c16356f4a7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6vh2d" Jul 14 02:13:57.951442 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:57.951416 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c950ba9-5f49-4d76-9608-b9c16356f4a7-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6vh2d\" (UID: \"2c950ba9-5f49-4d76-9608-b9c16356f4a7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6vh2d" Jul 14 02:13:57.951685 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:57.951668 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c950ba9-5f49-4d76-9608-b9c16356f4a7-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6vh2d\" (UID: \"2c950ba9-5f49-4d76-9608-b9c16356f4a7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6vh2d" Jul 14 02:13:57.956300 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:57.956263 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crv9h\" (UniqueName: \"kubernetes.io/projected/2c950ba9-5f49-4d76-9608-b9c16356f4a7-kube-api-access-crv9h\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6vh2d\" (UID: \"2c950ba9-5f49-4d76-9608-b9c16356f4a7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6vh2d" Jul 14 02:13:58.010050 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:58.010007 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6vh2d" Jul 14 02:13:58.145657 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:58.145531 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6vh2d"] Jul 14 02:13:58.147859 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:13:58.147832 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c950ba9_5f49_4d76_9608_b9c16356f4a7.slice/crio-db2ac6db83048e1cbbada4ed2264f464a9566f7dc275780ec6b0001bb02d17ad WatchSource:0}: Error finding container db2ac6db83048e1cbbada4ed2264f464a9566f7dc275780ec6b0001bb02d17ad: Status 404 returned error can't find the container with id db2ac6db83048e1cbbada4ed2264f464a9566f7dc275780ec6b0001bb02d17ad Jul 14 02:13:58.665117 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:58.665082 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6vh2d" event={"ID":"2c950ba9-5f49-4d76-9608-b9c16356f4a7","Type":"ContainerStarted","Data":"529f895d6ce607f514cacf6b79e4a2581d42521bfb803b16c84073d72500bfec"} Jul 14 02:13:58.665318 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:58.665144 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6vh2d" event={"ID":"2c950ba9-5f49-4d76-9608-b9c16356f4a7","Type":"ContainerStarted","Data":"db2ac6db83048e1cbbada4ed2264f464a9566f7dc275780ec6b0001bb02d17ad"} Jul 14 02:13:58.681827 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:13:58.681774 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6vh2d" podStartSLOduration=1.496619647 podStartE2EDuration="1.681756818s" totalImagesPullingTime="185.137171ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:13:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:13:58.679976349 +0000 UTC m=+740.834812998" watchObservedRunningTime="2026-07-14 02:13:58.681756818 +0000 UTC m=+740.836593466" Jul 14 02:14:08.010561 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:14:08.010513 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6vh2d" Jul 14 02:14:08.010979 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:14:08.010582 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6vh2d" Jul 14 02:14:08.023602 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:14:08.023574 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6vh2d" Jul 14 02:14:08.722956 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:14:08.722924 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6vh2d" Jul 14 02:14:19.229514 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:14:19.229485 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:15:25.695355 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:15:25.695323 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:15:25.795359 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:15:25.795325 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:15:26.436202 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:15:26.436167 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-669564f77f-r4xbn"] Jul 14 02:15:26.892686 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:15:26.892650 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:15:26.989257 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:15:26.989224 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:15:27.083066 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:15:27.083029 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-669564f77f-r4xbn" podUID="c3f9a65c-c505-446a-b0db-9ced2378feb2" containerName="authorino" containerID="cri-o://d057aa42ddbba328b25bc62520029800681a63247f8a15a053c1f6dcd0528d29" gracePeriod=30 Jul 14 02:15:27.342464 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:15:27.342437 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-669564f77f-r4xbn" Jul 14 02:15:27.490468 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:15:27.490374 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rk2bn\" (UniqueName: \"kubernetes.io/projected/c3f9a65c-c505-446a-b0db-9ced2378feb2-kube-api-access-rk2bn\") pod \"c3f9a65c-c505-446a-b0db-9ced2378feb2\" (UID: \"c3f9a65c-c505-446a-b0db-9ced2378feb2\") " Jul 14 02:15:27.490614 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:15:27.490526 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c3f9a65c-c505-446a-b0db-9ced2378feb2-tls-cert\") pod \"c3f9a65c-c505-446a-b0db-9ced2378feb2\" (UID: \"c3f9a65c-c505-446a-b0db-9ced2378feb2\") " Jul 14 02:15:27.492556 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:15:27.492525 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3f9a65c-c505-446a-b0db-9ced2378feb2-kube-api-access-rk2bn" (OuterVolumeSpecName: "kube-api-access-rk2bn") pod "c3f9a65c-c505-446a-b0db-9ced2378feb2" (UID: "c3f9a65c-c505-446a-b0db-9ced2378feb2"). InnerVolumeSpecName "kube-api-access-rk2bn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:15:27.500376 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:15:27.500351 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3f9a65c-c505-446a-b0db-9ced2378feb2-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "c3f9a65c-c505-446a-b0db-9ced2378feb2" (UID: "c3f9a65c-c505-446a-b0db-9ced2378feb2"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 02:15:27.591381 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:15:27.591351 2577 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c3f9a65c-c505-446a-b0db-9ced2378feb2-tls-cert\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:15:27.591381 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:15:27.591376 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rk2bn\" (UniqueName: \"kubernetes.io/projected/c3f9a65c-c505-446a-b0db-9ced2378feb2-kube-api-access-rk2bn\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:15:28.089175 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:15:28.089142 2577 generic.go:356] "Generic (PLEG): container finished" podID="c3f9a65c-c505-446a-b0db-9ced2378feb2" containerID="d057aa42ddbba328b25bc62520029800681a63247f8a15a053c1f6dcd0528d29" exitCode=0 Jul 14 02:15:28.089635 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:15:28.089185 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-669564f77f-r4xbn" Jul 14 02:15:28.089635 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:15:28.089216 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-669564f77f-r4xbn" event={"ID":"c3f9a65c-c505-446a-b0db-9ced2378feb2","Type":"ContainerDied","Data":"d057aa42ddbba328b25bc62520029800681a63247f8a15a053c1f6dcd0528d29"} Jul 14 02:15:28.089635 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:15:28.089246 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-669564f77f-r4xbn" event={"ID":"c3f9a65c-c505-446a-b0db-9ced2378feb2","Type":"ContainerDied","Data":"61d751c8904bbcb1a4fb72e01373384640961f1c2d4bd4aef9fc0aba77bfbacc"} Jul 14 02:15:28.089635 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:15:28.089263 2577 scope.go:117] "RemoveContainer" containerID="d057aa42ddbba328b25bc62520029800681a63247f8a15a053c1f6dcd0528d29" Jul 14 02:15:28.097899 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:15:28.097882 2577 scope.go:117] "RemoveContainer" containerID="d057aa42ddbba328b25bc62520029800681a63247f8a15a053c1f6dcd0528d29" Jul 14 02:15:28.098137 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:15:28.098119 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d057aa42ddbba328b25bc62520029800681a63247f8a15a053c1f6dcd0528d29\": container with ID starting with d057aa42ddbba328b25bc62520029800681a63247f8a15a053c1f6dcd0528d29 not found: ID does not exist" containerID="d057aa42ddbba328b25bc62520029800681a63247f8a15a053c1f6dcd0528d29" Jul 14 02:15:28.098174 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:15:28.098148 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d057aa42ddbba328b25bc62520029800681a63247f8a15a053c1f6dcd0528d29"} err="failed to get container status \"d057aa42ddbba328b25bc62520029800681a63247f8a15a053c1f6dcd0528d29\": rpc error: code = NotFound desc = could not find container \"d057aa42ddbba328b25bc62520029800681a63247f8a15a053c1f6dcd0528d29\": container with ID starting with d057aa42ddbba328b25bc62520029800681a63247f8a15a053c1f6dcd0528d29 not found: ID does not exist" Jul 14 02:15:28.111067 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:15:28.111043 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-669564f77f-r4xbn"] Jul 14 02:15:28.114790 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:15:28.114767 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-669564f77f-r4xbn"] Jul 14 02:15:28.265066 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:15:28.265038 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3f9a65c-c505-446a-b0db-9ced2378feb2" path="/var/lib/kubelet/pods/c3f9a65c-c505-446a-b0db-9ced2378feb2/volumes" Jul 14 02:15:38.542514 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:15:38.542429 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:16:37.321989 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:16:37.321908 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:16:39.301136 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:16:39.301100 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:17:19.906320 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:17:19.906267 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:17:21.665460 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:17:21.665426 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:17:25.650174 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:17:25.650144 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:17:25.749912 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:17:25.749879 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:17:25.849073 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:17:25.849038 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:17:26.004080 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:17:26.003987 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:17:31.622753 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:17:31.622713 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:17:31.722468 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:17:31.722438 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:17:34.234931 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:17:34.234895 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:18:11.192504 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:18:11.192426 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:18:11.292397 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:18:11.292356 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:19:11.879309 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:19:11.879237 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:21:27.614838 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:21:27.614747 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phnk9"] Jul 14 02:21:27.615376 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:21:27.614952 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phnk9" podUID="fcc04ee9-b14a-472a-81d0-b307a428b1c2" containerName="manager" containerID="cri-o://4484c4a1fe858068a18953942a7f081f8001e582a8e85e6dd18854efc2d002ae" gracePeriod=10 Jul 14 02:21:30.796266 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:21:30.796180 2577 generic.go:356] "Generic (PLEG): container finished" podID="fcc04ee9-b14a-472a-81d0-b307a428b1c2" containerID="4484c4a1fe858068a18953942a7f081f8001e582a8e85e6dd18854efc2d002ae" exitCode=0 Jul 14 02:21:30.796266 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:21:30.796252 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phnk9" event={"ID":"fcc04ee9-b14a-472a-81d0-b307a428b1c2","Type":"ContainerDied","Data":"4484c4a1fe858068a18953942a7f081f8001e582a8e85e6dd18854efc2d002ae"} Jul 14 02:21:31.074294 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:21:31.074252 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phnk9" Jul 14 02:21:31.190503 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:21:31.190467 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hxxvx\" (UniqueName: \"kubernetes.io/projected/fcc04ee9-b14a-472a-81d0-b307a428b1c2-kube-api-access-hxxvx\") pod \"fcc04ee9-b14a-472a-81d0-b307a428b1c2\" (UID: \"fcc04ee9-b14a-472a-81d0-b307a428b1c2\") " Jul 14 02:21:31.190707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:21:31.190539 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fcc04ee9-b14a-472a-81d0-b307a428b1c2-extensions-socket-volume\") pod \"fcc04ee9-b14a-472a-81d0-b307a428b1c2\" (UID: \"fcc04ee9-b14a-472a-81d0-b307a428b1c2\") " Jul 14 02:21:31.190930 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:21:31.190906 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcc04ee9-b14a-472a-81d0-b307a428b1c2-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "fcc04ee9-b14a-472a-81d0-b307a428b1c2" (UID: "fcc04ee9-b14a-472a-81d0-b307a428b1c2"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:21:31.192687 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:21:31.192663 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcc04ee9-b14a-472a-81d0-b307a428b1c2-kube-api-access-hxxvx" (OuterVolumeSpecName: "kube-api-access-hxxvx") pod "fcc04ee9-b14a-472a-81d0-b307a428b1c2" (UID: "fcc04ee9-b14a-472a-81d0-b307a428b1c2"). InnerVolumeSpecName "kube-api-access-hxxvx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:21:31.291484 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:21:31.291450 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxxvx\" (UniqueName: \"kubernetes.io/projected/fcc04ee9-b14a-472a-81d0-b307a428b1c2-kube-api-access-hxxvx\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:21:31.291484 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:21:31.291477 2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fcc04ee9-b14a-472a-81d0-b307a428b1c2-extensions-socket-volume\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:21:31.801581 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:21:31.801546 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phnk9" Jul 14 02:21:31.801581 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:21:31.801566 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phnk9" event={"ID":"fcc04ee9-b14a-472a-81d0-b307a428b1c2","Type":"ContainerDied","Data":"46006add1301a099939287434c0980f09e1fd5075bac19f1c9ab723a319e6c9a"} Jul 14 02:21:31.802092 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:21:31.801628 2577 scope.go:117] "RemoveContainer" containerID="4484c4a1fe858068a18953942a7f081f8001e582a8e85e6dd18854efc2d002ae" Jul 14 02:21:31.822084 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:21:31.822053 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phnk9"] Jul 14 02:21:31.825339 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:21:31.825312 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phnk9"] Jul 14 02:21:32.264801 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:21:32.264768 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcc04ee9-b14a-472a-81d0-b307a428b1c2" path="/var/lib/kubelet/pods/fcc04ee9-b14a-472a-81d0-b307a428b1c2/volumes" Jul 14 02:22:28.754348 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:22:28.754310 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9t4xc"] Jul 14 02:22:28.754862 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:22:28.754663 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3f9a65c-c505-446a-b0db-9ced2378feb2" containerName="authorino" Jul 14 02:22:28.754862 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:22:28.754674 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3f9a65c-c505-446a-b0db-9ced2378feb2" containerName="authorino" Jul 14 02:22:28.754862 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:22:28.754689 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcc04ee9-b14a-472a-81d0-b307a428b1c2" containerName="manager" Jul 14 02:22:28.754862 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:22:28.754694 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcc04ee9-b14a-472a-81d0-b307a428b1c2" containerName="manager" Jul 14 02:22:28.754862 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:22:28.754781 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3f9a65c-c505-446a-b0db-9ced2378feb2" containerName="authorino" Jul 14 02:22:28.754862 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:22:28.754789 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcc04ee9-b14a-472a-81d0-b307a428b1c2" containerName="manager" Jul 14 02:22:28.757801 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:22:28.757784 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9t4xc" Jul 14 02:22:28.760235 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:22:28.760217 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-st7t7\"" Jul 14 02:22:28.771978 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:22:28.771953 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9t4xc"] Jul 14 02:22:28.914594 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:22:28.914546 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8fh6\" (UniqueName: \"kubernetes.io/projected/98eaba39-c2ba-4cae-87fb-74089ac777d7-kube-api-access-t8fh6\") pod \"kuadrant-operator-controller-manager-86878c769f-9t4xc\" (UID: \"98eaba39-c2ba-4cae-87fb-74089ac777d7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9t4xc" Jul 14 02:22:28.914594 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:22:28.914608 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/98eaba39-c2ba-4cae-87fb-74089ac777d7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-9t4xc\" (UID: \"98eaba39-c2ba-4cae-87fb-74089ac777d7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9t4xc" Jul 14 02:22:29.016221 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:22:29.016124 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8fh6\" (UniqueName: \"kubernetes.io/projected/98eaba39-c2ba-4cae-87fb-74089ac777d7-kube-api-access-t8fh6\") pod \"kuadrant-operator-controller-manager-86878c769f-9t4xc\" (UID: \"98eaba39-c2ba-4cae-87fb-74089ac777d7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9t4xc" Jul 14 02:22:29.016221 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:22:29.016202 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/98eaba39-c2ba-4cae-87fb-74089ac777d7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-9t4xc\" (UID: \"98eaba39-c2ba-4cae-87fb-74089ac777d7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9t4xc" Jul 14 02:22:29.016597 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:22:29.016576 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/98eaba39-c2ba-4cae-87fb-74089ac777d7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-9t4xc\" (UID: \"98eaba39-c2ba-4cae-87fb-74089ac777d7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9t4xc" Jul 14 02:22:29.023768 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:22:29.023736 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8fh6\" (UniqueName: \"kubernetes.io/projected/98eaba39-c2ba-4cae-87fb-74089ac777d7-kube-api-access-t8fh6\") pod \"kuadrant-operator-controller-manager-86878c769f-9t4xc\" (UID: \"98eaba39-c2ba-4cae-87fb-74089ac777d7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9t4xc" Jul 14 02:22:29.067825 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:22:29.067791 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9t4xc" Jul 14 02:22:29.202781 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:22:29.202755 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9t4xc"] Jul 14 02:22:29.208420 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:22:29.205558 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98eaba39_c2ba_4cae_87fb_74089ac777d7.slice/crio-8f0582d9dbafaa542a463740ef670d7df9ad9672eb0f78f700772258136b4a4f WatchSource:0}: Error finding container 8f0582d9dbafaa542a463740ef670d7df9ad9672eb0f78f700772258136b4a4f: Status 404 returned error can't find the container with id 8f0582d9dbafaa542a463740ef670d7df9ad9672eb0f78f700772258136b4a4f Jul 14 02:22:30.069149 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:22:30.069054 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9t4xc" event={"ID":"98eaba39-c2ba-4cae-87fb-74089ac777d7","Type":"ContainerStarted","Data":"4238fbaf8cf81ac68562e8307ccca02d18d1ad86b54672c43051d212d5fc91aa"} Jul 14 02:22:30.069149 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:22:30.069112 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9t4xc" event={"ID":"98eaba39-c2ba-4cae-87fb-74089ac777d7","Type":"ContainerStarted","Data":"8f0582d9dbafaa542a463740ef670d7df9ad9672eb0f78f700772258136b4a4f"} Jul 14 02:22:30.069149 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:22:30.069141 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9t4xc" Jul 14 02:22:30.088514 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:22:30.088448 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9t4xc" podStartSLOduration=2.088430043 podStartE2EDuration="2.088430043s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:22:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:22:30.087801917 +0000 UTC m=+1252.242638590" watchObservedRunningTime="2026-07-14 02:22:30.088430043 +0000 UTC m=+1252.243266690" Jul 14 02:22:41.076578 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:22:41.076549 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9t4xc" Jul 14 02:23:47.822192 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:23:47.822158 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:23:49.012806 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:23:49.012769 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:24:19.892878 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:24:19.892801 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:25:32.526731 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:25:32.526645 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:28:03.036429 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:03.036386 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:28:18.216962 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:18.216931 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:28:23.535561 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:23.535505 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:28:31.791433 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:31.791345 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:28:36.896507 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:36.896469 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:28:39.837713 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:39.837678 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:28:39.937833 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:39.937804 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:28:40.346645 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:40.346609 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:28:46.018437 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.018396 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:28:46.121031 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.120997 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:28:46.174693 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.174656 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r"] Jul 14 02:28:46.178204 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.178178 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:28:46.180411 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.180386 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-4535243d-openshift-default-dockercfg-tl2bq\"" Jul 14 02:28:46.203097 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.203053 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r"] Jul 14 02:28:46.204646 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.204610 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:28:46.234096 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.234051 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-istio-envoy\") pod \"e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r\" (UID: \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\") " pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:28:46.234292 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.234103 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-credential-socket\") pod \"e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r\" (UID: \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\") " pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:28:46.234354 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.234255 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-workload-socket\") pod \"e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r\" (UID: \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\") " pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:28:46.234410 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.234364 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-istio-podinfo\") pod \"e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r\" (UID: \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\") " pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:28:46.234410 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.234404 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-istio-data\") pod \"e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r\" (UID: \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\") " pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:28:46.234509 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.234447 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-istio-token\") pod \"e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r\" (UID: \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\") " pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:28:46.234509 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.234476 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-workload-certs\") pod \"e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r\" (UID: \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\") " pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:28:46.234509 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.234498 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcc24\" (UniqueName: \"kubernetes.io/projected/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-kube-api-access-hcc24\") pod \"e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r\" (UID: \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\") " pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:28:46.234641 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.234542 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-istiod-ca-cert\") pod \"e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r\" (UID: \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\") " pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:28:46.260746 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.260712 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:28:46.335434 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.335351 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-istio-podinfo\") pod \"e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r\" (UID: \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\") " pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:28:46.335434 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.335397 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-istio-data\") pod \"e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r\" (UID: \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\") " pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:28:46.335678 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.335438 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-istio-token\") pod \"e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r\" (UID: \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\") " pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:28:46.335678 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.335466 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-workload-certs\") pod \"e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r\" (UID: \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\") " pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:28:46.335678 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.335489 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcc24\" (UniqueName: \"kubernetes.io/projected/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-kube-api-access-hcc24\") pod \"e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r\" (UID: \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\") " pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:28:46.335678 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.335521 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-istiod-ca-cert\") pod \"e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r\" (UID: \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\") " pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:28:46.335974 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.335946 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-workload-certs\") pod \"e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r\" (UID: \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\") " pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:28:46.336043 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.336004 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-istio-envoy\") pod \"e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r\" (UID: \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\") " pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:28:46.336043 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.336039 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-credential-socket\") pod \"e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r\" (UID: \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\") " pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:28:46.336150 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.336140 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-workload-socket\") pod \"e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r\" (UID: \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\") " pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:28:46.336582 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.336554 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-istio-data\") pod \"e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r\" (UID: \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\") " pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:28:46.336738 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.336715 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-istiod-ca-cert\") pod \"e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r\" (UID: \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\") " pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:28:46.336825 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.336590 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-credential-socket\") pod \"e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r\" (UID: \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\") " pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:28:46.337030 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.337009 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-workload-socket\") pod \"e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r\" (UID: \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\") " pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:28:46.338884 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.338851 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-istio-envoy\") pod \"e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r\" (UID: \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\") " pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:28:46.338974 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.338927 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-istio-podinfo\") pod \"e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r\" (UID: \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\") " pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:28:46.343722 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.343698 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-istio-token\") pod \"e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r\" (UID: \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\") " pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:28:46.343890 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.343873 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcc24\" (UniqueName: \"kubernetes.io/projected/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-kube-api-access-hcc24\") pod \"e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r\" (UID: \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\") " pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:28:46.488917 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.488871 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:28:46.622978 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.622951 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r"] Jul 14 02:28:46.624936 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:28:46.624908 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cbb4f9c_8a5d_4f91_aa45_41ee9ded255f.slice/crio-9fe3eac1ab85f0383dfce001dc484f27254794e56e102540520ff463ab213619 WatchSource:0}: Error finding container 9fe3eac1ab85f0383dfce001dc484f27254794e56e102540520ff463ab213619: Status 404 returned error can't find the container with id 9fe3eac1ab85f0383dfce001dc484f27254794e56e102540520ff463ab213619 Jul 14 02:28:46.627100 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.627064 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:28:46.627175 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.627154 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:28:46.627215 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.627201 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:28:46.842790 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.842748 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" event={"ID":"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f","Type":"ContainerStarted","Data":"e8c339a41b5bba84ad74155d93cc095912aa7990a5dd8e2d36c1466742e213ae"} Jul 14 02:28:46.842967 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.842819 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" event={"ID":"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f","Type":"ContainerStarted","Data":"9fe3eac1ab85f0383dfce001dc484f27254794e56e102540520ff463ab213619"} Jul 14 02:28:46.862602 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:46.862551 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" podStartSLOduration=0.862534449 podStartE2EDuration="862.534449ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:28:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:28:46.860887915 +0000 UTC m=+1629.015724564" watchObservedRunningTime="2026-07-14 02:28:46.862534449 +0000 UTC m=+1629.017371144" Jul 14 02:28:47.299577 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:47.299543 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:28:47.398135 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:47.398073 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:28:47.489118 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:47.489077 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:28:47.501296 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:47.501234 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:28:47.541620 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:47.541589 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:28:47.680343 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:47.680307 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:28:47.847427 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:47.847398 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:28:47.848538 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:47.848512 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:28:47.978934 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:47.978900 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:28:48.074285 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:48.074242 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:28:49.836085 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:49.836036 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:28:58.901920 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:58.901888 2577 generic.go:356] "Generic (PLEG): container finished" podID="3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f" containerID="e8c339a41b5bba84ad74155d93cc095912aa7990a5dd8e2d36c1466742e213ae" exitCode=0 Jul 14 02:28:58.902404 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:58.901949 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" event={"ID":"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f","Type":"ContainerDied","Data":"e8c339a41b5bba84ad74155d93cc095912aa7990a5dd8e2d36c1466742e213ae"} Jul 14 02:28:58.902404 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:58.902325 2577 scope.go:117] "RemoveContainer" containerID="e8c339a41b5bba84ad74155d93cc095912aa7990a5dd8e2d36c1466742e213ae" Jul 14 02:28:58.903328 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:58.903300 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:28:58.903390 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:58.903366 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:28:58.903425 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:58.903400 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:28:59.908774 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:28:59.908743 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" event={"ID":"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f","Type":"ContainerStarted","Data":"25268d706b5deee35a7f703eed912af8f11c89ab09d16e61f7fa69a69fe4473e"} Jul 14 02:29:00.489256 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:29:00.489212 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:29:00.490597 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:29:00.490569 2577 patch_prober.go:28] interesting pod/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 14 02:29:00.490742 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:29:00.490637 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" podUID="3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 14 02:29:01.489232 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:29:01.489190 2577 patch_prober.go:28] interesting pod/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 14 02:29:01.489731 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:29:01.489259 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" podUID="3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 14 02:29:02.489540 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:29:02.489504 2577 patch_prober.go:28] interesting pod/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 14 02:29:02.489923 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:29:02.489559 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" podUID="3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 14 02:29:02.847535 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:29:02.847431 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:29:03.489905 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:29:03.489871 2577 patch_prober.go:28] interesting pod/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 14 02:29:03.490327 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:29:03.489927 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" podUID="3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 14 02:29:04.493808 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:29:04.493777 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:29:04.494882 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:29:04.494866 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:29:52.892597 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:29:52.892553 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:29:57.485341 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:29:57.485292 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:30:03.029210 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:03.029098 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:30:03.081777 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:03.081706 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:30:07.651359 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:07.651323 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r"] Jul 14 02:30:07.651802 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:07.651525 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" podUID="3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f" containerName="istio-proxy" containerID="cri-o://25268d706b5deee35a7f703eed912af8f11c89ab09d16e61f7fa69a69fe4473e" gracePeriod=30 Jul 14 02:30:07.671866 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:07.671802 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:30:08.044755 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.044645 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:30:08.145358 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.145319 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:30:08.205068 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.205036 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l"] Jul 14 02:30:08.209262 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.209245 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:30:08.211529 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.211508 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-fc424a89-gw-openshift-default-dockercfg-pgqvw\"" Jul 14 02:30:08.221225 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.221197 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l"] Jul 14 02:30:08.238642 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.238608 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:30:08.321223 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.321104 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-workload-socket\") pod \"e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l\" (UID: \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\") " pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:30:08.321223 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.321185 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-workload-certs\") pod \"e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l\" (UID: \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\") " pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:30:08.321511 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.321298 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-istio-token\") pod \"e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l\" (UID: \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\") " pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:30:08.321511 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.321360 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-istio-data\") pod \"e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l\" (UID: \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\") " pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:30:08.321511 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.321412 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-credential-socket\") pod \"e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l\" (UID: \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\") " pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:30:08.321680 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.321514 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-istio-podinfo\") pod \"e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l\" (UID: \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\") " pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:30:08.321680 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.321546 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-istio-envoy\") pod \"e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l\" (UID: \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\") " pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:30:08.321680 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.321573 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7kp7\" (UniqueName: \"kubernetes.io/projected/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-kube-api-access-p7kp7\") pod \"e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l\" (UID: \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\") " pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:30:08.321680 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.321599 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-istiod-ca-cert\") pod \"e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l\" (UID: \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\") " pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:30:08.422538 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.422499 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-istio-token\") pod \"e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l\" (UID: \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\") " pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:30:08.422717 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.422565 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-istio-data\") pod \"e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l\" (UID: \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\") " pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:30:08.422717 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.422599 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-credential-socket\") pod \"e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l\" (UID: \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\") " pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:30:08.422717 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.422687 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-istio-podinfo\") pod \"e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l\" (UID: \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\") " pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:30:08.422875 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.422719 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-istio-envoy\") pod \"e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l\" (UID: \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\") " pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:30:08.422875 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.422847 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7kp7\" (UniqueName: \"kubernetes.io/projected/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-kube-api-access-p7kp7\") pod \"e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l\" (UID: \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\") " pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:30:08.422982 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.422901 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-istiod-ca-cert\") pod \"e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l\" (UID: \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\") " pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:30:08.422982 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.422931 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-workload-socket\") pod \"e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l\" (UID: \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\") " pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:30:08.422982 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.422962 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-workload-certs\") pod \"e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l\" (UID: \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\") " pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:30:08.423142 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.423049 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-istio-data\") pod \"e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l\" (UID: \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\") " pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:30:08.423142 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.423077 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-credential-socket\") pod \"e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l\" (UID: \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\") " pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:30:08.423317 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.423292 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-workload-socket\") pod \"e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l\" (UID: \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\") " pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:30:08.423396 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.423322 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-workload-certs\") pod \"e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l\" (UID: \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\") " pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:30:08.423696 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.423674 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-istiod-ca-cert\") pod \"e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l\" (UID: \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\") " pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:30:08.425609 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.425584 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-istio-envoy\") pod \"e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l\" (UID: \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\") " pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:30:08.425722 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.425704 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-istio-podinfo\") pod \"e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l\" (UID: \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\") " pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:30:08.444698 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.444662 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-istio-token\") pod \"e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l\" (UID: \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\") " pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:30:08.446904 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.446873 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7kp7\" (UniqueName: \"kubernetes.io/projected/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-kube-api-access-p7kp7\") pod \"e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l\" (UID: \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\") " pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:30:08.523374 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.523345 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:30:08.667682 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.667469 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l"] Jul 14 02:30:08.670323 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:30:08.670293 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f7e08b7_f24d_49e0_a75f_d822b7fa2bd0.slice/crio-f1db7e6637b6448ba63b5c4bc7934a2557b9841c89e9c4ec974be827d7843c22 WatchSource:0}: Error finding container f1db7e6637b6448ba63b5c4bc7934a2557b9841c89e9c4ec974be827d7843c22: Status 404 returned error can't find the container with id f1db7e6637b6448ba63b5c4bc7934a2557b9841c89e9c4ec974be827d7843c22 Jul 14 02:30:08.672495 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.672460 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:30:08.672588 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.672550 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:30:08.672638 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:08.672589 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:30:09.240901 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:09.240864 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" event={"ID":"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0","Type":"ContainerStarted","Data":"6ee6a8b028f112c1dd035105f31b8fa571e4b8766b98f2370092aac53273d840"} Jul 14 02:30:09.241103 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:09.240926 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" event={"ID":"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0","Type":"ContainerStarted","Data":"f1db7e6637b6448ba63b5c4bc7934a2557b9841c89e9c4ec974be827d7843c22"} Jul 14 02:30:09.259181 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:09.259125 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" podStartSLOduration=1.2591093469999999 podStartE2EDuration="1.259109347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:30:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:30:09.257623605 +0000 UTC m=+1711.412460268" watchObservedRunningTime="2026-07-14 02:30:09.259109347 +0000 UTC m=+1711.413945994" Jul 14 02:30:09.524178 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:09.524103 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:30:09.528527 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:09.528504 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:30:09.573648 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:09.573620 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:30:09.672494 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:09.672452 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:30:09.774337 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:09.774253 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:30:10.126861 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:10.126826 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:30:10.227520 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:10.227486 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:30:10.246876 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:10.246847 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:30:10.247949 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:10.247929 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:30:12.897079 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:12.897054 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:30:12.970562 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:12.970535 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-workload-socket\") pod \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\" (UID: \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\") " Jul 14 02:30:12.970732 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:12.970582 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-credential-socket\") pod \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\" (UID: \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\") " Jul 14 02:30:12.970732 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:12.970619 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-istio-podinfo\") pod \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\" (UID: \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\") " Jul 14 02:30:12.970732 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:12.970662 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-istio-envoy\") pod \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\" (UID: \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\") " Jul 14 02:30:12.970732 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:12.970693 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hcc24\" (UniqueName: \"kubernetes.io/projected/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-kube-api-access-hcc24\") pod \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\" (UID: \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\") " Jul 14 02:30:12.970944 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:12.970812 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f" (UID: "3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:30:12.971002 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:12.970949 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f" (UID: "3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:30:12.971109 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:12.971085 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-istio-token\") pod \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\" (UID: \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\") " Jul 14 02:30:12.971191 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:12.971175 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-istio-data\") pod \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\" (UID: \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\") " Jul 14 02:30:12.971253 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:12.971216 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-workload-certs\") pod \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\" (UID: \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\") " Jul 14 02:30:12.971253 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:12.971242 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-istiod-ca-cert\") pod \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\" (UID: \"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f\") " Jul 14 02:30:12.971709 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:12.971683 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-workload-socket\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:30:12.971821 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:12.971715 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-credential-socket\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:30:12.971933 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:12.971902 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f" (UID: "3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:30:12.972767 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:12.972059 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f" (UID: "3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 02:30:12.973008 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:12.972977 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-istio-data" (OuterVolumeSpecName: "istio-data") pod "3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f" (UID: "3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:30:12.973148 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:12.973130 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f" (UID: "3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:30:12.973215 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:12.973179 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f" (UID: "3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 02:30:12.973260 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:12.973236 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-kube-api-access-hcc24" (OuterVolumeSpecName: "kube-api-access-hcc24") pod "3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f" (UID: "3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f"). InnerVolumeSpecName "kube-api-access-hcc24". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:30:12.973489 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:12.973467 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-istio-token" (OuterVolumeSpecName: "istio-token") pod "3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f" (UID: "3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:30:13.072501 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:13.072467 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-istio-podinfo\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:30:13.072501 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:13.072498 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-istio-envoy\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:30:13.072501 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:13.072510 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hcc24\" (UniqueName: \"kubernetes.io/projected/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-kube-api-access-hcc24\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:30:13.072741 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:13.072522 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-istio-token\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:30:13.072741 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:13.072533 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-istio-data\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:30:13.072741 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:13.072543 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-workload-certs\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:30:13.072741 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:13.072553 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f-istiod-ca-cert\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:30:13.262687 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:13.262603 2577 generic.go:356] "Generic (PLEG): container finished" podID="3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f" containerID="25268d706b5deee35a7f703eed912af8f11c89ab09d16e61f7fa69a69fe4473e" exitCode=0 Jul 14 02:30:13.262687 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:13.262650 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" Jul 14 02:30:13.262687 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:13.262663 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" event={"ID":"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f","Type":"ContainerDied","Data":"25268d706b5deee35a7f703eed912af8f11c89ab09d16e61f7fa69a69fe4473e"} Jul 14 02:30:13.262940 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:13.262711 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r" event={"ID":"3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f","Type":"ContainerDied","Data":"9fe3eac1ab85f0383dfce001dc484f27254794e56e102540520ff463ab213619"} Jul 14 02:30:13.262940 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:13.262740 2577 scope.go:117] "RemoveContainer" containerID="25268d706b5deee35a7f703eed912af8f11c89ab09d16e61f7fa69a69fe4473e" Jul 14 02:30:13.271837 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:13.271818 2577 scope.go:117] "RemoveContainer" containerID="e8c339a41b5bba84ad74155d93cc095912aa7990a5dd8e2d36c1466742e213ae" Jul 14 02:30:13.279594 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:13.279576 2577 scope.go:117] "RemoveContainer" containerID="25268d706b5deee35a7f703eed912af8f11c89ab09d16e61f7fa69a69fe4473e" Jul 14 02:30:13.279924 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:30:13.279901 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25268d706b5deee35a7f703eed912af8f11c89ab09d16e61f7fa69a69fe4473e\": container with ID starting with 25268d706b5deee35a7f703eed912af8f11c89ab09d16e61f7fa69a69fe4473e not found: ID does not exist" containerID="25268d706b5deee35a7f703eed912af8f11c89ab09d16e61f7fa69a69fe4473e" Jul 14 02:30:13.280019 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:13.279933 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25268d706b5deee35a7f703eed912af8f11c89ab09d16e61f7fa69a69fe4473e"} err="failed to get container status \"25268d706b5deee35a7f703eed912af8f11c89ab09d16e61f7fa69a69fe4473e\": rpc error: code = NotFound desc = could not find container \"25268d706b5deee35a7f703eed912af8f11c89ab09d16e61f7fa69a69fe4473e\": container with ID starting with 25268d706b5deee35a7f703eed912af8f11c89ab09d16e61f7fa69a69fe4473e not found: ID does not exist" Jul 14 02:30:13.280019 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:13.279956 2577 scope.go:117] "RemoveContainer" containerID="e8c339a41b5bba84ad74155d93cc095912aa7990a5dd8e2d36c1466742e213ae" Jul 14 02:30:13.280315 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:30:13.280252 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8c339a41b5bba84ad74155d93cc095912aa7990a5dd8e2d36c1466742e213ae\": container with ID starting with e8c339a41b5bba84ad74155d93cc095912aa7990a5dd8e2d36c1466742e213ae not found: ID does not exist" containerID="e8c339a41b5bba84ad74155d93cc095912aa7990a5dd8e2d36c1466742e213ae" Jul 14 02:30:13.280315 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:13.280299 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8c339a41b5bba84ad74155d93cc095912aa7990a5dd8e2d36c1466742e213ae"} err="failed to get container status \"e8c339a41b5bba84ad74155d93cc095912aa7990a5dd8e2d36c1466742e213ae\": rpc error: code = NotFound desc = could not find container \"e8c339a41b5bba84ad74155d93cc095912aa7990a5dd8e2d36c1466742e213ae\": container with ID starting with e8c339a41b5bba84ad74155d93cc095912aa7990a5dd8e2d36c1466742e213ae not found: ID does not exist" Jul 14 02:30:13.282875 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:13.282854 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r"] Jul 14 02:30:13.286284 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:13.286254 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-4535243d-openshift-default-dfcffcb4c-k5s5r"] Jul 14 02:30:13.994754 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:13.994723 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:30:14.053169 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:14.053139 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:30:14.265797 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:14.265717 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f" path="/var/lib/kubelet/pods/3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f/volumes" Jul 14 02:30:17.344236 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:17.344204 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:30:25.248687 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:25.248651 2577 patch_prober.go:28] interesting pod/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 14 02:30:25.249156 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:25.248713 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" podUID="3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 14 02:30:25.321255 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:25.321208 2577 generic.go:356] "Generic (PLEG): container finished" podID="3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0" containerID="6ee6a8b028f112c1dd035105f31b8fa571e4b8766b98f2370092aac53273d840" exitCode=0 Jul 14 02:30:25.321481 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:25.321363 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" event={"ID":"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0","Type":"ContainerDied","Data":"6ee6a8b028f112c1dd035105f31b8fa571e4b8766b98f2370092aac53273d840"} Jul 14 02:30:25.321869 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:25.321845 2577 scope.go:117] "RemoveContainer" containerID="6ee6a8b028f112c1dd035105f31b8fa571e4b8766b98f2370092aac53273d840" Jul 14 02:30:25.323567 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:25.323535 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:30:25.323677 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:25.323607 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:30:25.323677 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:25.323638 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:30:26.328154 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:26.328113 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" event={"ID":"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0","Type":"ContainerStarted","Data":"d47b8f8c560d9b75a0863a7bcc16f1b0578f47d51040a145a88b9e0c666d4cf4"} Jul 14 02:30:26.523811 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:26.523777 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:30:26.523984 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:26.523899 2577 patch_prober.go:28] interesting pod/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 14 02:30:26.523984 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:26.523943 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" podUID="3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 14 02:30:27.524352 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:27.524321 2577 patch_prober.go:28] interesting pod/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 14 02:30:27.524763 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:27.524376 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" podUID="3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 14 02:30:28.523783 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:28.523748 2577 patch_prober.go:28] interesting pod/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 14 02:30:28.523964 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:28.523816 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" podUID="3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 14 02:30:29.524038 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:29.524005 2577 patch_prober.go:28] interesting pod/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 14 02:30:29.524430 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:29.524060 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" podUID="3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 14 02:30:30.527490 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:30.527461 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:30:30.527953 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:30.527804 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:30:30.528455 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:30:30.528438 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:31:14.151541 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:14.151508 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l"] Jul 14 02:31:14.152000 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:14.151738 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" podUID="3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0" containerName="istio-proxy" containerID="cri-o://d47b8f8c560d9b75a0863a7bcc16f1b0578f47d51040a145a88b9e0c666d4cf4" gracePeriod=30 Jul 14 02:31:14.175476 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:14.175449 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:31:14.284070 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:14.284034 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:31:15.530311 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:15.530263 2577 patch_prober.go:28] interesting pod/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 02:31:15.530689 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:15.530339 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" podUID="3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 02:31:19.200212 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.200176 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:31:19.300063 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.300035 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:31:19.413989 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.413930 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:31:19.518111 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.518082 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-istio-podinfo\") pod \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\" (UID: \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\") " Jul 14 02:31:19.518257 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.518130 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-istiod-ca-cert\") pod \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\" (UID: \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\") " Jul 14 02:31:19.518257 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.518202 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-workload-socket\") pod \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\" (UID: \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\") " Jul 14 02:31:19.518257 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.518246 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-istio-token\") pod \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\" (UID: \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\") " Jul 14 02:31:19.518454 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.518290 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-workload-certs\") pod \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\" (UID: \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\") " Jul 14 02:31:19.518454 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.518349 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-istio-data\") pod \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\" (UID: \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\") " Jul 14 02:31:19.518454 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.518440 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-istio-envoy\") pod \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\" (UID: \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\") " Jul 14 02:31:19.518603 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.518488 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-credential-socket\") pod \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\" (UID: \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\") " Jul 14 02:31:19.518603 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.518519 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7kp7\" (UniqueName: \"kubernetes.io/projected/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-kube-api-access-p7kp7\") pod \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\" (UID: \"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0\") " Jul 14 02:31:19.518603 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.518543 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0" (UID: "3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:31:19.518758 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.518611 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0" (UID: "3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:31:19.518758 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.518703 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0" (UID: "3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 02:31:19.518869 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.518819 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0" (UID: "3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:31:19.518942 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.518905 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-credential-socket\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:31:19.518942 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.518926 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-istiod-ca-cert\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:31:19.519059 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.518943 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-workload-socket\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:31:19.519059 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.518961 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-workload-certs\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:31:19.519160 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.519138 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-istio-data" (OuterVolumeSpecName: "istio-data") pod "3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0" (UID: "3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:31:19.520718 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.520695 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-kube-api-access-p7kp7" (OuterVolumeSpecName: "kube-api-access-p7kp7") pod "3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0" (UID: "3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0"). InnerVolumeSpecName "kube-api-access-p7kp7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:31:19.520893 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.520868 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0" (UID: "3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:31:19.520951 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.520931 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0" (UID: "3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 02:31:19.520990 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.520956 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-istio-token" (OuterVolumeSpecName: "istio-token") pod "3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0" (UID: "3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:31:19.589003 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.588967 2577 generic.go:356] "Generic (PLEG): container finished" podID="3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0" containerID="d47b8f8c560d9b75a0863a7bcc16f1b0578f47d51040a145a88b9e0c666d4cf4" exitCode=0 Jul 14 02:31:19.589176 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.589020 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" Jul 14 02:31:19.589176 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.589058 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" event={"ID":"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0","Type":"ContainerDied","Data":"d47b8f8c560d9b75a0863a7bcc16f1b0578f47d51040a145a88b9e0c666d4cf4"} Jul 14 02:31:19.589176 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.589121 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l" event={"ID":"3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0","Type":"ContainerDied","Data":"f1db7e6637b6448ba63b5c4bc7934a2557b9841c89e9c4ec974be827d7843c22"} Jul 14 02:31:19.589176 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.589148 2577 scope.go:117] "RemoveContainer" containerID="d47b8f8c560d9b75a0863a7bcc16f1b0578f47d51040a145a88b9e0c666d4cf4" Jul 14 02:31:19.598660 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.598645 2577 scope.go:117] "RemoveContainer" containerID="6ee6a8b028f112c1dd035105f31b8fa571e4b8766b98f2370092aac53273d840" Jul 14 02:31:19.606042 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.606026 2577 scope.go:117] "RemoveContainer" containerID="d47b8f8c560d9b75a0863a7bcc16f1b0578f47d51040a145a88b9e0c666d4cf4" Jul 14 02:31:19.606312 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:31:19.606292 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d47b8f8c560d9b75a0863a7bcc16f1b0578f47d51040a145a88b9e0c666d4cf4\": container with ID starting with d47b8f8c560d9b75a0863a7bcc16f1b0578f47d51040a145a88b9e0c666d4cf4 not found: ID does not exist" containerID="d47b8f8c560d9b75a0863a7bcc16f1b0578f47d51040a145a88b9e0c666d4cf4" Jul 14 02:31:19.606396 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.606325 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d47b8f8c560d9b75a0863a7bcc16f1b0578f47d51040a145a88b9e0c666d4cf4"} err="failed to get container status \"d47b8f8c560d9b75a0863a7bcc16f1b0578f47d51040a145a88b9e0c666d4cf4\": rpc error: code = NotFound desc = could not find container \"d47b8f8c560d9b75a0863a7bcc16f1b0578f47d51040a145a88b9e0c666d4cf4\": container with ID starting with d47b8f8c560d9b75a0863a7bcc16f1b0578f47d51040a145a88b9e0c666d4cf4 not found: ID does not exist" Jul 14 02:31:19.606396 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.606343 2577 scope.go:117] "RemoveContainer" containerID="6ee6a8b028f112c1dd035105f31b8fa571e4b8766b98f2370092aac53273d840" Jul 14 02:31:19.606857 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:31:19.606663 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ee6a8b028f112c1dd035105f31b8fa571e4b8766b98f2370092aac53273d840\": container with ID starting with 6ee6a8b028f112c1dd035105f31b8fa571e4b8766b98f2370092aac53273d840 not found: ID does not exist" containerID="6ee6a8b028f112c1dd035105f31b8fa571e4b8766b98f2370092aac53273d840" Jul 14 02:31:19.606857 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.606823 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ee6a8b028f112c1dd035105f31b8fa571e4b8766b98f2370092aac53273d840"} err="failed to get container status \"6ee6a8b028f112c1dd035105f31b8fa571e4b8766b98f2370092aac53273d840\": rpc error: code = NotFound desc = could not find container \"6ee6a8b028f112c1dd035105f31b8fa571e4b8766b98f2370092aac53273d840\": container with ID starting with 6ee6a8b028f112c1dd035105f31b8fa571e4b8766b98f2370092aac53273d840 not found: ID does not exist" Jul 14 02:31:19.609263 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.609242 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l"] Jul 14 02:31:19.612228 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.612209 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-fc424a89-gw-openshift-default-64b66759f-jz62l"] Jul 14 02:31:19.619630 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.619613 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-istio-envoy\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:31:19.619704 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.619633 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7kp7\" (UniqueName: \"kubernetes.io/projected/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-kube-api-access-p7kp7\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:31:19.619704 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.619642 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-istio-podinfo\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:31:19.619704 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.619652 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-istio-token\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:31:19.619704 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.619660 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0-istio-data\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:31:19.786359 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:19.786257 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:31:20.148802 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:20.148773 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:31:20.265340 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:20.265312 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0" path="/var/lib/kubelet/pods/3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0/volumes" Jul 14 02:31:20.690324 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:20.690252 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:31:25.556391 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.556355 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:31:25.656785 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.656742 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:31:25.727970 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.727936 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9"] Jul 14 02:31:25.728467 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.728436 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f" containerName="istio-proxy" Jul 14 02:31:25.728467 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.728461 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f" containerName="istio-proxy" Jul 14 02:31:25.728467 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.728473 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f" containerName="istio-proxy" Jul 14 02:31:25.728711 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.728482 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f" containerName="istio-proxy" Jul 14 02:31:25.728711 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.728497 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0" containerName="istio-proxy" Jul 14 02:31:25.728711 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.728504 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0" containerName="istio-proxy" Jul 14 02:31:25.728711 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.728530 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0" containerName="istio-proxy" Jul 14 02:31:25.728711 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.728538 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0" containerName="istio-proxy" Jul 14 02:31:25.728711 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.728632 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0" containerName="istio-proxy" Jul 14 02:31:25.728711 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.728645 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f7e08b7-f24d-49e0-a75f-d822b7fa2bd0" containerName="istio-proxy" Jul 14 02:31:25.728711 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.728658 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f" containerName="istio-proxy" Jul 14 02:31:25.732977 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.732961 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:31:25.735154 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.735131 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-b25e0f88-openshift-default-dockercfg-5t2jw\"" Jul 14 02:31:25.745259 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.745236 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9"] Jul 14 02:31:25.769228 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.769188 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:31:25.865118 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.865092 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:31:25.878263 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.878234 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-istio-data\") pod \"e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9\" (UID: \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\") " pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:31:25.878417 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.878295 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-istio-envoy\") pod \"e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9\" (UID: \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\") " pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:31:25.878417 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.878359 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-workload-socket\") pod \"e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9\" (UID: \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\") " pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:31:25.878504 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.878417 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-istiod-ca-cert\") pod \"e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9\" (UID: \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\") " pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:31:25.878504 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.878439 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-credential-socket\") pod \"e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9\" (UID: \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\") " pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:31:25.878588 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.878506 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-istio-podinfo\") pod \"e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9\" (UID: \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\") " pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:31:25.878588 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.878524 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-istio-token\") pod \"e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9\" (UID: \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\") " pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:31:25.878588 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.878557 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wwhs\" (UniqueName: \"kubernetes.io/projected/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-kube-api-access-4wwhs\") pod \"e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9\" (UID: \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\") " pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:31:25.878700 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.878598 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-workload-certs\") pod \"e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9\" (UID: \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\") " pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:31:25.979748 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.979712 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-istio-podinfo\") pod \"e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9\" (UID: \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\") " pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:31:25.979748 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.979747 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-istio-token\") pod \"e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9\" (UID: \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\") " pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:31:25.979996 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.979776 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wwhs\" (UniqueName: \"kubernetes.io/projected/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-kube-api-access-4wwhs\") pod \"e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9\" (UID: \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\") " pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:31:25.979996 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.979816 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-workload-certs\") pod \"e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9\" (UID: \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\") " pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:31:25.979996 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.979847 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-istio-data\") pod \"e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9\" (UID: \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\") " pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:31:25.979996 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.979873 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-istio-envoy\") pod \"e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9\" (UID: \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\") " pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:31:25.979996 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.979909 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-workload-socket\") pod \"e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9\" (UID: \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\") " pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:31:25.979996 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.979970 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-istiod-ca-cert\") pod \"e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9\" (UID: \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\") " pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:31:25.979996 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.979994 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-credential-socket\") pod \"e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9\" (UID: \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\") " pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:31:25.980387 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.980332 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-workload-certs\") pod \"e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9\" (UID: \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\") " pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:31:25.980446 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.980389 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-workload-socket\") pod \"e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9\" (UID: \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\") " pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:31:25.980503 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.980464 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-istio-data\") pod \"e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9\" (UID: \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\") " pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:31:25.980542 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.980470 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-credential-socket\") pod \"e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9\" (UID: \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\") " pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:31:25.980750 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.980726 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-istiod-ca-cert\") pod \"e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9\" (UID: \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\") " pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:31:25.982313 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.982292 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-istio-envoy\") pod \"e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9\" (UID: \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\") " pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:31:25.982398 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.982354 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-istio-podinfo\") pod \"e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9\" (UID: \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\") " pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:31:25.986877 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.986856 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wwhs\" (UniqueName: \"kubernetes.io/projected/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-kube-api-access-4wwhs\") pod \"e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9\" (UID: \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\") " pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:31:25.986963 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:25.986943 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-istio-token\") pod \"e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9\" (UID: \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\") " pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:31:26.048459 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:26.048421 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:31:26.189867 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:26.189800 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9"] Jul 14 02:31:26.192003 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:31:26.191978 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c1e8608_cb0c_4687_b4c0_83a5427ca80a.slice/crio-f18bc38398e84c88ca99baa7425eaa1df63610eade82dce251a8561b1fc9e292 WatchSource:0}: Error finding container f18bc38398e84c88ca99baa7425eaa1df63610eade82dce251a8561b1fc9e292: Status 404 returned error can't find the container with id f18bc38398e84c88ca99baa7425eaa1df63610eade82dce251a8561b1fc9e292 Jul 14 02:31:26.194325 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:26.194260 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:31:26.194420 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:26.194401 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:31:26.194469 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:26.194453 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:31:26.624339 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:26.624307 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" event={"ID":"4c1e8608-cb0c-4687-b4c0-83a5427ca80a","Type":"ContainerStarted","Data":"5523d7cb3687f342f0d5e0446054d32b0505ee728bc7150c9a5882a639a13da1"} Jul 14 02:31:26.624779 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:26.624364 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" event={"ID":"4c1e8608-cb0c-4687-b4c0-83a5427ca80a","Type":"ContainerStarted","Data":"f18bc38398e84c88ca99baa7425eaa1df63610eade82dce251a8561b1fc9e292"} Jul 14 02:31:26.640960 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:26.640903 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" podStartSLOduration=1.6408825500000002 podStartE2EDuration="1.64088255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:31:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:31:26.639916285 +0000 UTC m=+1788.794752934" watchObservedRunningTime="2026-07-14 02:31:26.64088255 +0000 UTC m=+1788.795719202" Jul 14 02:31:26.881847 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:26.881764 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:31:26.958054 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:26.958017 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:31:27.040439 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:27.040409 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:31:27.049686 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:27.049639 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:31:27.068460 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:27.068431 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:31:27.292060 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:27.291989 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:31:27.493413 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:27.493381 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:31:27.590656 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:27.590569 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:31:27.629831 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:27.629794 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:31:27.630934 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:27.630906 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:31:28.818076 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:28.818049 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:31:38.688912 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:38.688813 2577 generic.go:356] "Generic (PLEG): container finished" podID="4c1e8608-cb0c-4687-b4c0-83a5427ca80a" containerID="5523d7cb3687f342f0d5e0446054d32b0505ee728bc7150c9a5882a639a13da1" exitCode=0 Jul 14 02:31:38.688912 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:38.688891 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" event={"ID":"4c1e8608-cb0c-4687-b4c0-83a5427ca80a","Type":"ContainerDied","Data":"5523d7cb3687f342f0d5e0446054d32b0505ee728bc7150c9a5882a639a13da1"} Jul 14 02:31:38.689497 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:38.689371 2577 scope.go:117] "RemoveContainer" containerID="5523d7cb3687f342f0d5e0446054d32b0505ee728bc7150c9a5882a639a13da1" Jul 14 02:31:38.690509 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:38.690481 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:31:38.690606 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:38.690566 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:31:38.690666 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:38.690630 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:31:39.695038 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:39.695003 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" event={"ID":"4c1e8608-cb0c-4687-b4c0-83a5427ca80a","Type":"ContainerStarted","Data":"bc2a58ad30460f891ad6a42cd2e717f1aef6dba97710286cf0cff77877854693"} Jul 14 02:31:40.049667 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:40.049570 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:31:40.050572 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:40.050537 2577 patch_prober.go:28] interesting pod/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Jul 14 02:31:40.050703 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:40.050601 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" podUID="4c1e8608-cb0c-4687-b4c0-83a5427ca80a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Jul 14 02:31:41.049700 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:41.049656 2577 patch_prober.go:28] interesting pod/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Jul 14 02:31:41.050188 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:41.049730 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" podUID="4c1e8608-cb0c-4687-b4c0-83a5427ca80a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Jul 14 02:31:42.049538 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:42.049491 2577 patch_prober.go:28] interesting pod/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Jul 14 02:31:42.049761 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:42.049565 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" podUID="4c1e8608-cb0c-4687-b4c0-83a5427ca80a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Jul 14 02:31:42.629868 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:42.629830 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:31:43.048936 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:43.048844 2577 patch_prober.go:28] interesting pod/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Jul 14 02:31:43.049217 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:43.049180 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" podUID="4c1e8608-cb0c-4687-b4c0-83a5427ca80a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Jul 14 02:31:44.053360 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:44.053330 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:31:44.054229 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:44.054213 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:31:59.730067 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:31:59.730036 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:32:00.968252 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:00.968212 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:32:01.069539 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:01.069504 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:32:02.581818 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:02.581783 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:32:02.683864 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:02.683832 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:32:42.354418 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:42.354385 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:32:47.076218 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:47.076186 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:32:52.821556 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:52.821525 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:32:52.922009 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:52.921977 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:32:58.079854 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.079821 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9"] Jul 14 02:32:58.080355 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.080016 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" podUID="4c1e8608-cb0c-4687-b4c0-83a5427ca80a" containerName="istio-proxy" containerID="cri-o://bc2a58ad30460f891ad6a42cd2e717f1aef6dba97710286cf0cff77877854693" gracePeriod=30 Jul 14 02:32:58.102459 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.102426 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:32:58.373080 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.373049 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28"] Jul 14 02:32:58.373980 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.373950 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cbb4f9c-8a5d-4f91-aa45-41ee9ded255f" containerName="istio-proxy" Jul 14 02:32:58.377428 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.377405 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:32:58.379649 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.379627 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-a5aba0cb-openshift-default-dockercfg-568jl\"" Jul 14 02:32:58.388897 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.388876 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28"] Jul 14 02:32:58.408106 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.408086 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:32:58.472652 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.472624 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-workload-certs\") pod \"e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28\" (UID: \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\") " pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:32:58.472797 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.472659 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-credential-socket\") pod \"e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28\" (UID: \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\") " pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:32:58.472797 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.472680 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-workload-socket\") pod \"e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28\" (UID: \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\") " pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:32:58.472797 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.472703 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-istio-data\") pod \"e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28\" (UID: \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\") " pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:32:58.472797 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.472770 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-istio-envoy\") pod \"e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28\" (UID: \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\") " pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:32:58.472936 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.472813 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-istio-token\") pod \"e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28\" (UID: \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\") " pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:32:58.472936 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.472842 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-istio-podinfo\") pod \"e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28\" (UID: \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\") " pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:32:58.472936 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.472893 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-istiod-ca-cert\") pod \"e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28\" (UID: \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\") " pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:32:58.472936 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.472918 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvm5n\" (UniqueName: \"kubernetes.io/projected/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-kube-api-access-gvm5n\") pod \"e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28\" (UID: \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\") " pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:32:58.573878 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.573841 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-istiod-ca-cert\") pod \"e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28\" (UID: \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\") " pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:32:58.573878 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.573880 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvm5n\" (UniqueName: \"kubernetes.io/projected/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-kube-api-access-gvm5n\") pod \"e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28\" (UID: \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\") " pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:32:58.574132 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.573934 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-workload-certs\") pod \"e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28\" (UID: \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\") " pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:32:58.574132 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.573956 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-credential-socket\") pod \"e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28\" (UID: \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\") " pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:32:58.574132 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.573974 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-workload-socket\") pod \"e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28\" (UID: \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\") " pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:32:58.574132 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.573998 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-istio-data\") pod \"e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28\" (UID: \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\") " pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:32:58.574132 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.574030 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-istio-envoy\") pod \"e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28\" (UID: \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\") " pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:32:58.574132 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.574072 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-istio-token\") pod \"e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28\" (UID: \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\") " pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:32:58.574132 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.574112 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-istio-podinfo\") pod \"e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28\" (UID: \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\") " pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:32:58.574547 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.574451 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-credential-socket\") pod \"e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28\" (UID: \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\") " pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:32:58.574547 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.574517 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-workload-socket\") pod \"e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28\" (UID: \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\") " pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:32:58.574656 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.574539 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-istio-data\") pod \"e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28\" (UID: \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\") " pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:32:58.574656 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.574595 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-workload-certs\") pod \"e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28\" (UID: \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\") " pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:32:58.574976 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.574957 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-istiod-ca-cert\") pod \"e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28\" (UID: \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\") " pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:32:58.576428 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.576409 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-istio-envoy\") pod \"e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28\" (UID: \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\") " pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:32:58.576547 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.576530 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-istio-podinfo\") pod \"e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28\" (UID: \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\") " pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:32:58.581034 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.581013 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-istio-token\") pod \"e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28\" (UID: \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\") " pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:32:58.581387 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.581366 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvm5n\" (UniqueName: \"kubernetes.io/projected/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-kube-api-access-gvm5n\") pod \"e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28\" (UID: \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\") " pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:32:58.689573 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.689478 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:32:58.825375 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.825346 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28"] Jul 14 02:32:58.826429 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:32:58.826404 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4bff5dc_ec1d_4abe_bdf5_d81f9e2cf2c0.slice/crio-4922e680cdddfe98b95b4bbb7976caa2309d474bf922d6764daa1a8b5539037e WatchSource:0}: Error finding container 4922e680cdddfe98b95b4bbb7976caa2309d474bf922d6764daa1a8b5539037e: Status 404 returned error can't find the container with id 4922e680cdddfe98b95b4bbb7976caa2309d474bf922d6764daa1a8b5539037e Jul 14 02:32:58.828694 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.828660 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:32:58.828806 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.828722 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:32:58.828806 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:58.828751 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:32:59.056419 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:59.056329 2577 patch_prober.go:28] interesting pod/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 02:32:59.056419 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:59.056393 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" podUID="4c1e8608-cb0c-4687-b4c0-83a5427ca80a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 02:32:59.068135 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:59.068096 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" event={"ID":"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0","Type":"ContainerStarted","Data":"bea8f334629b4b6a79c60b1565847c84a7e965d47b99d20f7fd1cfe106b58386"} Jul 14 02:32:59.068323 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:59.068151 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" event={"ID":"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0","Type":"ContainerStarted","Data":"4922e680cdddfe98b95b4bbb7976caa2309d474bf922d6764daa1a8b5539037e"} Jul 14 02:32:59.085776 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:59.085732 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" podStartSLOduration=1.085717405 podStartE2EDuration="1.085717405s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:32:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:32:59.083383534 +0000 UTC m=+1881.238220183" watchObservedRunningTime="2026-07-14 02:32:59.085717405 +0000 UTC m=+1881.240554053" Jul 14 02:32:59.501047 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:59.501016 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:32:59.584016 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:59.583971 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:32:59.665894 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:59.665859 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:32:59.690598 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:59.690547 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:32:59.712428 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:59.712400 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:32:59.878092 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:32:59.878063 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:33:00.072882 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:00.072850 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:33:00.073965 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:00.073947 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:33:00.132995 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:00.132915 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:33:00.232906 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:00.232874 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:33:03.384042 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:03.384020 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:33:03.421621 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:03.421593 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-istio-envoy\") pod \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\" (UID: \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\") " Jul 14 02:33:03.421782 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:03.421647 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-workload-certs\") pod \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\" (UID: \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\") " Jul 14 02:33:03.421782 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:03.421682 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-workload-socket\") pod \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\" (UID: \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\") " Jul 14 02:33:03.421782 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:03.421715 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-istiod-ca-cert\") pod \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\" (UID: \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\") " Jul 14 02:33:03.421782 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:03.421764 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-credential-socket\") pod \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\" (UID: \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\") " Jul 14 02:33:03.421995 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:03.421789 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-istio-podinfo\") pod \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\" (UID: \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\") " Jul 14 02:33:03.421995 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:03.421836 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-istio-data\") pod \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\" (UID: \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\") " Jul 14 02:33:03.421995 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:03.421870 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4wwhs\" (UniqueName: \"kubernetes.io/projected/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-kube-api-access-4wwhs\") pod \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\" (UID: \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\") " Jul 14 02:33:03.421995 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:03.421936 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-istio-token\") pod \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\" (UID: \"4c1e8608-cb0c-4687-b4c0-83a5427ca80a\") " Jul 14 02:33:03.422232 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:03.422204 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4c1e8608-cb0c-4687-b4c0-83a5427ca80a" (UID: "4c1e8608-cb0c-4687-b4c0-83a5427ca80a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:33:03.422326 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:03.422232 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4c1e8608-cb0c-4687-b4c0-83a5427ca80a" (UID: "4c1e8608-cb0c-4687-b4c0-83a5427ca80a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:33:03.422397 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:03.422352 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-credential-socket\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:33:03.422397 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:03.422374 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-workload-socket\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:33:03.423008 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:03.422600 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4c1e8608-cb0c-4687-b4c0-83a5427ca80a" (UID: "4c1e8608-cb0c-4687-b4c0-83a5427ca80a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 02:33:03.423008 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:03.422645 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4c1e8608-cb0c-4687-b4c0-83a5427ca80a" (UID: "4c1e8608-cb0c-4687-b4c0-83a5427ca80a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:33:03.423498 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:03.423473 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-istio-data" (OuterVolumeSpecName: "istio-data") pod "4c1e8608-cb0c-4687-b4c0-83a5427ca80a" (UID: "4c1e8608-cb0c-4687-b4c0-83a5427ca80a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:33:03.424726 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:03.424701 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-kube-api-access-4wwhs" (OuterVolumeSpecName: "kube-api-access-4wwhs") pod "4c1e8608-cb0c-4687-b4c0-83a5427ca80a" (UID: "4c1e8608-cb0c-4687-b4c0-83a5427ca80a"). InnerVolumeSpecName "kube-api-access-4wwhs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:33:03.424944 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:03.424922 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4c1e8608-cb0c-4687-b4c0-83a5427ca80a" (UID: "4c1e8608-cb0c-4687-b4c0-83a5427ca80a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:33:03.425117 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:03.425088 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4c1e8608-cb0c-4687-b4c0-83a5427ca80a" (UID: "4c1e8608-cb0c-4687-b4c0-83a5427ca80a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 02:33:03.425218 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:03.425183 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-istio-token" (OuterVolumeSpecName: "istio-token") pod "4c1e8608-cb0c-4687-b4c0-83a5427ca80a" (UID: "4c1e8608-cb0c-4687-b4c0-83a5427ca80a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:33:03.523729 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:03.523696 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-istio-token\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:33:03.523729 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:03.523722 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-istio-envoy\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:33:03.523729 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:03.523731 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-workload-certs\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:33:03.523729 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:03.523740 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-istiod-ca-cert\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:33:03.524029 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:03.523748 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-istio-podinfo\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:33:03.524029 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:03.523755 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-istio-data\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:33:03.524029 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:03.523763 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4wwhs\" (UniqueName: \"kubernetes.io/projected/4c1e8608-cb0c-4687-b4c0-83a5427ca80a-kube-api-access-4wwhs\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:33:04.094632 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:04.094598 2577 generic.go:356] "Generic (PLEG): container finished" podID="4c1e8608-cb0c-4687-b4c0-83a5427ca80a" containerID="bc2a58ad30460f891ad6a42cd2e717f1aef6dba97710286cf0cff77877854693" exitCode=0 Jul 14 02:33:04.094830 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:04.094647 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" Jul 14 02:33:04.094830 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:04.094688 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" event={"ID":"4c1e8608-cb0c-4687-b4c0-83a5427ca80a","Type":"ContainerDied","Data":"bc2a58ad30460f891ad6a42cd2e717f1aef6dba97710286cf0cff77877854693"} Jul 14 02:33:04.094830 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:04.094743 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9" event={"ID":"4c1e8608-cb0c-4687-b4c0-83a5427ca80a","Type":"ContainerDied","Data":"f18bc38398e84c88ca99baa7425eaa1df63610eade82dce251a8561b1fc9e292"} Jul 14 02:33:04.094830 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:04.094766 2577 scope.go:117] "RemoveContainer" containerID="bc2a58ad30460f891ad6a42cd2e717f1aef6dba97710286cf0cff77877854693" Jul 14 02:33:04.104156 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:04.104139 2577 scope.go:117] "RemoveContainer" containerID="5523d7cb3687f342f0d5e0446054d32b0505ee728bc7150c9a5882a639a13da1" Jul 14 02:33:04.112221 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:04.112197 2577 scope.go:117] "RemoveContainer" containerID="bc2a58ad30460f891ad6a42cd2e717f1aef6dba97710286cf0cff77877854693" Jul 14 02:33:04.112548 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:33:04.112527 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc2a58ad30460f891ad6a42cd2e717f1aef6dba97710286cf0cff77877854693\": container with ID starting with bc2a58ad30460f891ad6a42cd2e717f1aef6dba97710286cf0cff77877854693 not found: ID does not exist" containerID="bc2a58ad30460f891ad6a42cd2e717f1aef6dba97710286cf0cff77877854693" Jul 14 02:33:04.112640 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:04.112560 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc2a58ad30460f891ad6a42cd2e717f1aef6dba97710286cf0cff77877854693"} err="failed to get container status \"bc2a58ad30460f891ad6a42cd2e717f1aef6dba97710286cf0cff77877854693\": rpc error: code = NotFound desc = could not find container \"bc2a58ad30460f891ad6a42cd2e717f1aef6dba97710286cf0cff77877854693\": container with ID starting with bc2a58ad30460f891ad6a42cd2e717f1aef6dba97710286cf0cff77877854693 not found: ID does not exist" Jul 14 02:33:04.112640 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:04.112583 2577 scope.go:117] "RemoveContainer" containerID="5523d7cb3687f342f0d5e0446054d32b0505ee728bc7150c9a5882a639a13da1" Jul 14 02:33:04.112833 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:33:04.112817 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5523d7cb3687f342f0d5e0446054d32b0505ee728bc7150c9a5882a639a13da1\": container with ID starting with 5523d7cb3687f342f0d5e0446054d32b0505ee728bc7150c9a5882a639a13da1 not found: ID does not exist" containerID="5523d7cb3687f342f0d5e0446054d32b0505ee728bc7150c9a5882a639a13da1" Jul 14 02:33:04.112891 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:04.112837 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5523d7cb3687f342f0d5e0446054d32b0505ee728bc7150c9a5882a639a13da1"} err="failed to get container status \"5523d7cb3687f342f0d5e0446054d32b0505ee728bc7150c9a5882a639a13da1\": rpc error: code = NotFound desc = could not find container \"5523d7cb3687f342f0d5e0446054d32b0505ee728bc7150c9a5882a639a13da1\": container with ID starting with 5523d7cb3687f342f0d5e0446054d32b0505ee728bc7150c9a5882a639a13da1 not found: ID does not exist" Jul 14 02:33:04.114287 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:04.114251 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9"] Jul 14 02:33:04.120099 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:04.120077 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-b25e0f88-openshift-default-645bf654b7-5gpr9"] Jul 14 02:33:04.267504 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:04.267466 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c1e8608-cb0c-4687-b4c0-83a5427ca80a" path="/var/lib/kubelet/pods/4c1e8608-cb0c-4687-b4c0-83a5427ca80a/volumes" Jul 14 02:33:04.666507 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:04.666468 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:33:04.766111 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:04.766069 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:33:07.963429 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:07.963389 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:33:11.134539 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:11.134505 2577 generic.go:356] "Generic (PLEG): container finished" podID="b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0" containerID="bea8f334629b4b6a79c60b1565847c84a7e965d47b99d20f7fd1cfe106b58386" exitCode=0 Jul 14 02:33:11.135072 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:11.134578 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" event={"ID":"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0","Type":"ContainerDied","Data":"bea8f334629b4b6a79c60b1565847c84a7e965d47b99d20f7fd1cfe106b58386"} Jul 14 02:33:11.135072 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:11.135010 2577 scope.go:117] "RemoveContainer" containerID="bea8f334629b4b6a79c60b1565847c84a7e965d47b99d20f7fd1cfe106b58386" Jul 14 02:33:11.136050 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:11.136021 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:33:11.136121 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:11.136091 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:33:11.136178 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:11.136148 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:33:12.144572 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:12.144525 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" event={"ID":"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0","Type":"ContainerStarted","Data":"027f60841ced639d788c1415756fdb22886fc0306241e170a83a89605490930e"} Jul 14 02:33:12.690545 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:12.690502 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:33:12.691669 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:12.691645 2577 patch_prober.go:28] interesting pod/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 14 02:33:12.691779 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:12.691702 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" podUID="b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 14 02:33:13.690117 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:13.690081 2577 patch_prober.go:28] interesting pod/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 14 02:33:13.690714 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:13.690145 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" podUID="b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 14 02:33:14.690597 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:14.690557 2577 patch_prober.go:28] interesting pod/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 14 02:33:14.690986 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:14.690626 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" podUID="b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 14 02:33:15.073049 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:15.072942 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:33:15.689932 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:15.689894 2577 patch_prober.go:28] interesting pod/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 14 02:33:15.690127 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:15.689961 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" podUID="b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 14 02:33:16.694330 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:16.694300 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:33:16.695300 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:16.695283 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:33:58.445477 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:33:58.445443 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:34:04.301359 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:04.301324 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:34:09.021893 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:09.021861 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:34:14.768155 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:14.768102 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:34:14.868478 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:14.868449 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:34:19.176558 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:19.176518 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28"] Jul 14 02:34:19.176961 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:19.176761 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" podUID="b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0" containerName="istio-proxy" containerID="cri-o://027f60841ced639d788c1415756fdb22886fc0306241e170a83a89605490930e" gracePeriod=30 Jul 14 02:34:19.197821 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:19.197787 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:34:19.808585 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:19.808543 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:34:19.908492 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:19.908455 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:34:19.960466 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:19.960433 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:34:21.416435 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:21.416401 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:34:24.433118 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.433090 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:34:24.488459 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.488424 2577 generic.go:356] "Generic (PLEG): container finished" podID="b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0" containerID="027f60841ced639d788c1415756fdb22886fc0306241e170a83a89605490930e" exitCode=0 Jul 14 02:34:24.488637 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.488476 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" Jul 14 02:34:24.488637 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.488507 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" event={"ID":"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0","Type":"ContainerDied","Data":"027f60841ced639d788c1415756fdb22886fc0306241e170a83a89605490930e"} Jul 14 02:34:24.488637 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.488567 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28" event={"ID":"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0","Type":"ContainerDied","Data":"4922e680cdddfe98b95b4bbb7976caa2309d474bf922d6764daa1a8b5539037e"} Jul 14 02:34:24.488637 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.488589 2577 scope.go:117] "RemoveContainer" containerID="027f60841ced639d788c1415756fdb22886fc0306241e170a83a89605490930e" Jul 14 02:34:24.497829 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.497807 2577 scope.go:117] "RemoveContainer" containerID="bea8f334629b4b6a79c60b1565847c84a7e965d47b99d20f7fd1cfe106b58386" Jul 14 02:34:24.506735 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.506709 2577 scope.go:117] "RemoveContainer" containerID="027f60841ced639d788c1415756fdb22886fc0306241e170a83a89605490930e" Jul 14 02:34:24.507074 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:34:24.507057 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"027f60841ced639d788c1415756fdb22886fc0306241e170a83a89605490930e\": container with ID starting with 027f60841ced639d788c1415756fdb22886fc0306241e170a83a89605490930e not found: ID does not exist" containerID="027f60841ced639d788c1415756fdb22886fc0306241e170a83a89605490930e" Jul 14 02:34:24.507149 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.507085 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"027f60841ced639d788c1415756fdb22886fc0306241e170a83a89605490930e"} err="failed to get container status \"027f60841ced639d788c1415756fdb22886fc0306241e170a83a89605490930e\": rpc error: code = NotFound desc = could not find container \"027f60841ced639d788c1415756fdb22886fc0306241e170a83a89605490930e\": container with ID starting with 027f60841ced639d788c1415756fdb22886fc0306241e170a83a89605490930e not found: ID does not exist" Jul 14 02:34:24.507149 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.507106 2577 scope.go:117] "RemoveContainer" containerID="bea8f334629b4b6a79c60b1565847c84a7e965d47b99d20f7fd1cfe106b58386" Jul 14 02:34:24.507420 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:34:24.507403 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bea8f334629b4b6a79c60b1565847c84a7e965d47b99d20f7fd1cfe106b58386\": container with ID starting with bea8f334629b4b6a79c60b1565847c84a7e965d47b99d20f7fd1cfe106b58386 not found: ID does not exist" containerID="bea8f334629b4b6a79c60b1565847c84a7e965d47b99d20f7fd1cfe106b58386" Jul 14 02:34:24.507481 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.507426 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bea8f334629b4b6a79c60b1565847c84a7e965d47b99d20f7fd1cfe106b58386"} err="failed to get container status \"bea8f334629b4b6a79c60b1565847c84a7e965d47b99d20f7fd1cfe106b58386\": rpc error: code = NotFound desc = could not find container \"bea8f334629b4b6a79c60b1565847c84a7e965d47b99d20f7fd1cfe106b58386\": container with ID starting with bea8f334629b4b6a79c60b1565847c84a7e965d47b99d20f7fd1cfe106b58386 not found: ID does not exist" Jul 14 02:34:24.512187 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.512162 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-credential-socket\") pod \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\" (UID: \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\") " Jul 14 02:34:24.512333 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.512216 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gvm5n\" (UniqueName: \"kubernetes.io/projected/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-kube-api-access-gvm5n\") pod \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\" (UID: \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\") " Jul 14 02:34:24.512333 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.512256 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-workload-certs\") pod \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\" (UID: \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\") " Jul 14 02:34:24.512333 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.512302 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-istio-envoy\") pod \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\" (UID: \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\") " Jul 14 02:34:24.512498 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.512348 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-istio-data\") pod \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\" (UID: \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\") " Jul 14 02:34:24.512498 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.512405 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-istio-token\") pod \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\" (UID: \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\") " Jul 14 02:34:24.512498 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.512464 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-istiod-ca-cert\") pod \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\" (UID: \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\") " Jul 14 02:34:24.512657 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.512510 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-istio-podinfo\") pod \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\" (UID: \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\") " Jul 14 02:34:24.512657 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.512546 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-workload-socket\") pod \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\" (UID: \"b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0\") " Jul 14 02:34:24.512657 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.512548 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0" (UID: "b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:34:24.512657 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.512570 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0" (UID: "b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:34:24.512862 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.512828 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-credential-socket\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:34:24.512862 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.512837 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0" (UID: "b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 02:34:24.512862 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.512844 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-workload-certs\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:34:24.513077 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.513057 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0" (UID: "b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:34:24.513490 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.513454 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-istio-data" (OuterVolumeSpecName: "istio-data") pod "b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0" (UID: "b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:34:24.515028 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.515002 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0" (UID: "b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 02:34:24.515154 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.515026 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-kube-api-access-gvm5n" (OuterVolumeSpecName: "kube-api-access-gvm5n") pod "b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0" (UID: "b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0"). InnerVolumeSpecName "kube-api-access-gvm5n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:34:24.515154 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.515111 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0" (UID: "b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:34:24.515154 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.515135 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-istio-token" (OuterVolumeSpecName: "istio-token") pod "b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0" (UID: "b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:34:24.613753 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.613715 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gvm5n\" (UniqueName: \"kubernetes.io/projected/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-kube-api-access-gvm5n\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:34:24.613753 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.613755 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-istio-envoy\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:34:24.613985 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.613768 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-istio-data\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:34:24.613985 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.613779 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-istio-token\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:34:24.613985 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.613790 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-istiod-ca-cert\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:34:24.613985 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.613802 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-istio-podinfo\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:34:24.613985 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.613812 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0-workload-socket\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:34:24.813320 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.813286 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28"] Jul 14 02:34:24.816966 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.816940 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-a5aba0cb-openshift-default-655f44d88c-s5z28"] Jul 14 02:34:24.889608 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:24.889577 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:34:26.239884 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:26.239849 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:34:26.266069 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:26.266033 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0" path="/var/lib/kubelet/pods/b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0/volumes" Jul 14 02:34:27.147240 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:27.147206 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:34:27.199241 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:27.199213 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:34:27.279437 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:27.279404 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:34:27.378604 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:27.378565 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:34:27.478541 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:27.478432 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:34:27.878142 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:27.878106 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:34:53.849472 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:53.849431 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:34:59.384061 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:59.384023 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:34:59.442223 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:59.442188 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:34:59.520436 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:59.520403 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:34:59.622810 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:34:59.622778 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:35:15.473081 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:15.473049 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:35:21.182610 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:21.182572 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:35:21.292224 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:21.291213 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:35:21.382484 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:21.382453 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:35:31.897552 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:31.897521 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:35:32.677249 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:32.677214 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:35:38.480462 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:38.480430 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:35:38.538617 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:38.538588 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:35:38.628545 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:38.628515 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:35:38.741772 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:38.741687 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:35:38.851705 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:38.851672 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:35:39.555231 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:39.555199 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:35:39.654587 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:39.654556 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:35:39.706000 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:39.705964 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:35:41.343421 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:41.343387 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:35:47.048122 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:47.048080 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:35:47.197918 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:47.197883 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:35:47.708830 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:47.708792 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:35:53.380107 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:53.380071 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:35:53.441809 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:53.441775 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:35:53.628922 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:53.628890 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:35:53.910600 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:53.910565 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm"] Jul 14 02:35:53.911052 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:53.911028 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c1e8608-cb0c-4687-b4c0-83a5427ca80a" containerName="istio-proxy" Jul 14 02:35:53.911052 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:53.911047 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c1e8608-cb0c-4687-b4c0-83a5427ca80a" containerName="istio-proxy" Jul 14 02:35:53.911207 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:53.911061 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0" containerName="istio-proxy" Jul 14 02:35:53.911207 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:53.911067 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0" containerName="istio-proxy" Jul 14 02:35:53.911207 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:53.911075 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0" containerName="istio-proxy" Jul 14 02:35:53.911207 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:53.911081 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0" containerName="istio-proxy" Jul 14 02:35:53.911207 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:53.911095 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c1e8608-cb0c-4687-b4c0-83a5427ca80a" containerName="istio-proxy" Jul 14 02:35:53.911207 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:53.911101 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c1e8608-cb0c-4687-b4c0-83a5427ca80a" containerName="istio-proxy" Jul 14 02:35:53.911207 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:53.911156 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c1e8608-cb0c-4687-b4c0-83a5427ca80a" containerName="istio-proxy" Jul 14 02:35:53.911207 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:53.911164 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0" containerName="istio-proxy" Jul 14 02:35:53.911207 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:53.911171 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c1e8608-cb0c-4687-b4c0-83a5427ca80a" containerName="istio-proxy" Jul 14 02:35:53.911207 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:53.911178 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4bff5dc-ec1d-4abe-bdf5-d81f9e2cf2c0" containerName="istio-proxy" Jul 14 02:35:53.913663 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:53.913643 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:35:53.915741 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:53.915719 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-c40b4277-openshift-default-dockercfg-jg5tw\"" Jul 14 02:35:53.925032 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:53.925007 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm"] Jul 14 02:35:53.933333 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:53.933304 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:35:54.059989 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:54.059954 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7be6f530-5ba0-480a-ba50-24f8eb3fe693-istio-podinfo\") pod \"e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm\" (UID: \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\") " pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:35:54.060137 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:54.060026 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7be6f530-5ba0-480a-ba50-24f8eb3fe693-istio-data\") pod \"e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm\" (UID: \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\") " pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:35:54.060137 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:54.060058 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7be6f530-5ba0-480a-ba50-24f8eb3fe693-workload-socket\") pod \"e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm\" (UID: \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\") " pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:35:54.060137 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:54.060088 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7be6f530-5ba0-480a-ba50-24f8eb3fe693-istio-token\") pod \"e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm\" (UID: \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\") " pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:35:54.060312 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:54.060144 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7be6f530-5ba0-480a-ba50-24f8eb3fe693-istio-envoy\") pod \"e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm\" (UID: \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\") " pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:35:54.060312 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:54.060212 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7be6f530-5ba0-480a-ba50-24f8eb3fe693-istiod-ca-cert\") pod \"e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm\" (UID: \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\") " pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:35:54.060312 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:54.060289 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7be6f530-5ba0-480a-ba50-24f8eb3fe693-credential-socket\") pod \"e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm\" (UID: \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\") " pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:35:54.060421 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:54.060318 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7be6f530-5ba0-480a-ba50-24f8eb3fe693-workload-certs\") pod \"e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm\" (UID: \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\") " pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:35:54.060421 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:54.060341 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26z8f\" (UniqueName: \"kubernetes.io/projected/7be6f530-5ba0-480a-ba50-24f8eb3fe693-kube-api-access-26z8f\") pod \"e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm\" (UID: \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\") " pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:35:54.161769 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:54.161672 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7be6f530-5ba0-480a-ba50-24f8eb3fe693-istio-data\") pod \"e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm\" (UID: \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\") " pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:35:54.161769 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:54.161720 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7be6f530-5ba0-480a-ba50-24f8eb3fe693-workload-socket\") pod \"e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm\" (UID: \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\") " pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:35:54.161769 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:54.161766 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7be6f530-5ba0-480a-ba50-24f8eb3fe693-istio-token\") pod \"e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm\" (UID: \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\") " pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:35:54.162065 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:54.161792 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7be6f530-5ba0-480a-ba50-24f8eb3fe693-istio-envoy\") pod \"e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm\" (UID: \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\") " pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:35:54.162065 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:54.161925 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7be6f530-5ba0-480a-ba50-24f8eb3fe693-istiod-ca-cert\") pod \"e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm\" (UID: \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\") " pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:35:54.162065 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:54.161986 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7be6f530-5ba0-480a-ba50-24f8eb3fe693-credential-socket\") pod \"e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm\" (UID: \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\") " pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:35:54.162065 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:54.162018 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7be6f530-5ba0-480a-ba50-24f8eb3fe693-workload-certs\") pod \"e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm\" (UID: \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\") " pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:35:54.162065 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:54.162052 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26z8f\" (UniqueName: \"kubernetes.io/projected/7be6f530-5ba0-480a-ba50-24f8eb3fe693-kube-api-access-26z8f\") pod \"e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm\" (UID: \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\") " pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:35:54.162447 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:54.162091 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7be6f530-5ba0-480a-ba50-24f8eb3fe693-istio-podinfo\") pod \"e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm\" (UID: \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\") " pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:35:54.162447 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:54.162131 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7be6f530-5ba0-480a-ba50-24f8eb3fe693-istio-data\") pod \"e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm\" (UID: \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\") " pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:35:54.162447 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:54.162420 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7be6f530-5ba0-480a-ba50-24f8eb3fe693-credential-socket\") pod \"e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm\" (UID: \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\") " pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:35:54.162582 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:54.162470 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7be6f530-5ba0-480a-ba50-24f8eb3fe693-workload-socket\") pod \"e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm\" (UID: \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\") " pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:35:54.162582 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:54.162562 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7be6f530-5ba0-480a-ba50-24f8eb3fe693-workload-certs\") pod \"e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm\" (UID: \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\") " pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:35:54.162890 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:54.162865 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7be6f530-5ba0-480a-ba50-24f8eb3fe693-istiod-ca-cert\") pod \"e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm\" (UID: \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\") " pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:35:54.164235 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:54.164211 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7be6f530-5ba0-480a-ba50-24f8eb3fe693-istio-envoy\") pod \"e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm\" (UID: \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\") " pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:35:54.164604 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:54.164585 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7be6f530-5ba0-480a-ba50-24f8eb3fe693-istio-podinfo\") pod \"e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm\" (UID: \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\") " pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:35:54.169309 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:54.169264 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7be6f530-5ba0-480a-ba50-24f8eb3fe693-istio-token\") pod \"e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm\" (UID: \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\") " pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:35:54.169802 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:54.169782 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26z8f\" (UniqueName: \"kubernetes.io/projected/7be6f530-5ba0-480a-ba50-24f8eb3fe693-kube-api-access-26z8f\") pod \"e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm\" (UID: \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\") " pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:35:54.225003 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:54.224968 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:35:54.227661 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:54.227636 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:35:54.360965 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:54.360943 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm"] Jul 14 02:35:54.364101 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:35:54.363637 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7be6f530_5ba0_480a_ba50_24f8eb3fe693.slice/crio-3859d9f60a0470b892272912b2c9347c99a55175f5057f1eb3aefdf90b500c62 WatchSource:0}: Error finding container 3859d9f60a0470b892272912b2c9347c99a55175f5057f1eb3aefdf90b500c62: Status 404 returned error can't find the container with id 3859d9f60a0470b892272912b2c9347c99a55175f5057f1eb3aefdf90b500c62 Jul 14 02:35:54.366049 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:54.366019 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:35:54.366138 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:54.366081 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:35:54.366138 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:54.366108 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:35:54.920016 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:54.919981 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" event={"ID":"7be6f530-5ba0-480a-ba50-24f8eb3fe693","Type":"ContainerStarted","Data":"11e64316b7ccd184998f74667c7e1e7b1ede91a3863908568580a3cc5ebef396"} Jul 14 02:35:54.920424 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:54.920032 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" event={"ID":"7be6f530-5ba0-480a-ba50-24f8eb3fe693","Type":"ContainerStarted","Data":"3859d9f60a0470b892272912b2c9347c99a55175f5057f1eb3aefdf90b500c62"} Jul 14 02:35:54.945291 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:54.940861 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" podStartSLOduration=1.9408411650000001 podStartE2EDuration="1.940841165s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:35:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:35:54.938361088 +0000 UTC m=+2057.093197729" watchObservedRunningTime="2026-07-14 02:35:54.940841165 +0000 UTC m=+2057.095677814" Jul 14 02:35:55.225757 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:55.225658 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:35:55.230679 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:55.230654 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:35:55.925525 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:55.925495 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:35:55.926474 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:55.926458 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:35:56.848764 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:56.848732 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:35:56.956605 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:56.956571 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:35:57.076381 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:57.076349 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:35:57.249753 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:57.249720 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:35:57.632430 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:57.632397 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:35:57.733145 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:35:57.733114 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:36:01.336506 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:01.336411 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:36:01.985300 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:01.985240 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:36:02.083667 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:02.083636 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:36:02.411475 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:02.411438 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:36:08.015489 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:08.015447 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:36:08.113531 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:08.113492 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:36:09.998687 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:09.998652 2577 generic.go:356] "Generic (PLEG): container finished" podID="7be6f530-5ba0-480a-ba50-24f8eb3fe693" containerID="11e64316b7ccd184998f74667c7e1e7b1ede91a3863908568580a3cc5ebef396" exitCode=0 Jul 14 02:36:09.999118 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:09.998728 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" event={"ID":"7be6f530-5ba0-480a-ba50-24f8eb3fe693","Type":"ContainerDied","Data":"11e64316b7ccd184998f74667c7e1e7b1ede91a3863908568580a3cc5ebef396"} Jul 14 02:36:09.999183 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:09.999169 2577 scope.go:117] "RemoveContainer" containerID="11e64316b7ccd184998f74667c7e1e7b1ede91a3863908568580a3cc5ebef396" Jul 14 02:36:10.000179 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:10.000155 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:36:10.000234 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:10.000221 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:36:10.000295 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:10.000255 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:36:10.926403 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:10.926372 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:36:11.005224 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:11.005192 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" event={"ID":"7be6f530-5ba0-480a-ba50-24f8eb3fe693","Type":"ContainerStarted","Data":"9020fb7508285e60f2b50b560840f7e033bc2e2f3a96c99b943c81d78ee4e979"} Jul 14 02:36:11.225619 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:11.225523 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:36:11.226770 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:11.226743 2577 patch_prober.go:28] interesting pod/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 14 02:36:11.226882 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:11.226794 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" podUID="7be6f530-5ba0-480a-ba50-24f8eb3fe693" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 14 02:36:12.226033 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:12.225998 2577 patch_prober.go:28] interesting pod/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 14 02:36:12.226461 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:12.226064 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" podUID="7be6f530-5ba0-480a-ba50-24f8eb3fe693" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 14 02:36:13.225638 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:13.225598 2577 patch_prober.go:28] interesting pod/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 14 02:36:13.225820 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:13.225691 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" podUID="7be6f530-5ba0-480a-ba50-24f8eb3fe693" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 14 02:36:14.225733 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:14.225694 2577 patch_prober.go:28] interesting pod/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 14 02:36:14.226138 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:14.225779 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" podUID="7be6f530-5ba0-480a-ba50-24f8eb3fe693" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 14 02:36:15.229871 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:15.229839 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:36:15.230324 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:15.230119 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:36:15.230813 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:15.230796 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:36:18.739326 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:18.739294 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:36:23.449148 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:23.449112 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:36:29.320507 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:29.320473 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:36:29.377293 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:29.377228 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:36:34.109856 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:34.109815 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm"] Jul 14 02:36:34.110677 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:34.110622 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" podUID="7be6f530-5ba0-480a-ba50-24f8eb3fe693" containerName="istio-proxy" containerID="cri-o://9020fb7508285e60f2b50b560840f7e033bc2e2f3a96c99b943c81d78ee4e979" gracePeriod=30 Jul 14 02:36:34.135472 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:34.135435 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:36:34.355617 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:34.355575 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:36:34.474476 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:34.474437 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:36:34.574646 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:34.574614 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:36:34.702169 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:34.702136 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:36:37.801861 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:37.801823 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:36:39.359697 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:39.359665 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:36:39.415791 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:39.415707 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7be6f530-5ba0-480a-ba50-24f8eb3fe693-istiod-ca-cert\") pod \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\" (UID: \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\") " Jul 14 02:36:39.415791 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:39.415771 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7be6f530-5ba0-480a-ba50-24f8eb3fe693-credential-socket\") pod \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\" (UID: \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\") " Jul 14 02:36:39.416041 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:39.415799 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7be6f530-5ba0-480a-ba50-24f8eb3fe693-istio-podinfo\") pod \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\" (UID: \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\") " Jul 14 02:36:39.416041 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:39.415822 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7be6f530-5ba0-480a-ba50-24f8eb3fe693-istio-data\") pod \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\" (UID: \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\") " Jul 14 02:36:39.416041 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:39.415853 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-26z8f\" (UniqueName: \"kubernetes.io/projected/7be6f530-5ba0-480a-ba50-24f8eb3fe693-kube-api-access-26z8f\") pod \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\" (UID: \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\") " Jul 14 02:36:39.416041 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:39.415870 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7be6f530-5ba0-480a-ba50-24f8eb3fe693-istio-envoy\") pod \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\" (UID: \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\") " Jul 14 02:36:39.416041 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:39.415897 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7be6f530-5ba0-480a-ba50-24f8eb3fe693-istio-token\") pod \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\" (UID: \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\") " Jul 14 02:36:39.416041 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:39.415931 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7be6f530-5ba0-480a-ba50-24f8eb3fe693-workload-certs\") pod \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\" (UID: \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\") " Jul 14 02:36:39.416041 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:39.415956 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7be6f530-5ba0-480a-ba50-24f8eb3fe693-workload-socket\") pod \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\" (UID: \"7be6f530-5ba0-480a-ba50-24f8eb3fe693\") " Jul 14 02:36:39.416417 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:39.416161 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7be6f530-5ba0-480a-ba50-24f8eb3fe693-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7be6f530-5ba0-480a-ba50-24f8eb3fe693" (UID: "7be6f530-5ba0-480a-ba50-24f8eb3fe693"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:36:39.416417 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:39.416212 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7be6f530-5ba0-480a-ba50-24f8eb3fe693-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7be6f530-5ba0-480a-ba50-24f8eb3fe693" (UID: "7be6f530-5ba0-480a-ba50-24f8eb3fe693"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 02:36:39.416417 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:39.416388 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7be6f530-5ba0-480a-ba50-24f8eb3fe693-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7be6f530-5ba0-480a-ba50-24f8eb3fe693" (UID: "7be6f530-5ba0-480a-ba50-24f8eb3fe693"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:36:39.416417 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:39.416402 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7be6f530-5ba0-480a-ba50-24f8eb3fe693-istiod-ca-cert\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:36:39.416632 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:39.416422 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7be6f530-5ba0-480a-ba50-24f8eb3fe693-credential-socket\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:36:39.416632 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:39.416513 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7be6f530-5ba0-480a-ba50-24f8eb3fe693-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7be6f530-5ba0-480a-ba50-24f8eb3fe693" (UID: "7be6f530-5ba0-480a-ba50-24f8eb3fe693"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:36:39.416844 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:39.416821 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7be6f530-5ba0-480a-ba50-24f8eb3fe693-istio-data" (OuterVolumeSpecName: "istio-data") pod "7be6f530-5ba0-480a-ba50-24f8eb3fe693" (UID: "7be6f530-5ba0-480a-ba50-24f8eb3fe693"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:36:39.418093 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:39.418062 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7be6f530-5ba0-480a-ba50-24f8eb3fe693-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7be6f530-5ba0-480a-ba50-24f8eb3fe693" (UID: "7be6f530-5ba0-480a-ba50-24f8eb3fe693"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 02:36:39.418250 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:39.418231 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7be6f530-5ba0-480a-ba50-24f8eb3fe693-istio-token" (OuterVolumeSpecName: "istio-token") pod "7be6f530-5ba0-480a-ba50-24f8eb3fe693" (UID: "7be6f530-5ba0-480a-ba50-24f8eb3fe693"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:36:39.418567 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:39.418540 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7be6f530-5ba0-480a-ba50-24f8eb3fe693-kube-api-access-26z8f" (OuterVolumeSpecName: "kube-api-access-26z8f") pod "7be6f530-5ba0-480a-ba50-24f8eb3fe693" (UID: "7be6f530-5ba0-480a-ba50-24f8eb3fe693"). InnerVolumeSpecName "kube-api-access-26z8f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:36:39.418567 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:39.418549 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7be6f530-5ba0-480a-ba50-24f8eb3fe693-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7be6f530-5ba0-480a-ba50-24f8eb3fe693" (UID: "7be6f530-5ba0-480a-ba50-24f8eb3fe693"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:36:39.517045 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:39.517006 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7be6f530-5ba0-480a-ba50-24f8eb3fe693-workload-certs\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:36:39.517045 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:39.517037 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7be6f530-5ba0-480a-ba50-24f8eb3fe693-workload-socket\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:36:39.517045 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:39.517053 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7be6f530-5ba0-480a-ba50-24f8eb3fe693-istio-podinfo\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:36:39.517340 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:39.517064 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7be6f530-5ba0-480a-ba50-24f8eb3fe693-istio-data\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:36:39.517340 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:39.517076 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26z8f\" (UniqueName: \"kubernetes.io/projected/7be6f530-5ba0-480a-ba50-24f8eb3fe693-kube-api-access-26z8f\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:36:39.517340 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:39.517087 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7be6f530-5ba0-480a-ba50-24f8eb3fe693-istio-envoy\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:36:39.517340 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:39.517098 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7be6f530-5ba0-480a-ba50-24f8eb3fe693-istio-token\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:36:40.143768 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:40.143724 2577 generic.go:356] "Generic (PLEG): container finished" podID="7be6f530-5ba0-480a-ba50-24f8eb3fe693" containerID="9020fb7508285e60f2b50b560840f7e033bc2e2f3a96c99b943c81d78ee4e979" exitCode=0 Jul 14 02:36:40.143950 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:40.143781 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" Jul 14 02:36:40.143950 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:40.143790 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" event={"ID":"7be6f530-5ba0-480a-ba50-24f8eb3fe693","Type":"ContainerDied","Data":"9020fb7508285e60f2b50b560840f7e033bc2e2f3a96c99b943c81d78ee4e979"} Jul 14 02:36:40.143950 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:40.143875 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm" event={"ID":"7be6f530-5ba0-480a-ba50-24f8eb3fe693","Type":"ContainerDied","Data":"3859d9f60a0470b892272912b2c9347c99a55175f5057f1eb3aefdf90b500c62"} Jul 14 02:36:40.143950 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:40.143905 2577 scope.go:117] "RemoveContainer" containerID="9020fb7508285e60f2b50b560840f7e033bc2e2f3a96c99b943c81d78ee4e979" Jul 14 02:36:40.153212 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:40.153194 2577 scope.go:117] "RemoveContainer" containerID="11e64316b7ccd184998f74667c7e1e7b1ede91a3863908568580a3cc5ebef396" Jul 14 02:36:40.161358 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:40.161337 2577 scope.go:117] "RemoveContainer" containerID="9020fb7508285e60f2b50b560840f7e033bc2e2f3a96c99b943c81d78ee4e979" Jul 14 02:36:40.162715 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:36:40.161887 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9020fb7508285e60f2b50b560840f7e033bc2e2f3a96c99b943c81d78ee4e979\": container with ID starting with 9020fb7508285e60f2b50b560840f7e033bc2e2f3a96c99b943c81d78ee4e979 not found: ID does not exist" containerID="9020fb7508285e60f2b50b560840f7e033bc2e2f3a96c99b943c81d78ee4e979" Jul 14 02:36:40.162715 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:40.161932 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9020fb7508285e60f2b50b560840f7e033bc2e2f3a96c99b943c81d78ee4e979"} err="failed to get container status \"9020fb7508285e60f2b50b560840f7e033bc2e2f3a96c99b943c81d78ee4e979\": rpc error: code = NotFound desc = could not find container \"9020fb7508285e60f2b50b560840f7e033bc2e2f3a96c99b943c81d78ee4e979\": container with ID starting with 9020fb7508285e60f2b50b560840f7e033bc2e2f3a96c99b943c81d78ee4e979 not found: ID does not exist" Jul 14 02:36:40.162715 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:40.161957 2577 scope.go:117] "RemoveContainer" containerID="11e64316b7ccd184998f74667c7e1e7b1ede91a3863908568580a3cc5ebef396" Jul 14 02:36:40.163487 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:36:40.163456 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11e64316b7ccd184998f74667c7e1e7b1ede91a3863908568580a3cc5ebef396\": container with ID starting with 11e64316b7ccd184998f74667c7e1e7b1ede91a3863908568580a3cc5ebef396 not found: ID does not exist" containerID="11e64316b7ccd184998f74667c7e1e7b1ede91a3863908568580a3cc5ebef396" Jul 14 02:36:40.163588 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:40.163494 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11e64316b7ccd184998f74667c7e1e7b1ede91a3863908568580a3cc5ebef396"} err="failed to get container status \"11e64316b7ccd184998f74667c7e1e7b1ede91a3863908568580a3cc5ebef396\": rpc error: code = NotFound desc = could not find container \"11e64316b7ccd184998f74667c7e1e7b1ede91a3863908568580a3cc5ebef396\": container with ID starting with 11e64316b7ccd184998f74667c7e1e7b1ede91a3863908568580a3cc5ebef396 not found: ID does not exist" Jul 14 02:36:40.164285 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:40.164255 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm"] Jul 14 02:36:40.168635 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:40.168612 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-c40b4277-openshift-default-6d78d4b9c5-48dlm"] Jul 14 02:36:40.261771 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:40.261730 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:36:40.266127 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:40.266096 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7be6f530-5ba0-480a-ba50-24f8eb3fe693" path="/var/lib/kubelet/pods/7be6f530-5ba0-480a-ba50-24f8eb3fe693/volumes" Jul 14 02:36:40.319753 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:40.319720 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:36:40.396026 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:40.395916 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:36:40.487532 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:40.487489 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:36:40.613043 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:40.613014 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:36:46.095238 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:46.095184 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:36:46.198459 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:46.198408 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:36:50.431607 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:50.431573 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:36:52.884248 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:52.884213 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:36:53.172499 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.172421 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv"] Jul 14 02:36:53.172848 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.172835 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7be6f530-5ba0-480a-ba50-24f8eb3fe693" containerName="istio-proxy" Jul 14 02:36:53.172895 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.172849 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7be6f530-5ba0-480a-ba50-24f8eb3fe693" containerName="istio-proxy" Jul 14 02:36:53.172895 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.172864 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7be6f530-5ba0-480a-ba50-24f8eb3fe693" containerName="istio-proxy" Jul 14 02:36:53.172895 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.172870 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7be6f530-5ba0-480a-ba50-24f8eb3fe693" containerName="istio-proxy" Jul 14 02:36:53.172991 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.172923 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7be6f530-5ba0-480a-ba50-24f8eb3fe693" containerName="istio-proxy" Jul 14 02:36:53.172991 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.172933 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7be6f530-5ba0-480a-ba50-24f8eb3fe693" containerName="istio-proxy" Jul 14 02:36:53.176018 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.175998 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:36:53.178940 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.178919 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-ca6f969a-openshift-default-dockercfg-nhxmv\"" Jul 14 02:36:53.201906 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.201879 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv"] Jul 14 02:36:53.209635 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.209607 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:36:53.245870 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.245838 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-istio-token\") pod \"e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv\" (UID: \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\") " pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:36:53.245870 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.245875 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-workload-certs\") pod \"e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv\" (UID: \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\") " pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:36:53.246143 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.245909 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-istio-envoy\") pod \"e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv\" (UID: \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\") " pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:36:53.246143 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.246018 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-istio-data\") pod \"e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv\" (UID: \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\") " pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:36:53.246143 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.246112 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-istio-podinfo\") pod \"e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv\" (UID: \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\") " pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:36:53.246143 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.246139 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-credential-socket\") pod \"e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv\" (UID: \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\") " pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:36:53.246360 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.246208 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-istiod-ca-cert\") pod \"e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv\" (UID: \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\") " pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:36:53.246360 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.246243 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26w8p\" (UniqueName: \"kubernetes.io/projected/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-kube-api-access-26w8p\") pod \"e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv\" (UID: \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\") " pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:36:53.246360 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.246291 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-workload-socket\") pod \"e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv\" (UID: \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\") " pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:36:53.347301 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.347241 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-istio-podinfo\") pod \"e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv\" (UID: \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\") " pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:36:53.347512 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.347329 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-credential-socket\") pod \"e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv\" (UID: \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\") " pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:36:53.347512 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.347387 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-istiod-ca-cert\") pod \"e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv\" (UID: \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\") " pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:36:53.347512 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.347413 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26w8p\" (UniqueName: \"kubernetes.io/projected/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-kube-api-access-26w8p\") pod \"e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv\" (UID: \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\") " pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:36:53.347512 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.347445 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-workload-socket\") pod \"e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv\" (UID: \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\") " pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:36:53.347512 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.347485 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-istio-token\") pod \"e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv\" (UID: \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\") " pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:36:53.347787 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.347515 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-workload-certs\") pod \"e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv\" (UID: \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\") " pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:36:53.347787 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.347560 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-istio-envoy\") pod \"e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv\" (UID: \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\") " pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:36:53.347787 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.347623 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-istio-data\") pod \"e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv\" (UID: \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\") " pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:36:53.347931 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.347851 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-workload-socket\") pod \"e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv\" (UID: \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\") " pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:36:53.347990 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.347951 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-workload-certs\") pod \"e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv\" (UID: \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\") " pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:36:53.348045 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.348012 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-istio-data\") pod \"e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv\" (UID: \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\") " pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:36:53.348376 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.348348 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-istiod-ca-cert\") pod \"e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv\" (UID: \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\") " pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:36:53.348512 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.348437 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-credential-socket\") pod \"e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv\" (UID: \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\") " pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:36:53.350108 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.350087 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-istio-podinfo\") pod \"e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv\" (UID: \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\") " pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:36:53.350214 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.350157 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-istio-envoy\") pod \"e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv\" (UID: \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\") " pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:36:53.354400 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.354379 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-istio-token\") pod \"e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv\" (UID: \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\") " pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:36:53.354559 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.354540 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26w8p\" (UniqueName: \"kubernetes.io/projected/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-kube-api-access-26w8p\") pod \"e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv\" (UID: \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\") " pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:36:53.486033 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.485938 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:36:53.624713 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.624688 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv"] Jul 14 02:36:53.627535 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:36:53.627505 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6ef5e39_b413_48b3_ac2f_01fae3fb42b9.slice/crio-4f3d98100486d9b3d1f5dcbb4a1cc51cf9ab7346b02e7483424dc1fc5850d83a WatchSource:0}: Error finding container 4f3d98100486d9b3d1f5dcbb4a1cc51cf9ab7346b02e7483424dc1fc5850d83a: Status 404 returned error can't find the container with id 4f3d98100486d9b3d1f5dcbb4a1cc51cf9ab7346b02e7483424dc1fc5850d83a Jul 14 02:36:53.630104 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.630071 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:36:53.630183 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.630137 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:36:53.630183 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:53.630168 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:36:54.180755 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:54.180722 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:36:54.213775 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:54.213739 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" event={"ID":"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9","Type":"ContainerStarted","Data":"94ec8d1faefbb55e6d99e1c7681eb52436dc6b6f983da7d33cbaa178c985b212"} Jul 14 02:36:54.213999 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:54.213794 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" event={"ID":"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9","Type":"ContainerStarted","Data":"4f3d98100486d9b3d1f5dcbb4a1cc51cf9ab7346b02e7483424dc1fc5850d83a"} Jul 14 02:36:54.231436 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:54.231386 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" podStartSLOduration=1.231369136 podStartE2EDuration="1.231369136s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:36:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:36:54.229458004 +0000 UTC m=+2116.384294672" watchObservedRunningTime="2026-07-14 02:36:54.231369136 +0000 UTC m=+2116.386205783" Jul 14 02:36:54.486164 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:54.486076 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:36:54.490937 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:54.490909 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:36:55.219532 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:55.219502 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:36:55.220554 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:55.220535 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:36:56.103690 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:56.103653 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:36:56.176946 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:56.176912 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:36:56.238097 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:56.238061 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:36:56.361372 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:56.361319 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:36:56.744648 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:56.744562 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:36:56.845306 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:36:56.845249 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:37:00.610853 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:00.610813 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:37:01.648812 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:01.648779 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:37:01.749518 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:01.749480 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:37:10.221531 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:10.221491 2577 patch_prober.go:28] interesting pod/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 14 02:37:10.222031 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:10.221585 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" podUID="d6ef5e39-b413-48b3-ac2f-01fae3fb42b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 14 02:37:10.297866 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:10.297829 2577 generic.go:356] "Generic (PLEG): container finished" podID="d6ef5e39-b413-48b3-ac2f-01fae3fb42b9" containerID="94ec8d1faefbb55e6d99e1c7681eb52436dc6b6f983da7d33cbaa178c985b212" exitCode=0 Jul 14 02:37:10.298032 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:10.297900 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" event={"ID":"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9","Type":"ContainerDied","Data":"94ec8d1faefbb55e6d99e1c7681eb52436dc6b6f983da7d33cbaa178c985b212"} Jul 14 02:37:10.298337 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:10.298323 2577 scope.go:117] "RemoveContainer" containerID="94ec8d1faefbb55e6d99e1c7681eb52436dc6b6f983da7d33cbaa178c985b212" Jul 14 02:37:10.299398 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:10.299374 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:37:10.299484 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:10.299471 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:37:10.299522 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:10.299506 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:37:11.304258 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:11.304226 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" event={"ID":"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9","Type":"ContainerStarted","Data":"efb32ca916dab390a140554be67e83faedf0527d1967506db1a3b884ba75eb96"} Jul 14 02:37:11.487000 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:11.486959 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:37:11.487237 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:11.487213 2577 patch_prober.go:28] interesting pod/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 14 02:37:11.487358 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:11.487338 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" podUID="d6ef5e39-b413-48b3-ac2f-01fae3fb42b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 14 02:37:12.487345 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:12.487301 2577 patch_prober.go:28] interesting pod/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 14 02:37:12.487747 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:12.487383 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" podUID="d6ef5e39-b413-48b3-ac2f-01fae3fb42b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 14 02:37:13.486786 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:13.486753 2577 patch_prober.go:28] interesting pod/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 14 02:37:13.486970 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:13.486805 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" podUID="d6ef5e39-b413-48b3-ac2f-01fae3fb42b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 14 02:37:14.486989 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:14.486949 2577 patch_prober.go:28] interesting pod/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 14 02:37:14.487408 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:14.487018 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" podUID="d6ef5e39-b413-48b3-ac2f-01fae3fb42b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 14 02:37:15.388338 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:15.388300 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-ca6f969a/e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7"] Jul 14 02:37:15.391769 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:15.391748 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ca6f969a/e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7" Jul 14 02:37:15.394446 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:15.394420 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-ca6f969a\"/\"kube-root-ca.crt\"" Jul 14 02:37:15.394572 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:15.394463 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-ca6f969a\"/\"default-dockercfg-ncxd5\"" Jul 14 02:37:15.394572 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:15.394422 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-ca6f969a\"/\"openshift-service-ca.crt\"" Jul 14 02:37:15.394572 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:15.394432 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-ca6f969a\"/\"e2e-lifecycle-model-ca6f969a-kserve-self-signed-certs\"" Jul 14 02:37:15.398780 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:15.398758 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-ca6f969a/e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7"] Jul 14 02:37:15.475677 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:15.475644 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/837b90b1-8822-4160-8819-5ae11e0fb475-dshm\") pod \"e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7\" (UID: \"837b90b1-8822-4160-8819-5ae11e0fb475\") " pod="ai-tenant-e2e-mt-ca6f969a/e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7" Jul 14 02:37:15.475853 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:15.475696 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxmsx\" (UniqueName: \"kubernetes.io/projected/837b90b1-8822-4160-8819-5ae11e0fb475-kube-api-access-kxmsx\") pod \"e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7\" (UID: \"837b90b1-8822-4160-8819-5ae11e0fb475\") " pod="ai-tenant-e2e-mt-ca6f969a/e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7" Jul 14 02:37:15.475853 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:15.475750 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/837b90b1-8822-4160-8819-5ae11e0fb475-home\") pod \"e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7\" (UID: \"837b90b1-8822-4160-8819-5ae11e0fb475\") " pod="ai-tenant-e2e-mt-ca6f969a/e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7" Jul 14 02:37:15.475853 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:15.475773 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/837b90b1-8822-4160-8819-5ae11e0fb475-model-cache\") pod \"e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7\" (UID: \"837b90b1-8822-4160-8819-5ae11e0fb475\") " pod="ai-tenant-e2e-mt-ca6f969a/e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7" Jul 14 02:37:15.475853 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:15.475803 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/837b90b1-8822-4160-8819-5ae11e0fb475-tls-certs\") pod \"e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7\" (UID: \"837b90b1-8822-4160-8819-5ae11e0fb475\") " pod="ai-tenant-e2e-mt-ca6f969a/e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7" Jul 14 02:37:15.475853 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:15.475838 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/837b90b1-8822-4160-8819-5ae11e0fb475-tmp-dir\") pod \"e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7\" (UID: \"837b90b1-8822-4160-8819-5ae11e0fb475\") " pod="ai-tenant-e2e-mt-ca6f969a/e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7" Jul 14 02:37:15.481546 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:15.481521 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:37:15.490051 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:15.490029 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:37:15.490457 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:15.490376 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:37:15.491174 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:15.491158 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:37:15.577227 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:15.577195 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/837b90b1-8822-4160-8819-5ae11e0fb475-tmp-dir\") pod \"e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7\" (UID: \"837b90b1-8822-4160-8819-5ae11e0fb475\") " pod="ai-tenant-e2e-mt-ca6f969a/e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7" Jul 14 02:37:15.577452 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:15.577350 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/837b90b1-8822-4160-8819-5ae11e0fb475-dshm\") pod \"e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7\" (UID: \"837b90b1-8822-4160-8819-5ae11e0fb475\") " pod="ai-tenant-e2e-mt-ca6f969a/e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7" Jul 14 02:37:15.577697 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:15.577673 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/837b90b1-8822-4160-8819-5ae11e0fb475-tmp-dir\") pod \"e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7\" (UID: \"837b90b1-8822-4160-8819-5ae11e0fb475\") " pod="ai-tenant-e2e-mt-ca6f969a/e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7" Jul 14 02:37:15.577855 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:15.577833 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxmsx\" (UniqueName: \"kubernetes.io/projected/837b90b1-8822-4160-8819-5ae11e0fb475-kube-api-access-kxmsx\") pod \"e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7\" (UID: \"837b90b1-8822-4160-8819-5ae11e0fb475\") " pod="ai-tenant-e2e-mt-ca6f969a/e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7" Jul 14 02:37:15.578082 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:15.578058 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/837b90b1-8822-4160-8819-5ae11e0fb475-home\") pod \"e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7\" (UID: \"837b90b1-8822-4160-8819-5ae11e0fb475\") " pod="ai-tenant-e2e-mt-ca6f969a/e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7" Jul 14 02:37:15.578197 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:15.578105 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/837b90b1-8822-4160-8819-5ae11e0fb475-model-cache\") pod \"e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7\" (UID: \"837b90b1-8822-4160-8819-5ae11e0fb475\") " pod="ai-tenant-e2e-mt-ca6f969a/e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7" Jul 14 02:37:15.578197 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:15.578131 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/837b90b1-8822-4160-8819-5ae11e0fb475-tls-certs\") pod \"e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7\" (UID: \"837b90b1-8822-4160-8819-5ae11e0fb475\") " pod="ai-tenant-e2e-mt-ca6f969a/e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7" Jul 14 02:37:15.578428 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:15.578401 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/837b90b1-8822-4160-8819-5ae11e0fb475-home\") pod \"e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7\" (UID: \"837b90b1-8822-4160-8819-5ae11e0fb475\") " pod="ai-tenant-e2e-mt-ca6f969a/e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7" Jul 14 02:37:15.578488 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:15.578429 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/837b90b1-8822-4160-8819-5ae11e0fb475-model-cache\") pod \"e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7\" (UID: \"837b90b1-8822-4160-8819-5ae11e0fb475\") " pod="ai-tenant-e2e-mt-ca6f969a/e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7" Jul 14 02:37:15.579878 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:15.579856 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/837b90b1-8822-4160-8819-5ae11e0fb475-dshm\") pod \"e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7\" (UID: \"837b90b1-8822-4160-8819-5ae11e0fb475\") " pod="ai-tenant-e2e-mt-ca6f969a/e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7" Jul 14 02:37:15.580738 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:15.580719 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/837b90b1-8822-4160-8819-5ae11e0fb475-tls-certs\") pod \"e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7\" (UID: \"837b90b1-8822-4160-8819-5ae11e0fb475\") " pod="ai-tenant-e2e-mt-ca6f969a/e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7" Jul 14 02:37:15.584999 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:15.584974 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxmsx\" (UniqueName: \"kubernetes.io/projected/837b90b1-8822-4160-8819-5ae11e0fb475-kube-api-access-kxmsx\") pod \"e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7\" (UID: \"837b90b1-8822-4160-8819-5ae11e0fb475\") " pod="ai-tenant-e2e-mt-ca6f969a/e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7" Jul 14 02:37:15.702867 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:15.702768 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ca6f969a/e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7" Jul 14 02:37:15.845760 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:15.845678 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-ca6f969a/e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7"] Jul 14 02:37:15.848105 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:37:15.848077 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod837b90b1_8822_4160_8819_5ae11e0fb475.slice/crio-a1c0b86e7f5d2e35ec07e134554f8ae6cc2a97870ee139b311da0077a899e1f1 WatchSource:0}: Error finding container a1c0b86e7f5d2e35ec07e134554f8ae6cc2a97870ee139b311da0077a899e1f1: Status 404 returned error can't find the container with id a1c0b86e7f5d2e35ec07e134554f8ae6cc2a97870ee139b311da0077a899e1f1 Jul 14 02:37:16.330474 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:16.330438 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ca6f969a/e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7" event={"ID":"837b90b1-8822-4160-8819-5ae11e0fb475","Type":"ContainerStarted","Data":"775a480fefe3e57eb87c1dab9f8e8c44d9da1cbfd7777873b60093ec1cdc8a68"} Jul 14 02:37:16.330474 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:16.330483 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ca6f969a/e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7" event={"ID":"837b90b1-8822-4160-8819-5ae11e0fb475","Type":"ContainerStarted","Data":"a1c0b86e7f5d2e35ec07e134554f8ae6cc2a97870ee139b311da0077a899e1f1"} Jul 14 02:37:16.345547 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:16.345493 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-ca6f969a/e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7" podStartSLOduration=1.345475099 podStartE2EDuration="1.345475099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:37:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:37:16.344784094 +0000 UTC m=+2138.499620747" watchObservedRunningTime="2026-07-14 02:37:16.345475099 +0000 UTC m=+2138.500311747" Jul 14 02:37:17.413425 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:17.413388 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:37:17.514722 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:17.514692 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:37:25.703167 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:25.703127 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-ca6f969a/e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7" Jul 14 02:37:25.703703 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:25.703328 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-ca6f969a/e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7" Jul 14 02:37:25.716125 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:25.716091 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-ca6f969a/e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7" Jul 14 02:37:26.398500 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:26.398470 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-ca6f969a/e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7" Jul 14 02:37:49.329429 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:49.329383 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:37:53.930512 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:53.930476 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:37:58.914585 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:58.914544 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-ca6f969a/e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7"] Jul 14 02:37:59.010621 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:37:59.010587 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:38:13.937647 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:13.937602 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-ca6f969a/e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7" podUID="837b90b1-8822-4160-8819-5ae11e0fb475" containerName="main" containerID="cri-o://775a480fefe3e57eb87c1dab9f8e8c44d9da1cbfd7777873b60093ec1cdc8a68" gracePeriod=45 Jul 14 02:38:14.191888 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:14.191829 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ca6f969a/e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7" Jul 14 02:38:14.259419 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:14.259389 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kxmsx\" (UniqueName: \"kubernetes.io/projected/837b90b1-8822-4160-8819-5ae11e0fb475-kube-api-access-kxmsx\") pod \"837b90b1-8822-4160-8819-5ae11e0fb475\" (UID: \"837b90b1-8822-4160-8819-5ae11e0fb475\") " Jul 14 02:38:14.259583 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:14.259454 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/837b90b1-8822-4160-8819-5ae11e0fb475-model-cache\") pod \"837b90b1-8822-4160-8819-5ae11e0fb475\" (UID: \"837b90b1-8822-4160-8819-5ae11e0fb475\") " Jul 14 02:38:14.259583 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:14.259503 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/837b90b1-8822-4160-8819-5ae11e0fb475-tmp-dir\") pod \"837b90b1-8822-4160-8819-5ae11e0fb475\" (UID: \"837b90b1-8822-4160-8819-5ae11e0fb475\") " Jul 14 02:38:14.259583 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:14.259531 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/837b90b1-8822-4160-8819-5ae11e0fb475-tls-certs\") pod \"837b90b1-8822-4160-8819-5ae11e0fb475\" (UID: \"837b90b1-8822-4160-8819-5ae11e0fb475\") " Jul 14 02:38:14.259583 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:14.259571 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/837b90b1-8822-4160-8819-5ae11e0fb475-dshm\") pod \"837b90b1-8822-4160-8819-5ae11e0fb475\" (UID: \"837b90b1-8822-4160-8819-5ae11e0fb475\") " Jul 14 02:38:14.259776 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:14.259660 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/837b90b1-8822-4160-8819-5ae11e0fb475-home\") pod \"837b90b1-8822-4160-8819-5ae11e0fb475\" (UID: \"837b90b1-8822-4160-8819-5ae11e0fb475\") " Jul 14 02:38:14.259776 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:14.259726 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/837b90b1-8822-4160-8819-5ae11e0fb475-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "837b90b1-8822-4160-8819-5ae11e0fb475" (UID: "837b90b1-8822-4160-8819-5ae11e0fb475"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:38:14.259776 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:14.259746 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/837b90b1-8822-4160-8819-5ae11e0fb475-model-cache" (OuterVolumeSpecName: "model-cache") pod "837b90b1-8822-4160-8819-5ae11e0fb475" (UID: "837b90b1-8822-4160-8819-5ae11e0fb475"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:38:14.259993 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:14.259953 2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/837b90b1-8822-4160-8819-5ae11e0fb475-model-cache\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:38:14.259993 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:14.259974 2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/837b90b1-8822-4160-8819-5ae11e0fb475-tmp-dir\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:38:14.260092 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:14.259994 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/837b90b1-8822-4160-8819-5ae11e0fb475-home" (OuterVolumeSpecName: "home") pod "837b90b1-8822-4160-8819-5ae11e0fb475" (UID: "837b90b1-8822-4160-8819-5ae11e0fb475"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:38:14.261697 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:14.261673 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/837b90b1-8822-4160-8819-5ae11e0fb475-dshm" (OuterVolumeSpecName: "dshm") pod "837b90b1-8822-4160-8819-5ae11e0fb475" (UID: "837b90b1-8822-4160-8819-5ae11e0fb475"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:38:14.261697 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:14.261688 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/837b90b1-8822-4160-8819-5ae11e0fb475-kube-api-access-kxmsx" (OuterVolumeSpecName: "kube-api-access-kxmsx") pod "837b90b1-8822-4160-8819-5ae11e0fb475" (UID: "837b90b1-8822-4160-8819-5ae11e0fb475"). InnerVolumeSpecName "kube-api-access-kxmsx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:38:14.261931 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:14.261916 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/837b90b1-8822-4160-8819-5ae11e0fb475-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "837b90b1-8822-4160-8819-5ae11e0fb475" (UID: "837b90b1-8822-4160-8819-5ae11e0fb475"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 02:38:14.360566 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:14.360529 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxmsx\" (UniqueName: \"kubernetes.io/projected/837b90b1-8822-4160-8819-5ae11e0fb475-kube-api-access-kxmsx\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:38:14.360566 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:14.360559 2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/837b90b1-8822-4160-8819-5ae11e0fb475-tls-certs\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:38:14.360566 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:14.360570 2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/837b90b1-8822-4160-8819-5ae11e0fb475-dshm\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:38:14.360566 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:14.360578 2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/837b90b1-8822-4160-8819-5ae11e0fb475-home\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:38:14.614928 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:14.614893 2577 generic.go:356] "Generic (PLEG): container finished" podID="837b90b1-8822-4160-8819-5ae11e0fb475" containerID="775a480fefe3e57eb87c1dab9f8e8c44d9da1cbfd7777873b60093ec1cdc8a68" exitCode=0 Jul 14 02:38:14.615090 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:14.614933 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ca6f969a/e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7" Jul 14 02:38:14.615090 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:14.614977 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ca6f969a/e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7" event={"ID":"837b90b1-8822-4160-8819-5ae11e0fb475","Type":"ContainerDied","Data":"775a480fefe3e57eb87c1dab9f8e8c44d9da1cbfd7777873b60093ec1cdc8a68"} Jul 14 02:38:14.615090 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:14.615020 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ca6f969a/e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7" event={"ID":"837b90b1-8822-4160-8819-5ae11e0fb475","Type":"ContainerDied","Data":"a1c0b86e7f5d2e35ec07e134554f8ae6cc2a97870ee139b311da0077a899e1f1"} Jul 14 02:38:14.615090 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:14.615038 2577 scope.go:117] "RemoveContainer" containerID="775a480fefe3e57eb87c1dab9f8e8c44d9da1cbfd7777873b60093ec1cdc8a68" Jul 14 02:38:14.625767 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:14.625297 2577 scope.go:117] "RemoveContainer" containerID="775a480fefe3e57eb87c1dab9f8e8c44d9da1cbfd7777873b60093ec1cdc8a68" Jul 14 02:38:14.625767 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:38:14.625737 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"775a480fefe3e57eb87c1dab9f8e8c44d9da1cbfd7777873b60093ec1cdc8a68\": container with ID starting with 775a480fefe3e57eb87c1dab9f8e8c44d9da1cbfd7777873b60093ec1cdc8a68 not found: ID does not exist" containerID="775a480fefe3e57eb87c1dab9f8e8c44d9da1cbfd7777873b60093ec1cdc8a68" Jul 14 02:38:14.625915 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:14.625772 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"775a480fefe3e57eb87c1dab9f8e8c44d9da1cbfd7777873b60093ec1cdc8a68"} err="failed to get container status \"775a480fefe3e57eb87c1dab9f8e8c44d9da1cbfd7777873b60093ec1cdc8a68\": rpc error: code = NotFound desc = could not find container \"775a480fefe3e57eb87c1dab9f8e8c44d9da1cbfd7777873b60093ec1cdc8a68\": container with ID starting with 775a480fefe3e57eb87c1dab9f8e8c44d9da1cbfd7777873b60093ec1cdc8a68 not found: ID does not exist" Jul 14 02:38:14.629200 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:14.629177 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-ca6f969a/e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7"] Jul 14 02:38:14.631794 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:14.631772 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-ca6f969a/e2e-lifecycle-model-ca6f969a-kserve-6f84bb5cdc-5nsz7"] Jul 14 02:38:16.264466 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:16.264433 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="837b90b1-8822-4160-8819-5ae11e0fb475" path="/var/lib/kubelet/pods/837b90b1-8822-4160-8819-5ae11e0fb475/volumes" Jul 14 02:38:35.044688 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:35.044659 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:38:35.929353 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:35.929316 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:38:41.419238 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:41.419202 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:38:41.526878 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:41.526836 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:38:46.103980 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:46.103943 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv"] Jul 14 02:38:46.104520 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:46.104161 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" podUID="d6ef5e39-b413-48b3-ac2f-01fae3fb42b9" containerName="istio-proxy" containerID="cri-o://efb32ca916dab390a140554be67e83faedf0527d1967506db1a3b884ba75eb96" gracePeriod=30 Jul 14 02:38:46.122810 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:46.122778 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:38:46.339326 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:46.339290 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:38:47.375259 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:47.375224 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:38:47.475157 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:47.475116 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:38:47.575676 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:47.575626 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:38:48.033529 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:48.033493 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:38:48.275315 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:48.275262 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:38:48.376803 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:48.376774 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:38:48.474481 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:48.474445 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:38:49.834067 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:49.834034 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:38:51.366344 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.366257 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:38:51.414548 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.414521 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-istiod-ca-cert\") pod \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\" (UID: \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\") " Jul 14 02:38:51.414722 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.414559 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-credential-socket\") pod \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\" (UID: \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\") " Jul 14 02:38:51.414722 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.414637 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-istio-envoy\") pod \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\" (UID: \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\") " Jul 14 02:38:51.414722 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.414683 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-workload-socket\") pod \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\" (UID: \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\") " Jul 14 02:38:51.414862 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.414754 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-26w8p\" (UniqueName: \"kubernetes.io/projected/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-kube-api-access-26w8p\") pod \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\" (UID: \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\") " Jul 14 02:38:51.414862 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.414782 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-istio-token\") pod \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\" (UID: \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\") " Jul 14 02:38:51.414862 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.414803 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-istio-data\") pod \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\" (UID: \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\") " Jul 14 02:38:51.414862 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.414834 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-workload-certs\") pod \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\" (UID: \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\") " Jul 14 02:38:51.414862 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.414859 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-istio-podinfo\") pod \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\" (UID: \"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9\") " Jul 14 02:38:51.415108 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.414897 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d6ef5e39-b413-48b3-ac2f-01fae3fb42b9" (UID: "d6ef5e39-b413-48b3-ac2f-01fae3fb42b9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:38:51.415108 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.415008 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d6ef5e39-b413-48b3-ac2f-01fae3fb42b9" (UID: "d6ef5e39-b413-48b3-ac2f-01fae3fb42b9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 02:38:51.415108 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.415008 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d6ef5e39-b413-48b3-ac2f-01fae3fb42b9" (UID: "d6ef5e39-b413-48b3-ac2f-01fae3fb42b9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:38:51.415289 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.415162 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d6ef5e39-b413-48b3-ac2f-01fae3fb42b9" (UID: "d6ef5e39-b413-48b3-ac2f-01fae3fb42b9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:38:51.415289 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.415281 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-workload-socket\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:38:51.415412 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.415303 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-workload-certs\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:38:51.415412 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.415319 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-istiod-ca-cert\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:38:51.415412 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.415332 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-credential-socket\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:38:51.415830 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.415804 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-istio-data" (OuterVolumeSpecName: "istio-data") pod "d6ef5e39-b413-48b3-ac2f-01fae3fb42b9" (UID: "d6ef5e39-b413-48b3-ac2f-01fae3fb42b9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:38:51.416931 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.416902 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d6ef5e39-b413-48b3-ac2f-01fae3fb42b9" (UID: "d6ef5e39-b413-48b3-ac2f-01fae3fb42b9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:38:51.417104 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.417081 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-kube-api-access-26w8p" (OuterVolumeSpecName: "kube-api-access-26w8p") pod "d6ef5e39-b413-48b3-ac2f-01fae3fb42b9" (UID: "d6ef5e39-b413-48b3-ac2f-01fae3fb42b9"). InnerVolumeSpecName "kube-api-access-26w8p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:38:51.417380 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.417359 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-istio-token" (OuterVolumeSpecName: "istio-token") pod "d6ef5e39-b413-48b3-ac2f-01fae3fb42b9" (UID: "d6ef5e39-b413-48b3-ac2f-01fae3fb42b9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:38:51.417458 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.417393 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d6ef5e39-b413-48b3-ac2f-01fae3fb42b9" (UID: "d6ef5e39-b413-48b3-ac2f-01fae3fb42b9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 02:38:51.516016 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.515981 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-istio-envoy\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:38:51.516016 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.516010 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26w8p\" (UniqueName: \"kubernetes.io/projected/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-kube-api-access-26w8p\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:38:51.516016 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.516021 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-istio-token\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:38:51.516257 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.516030 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-istio-data\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:38:51.516257 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.516038 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9-istio-podinfo\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:38:51.793468 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.793367 2577 generic.go:356] "Generic (PLEG): container finished" podID="d6ef5e39-b413-48b3-ac2f-01fae3fb42b9" containerID="efb32ca916dab390a140554be67e83faedf0527d1967506db1a3b884ba75eb96" exitCode=0 Jul 14 02:38:51.793468 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.793418 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" Jul 14 02:38:51.793468 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.793455 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" event={"ID":"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9","Type":"ContainerDied","Data":"efb32ca916dab390a140554be67e83faedf0527d1967506db1a3b884ba75eb96"} Jul 14 02:38:51.793730 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.793506 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv" event={"ID":"d6ef5e39-b413-48b3-ac2f-01fae3fb42b9","Type":"ContainerDied","Data":"4f3d98100486d9b3d1f5dcbb4a1cc51cf9ab7346b02e7483424dc1fc5850d83a"} Jul 14 02:38:51.793730 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.793529 2577 scope.go:117] "RemoveContainer" containerID="efb32ca916dab390a140554be67e83faedf0527d1967506db1a3b884ba75eb96" Jul 14 02:38:51.803394 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.803376 2577 scope.go:117] "RemoveContainer" containerID="94ec8d1faefbb55e6d99e1c7681eb52436dc6b6f983da7d33cbaa178c985b212" Jul 14 02:38:51.810927 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.810908 2577 scope.go:117] "RemoveContainer" containerID="efb32ca916dab390a140554be67e83faedf0527d1967506db1a3b884ba75eb96" Jul 14 02:38:51.811166 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:38:51.811143 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efb32ca916dab390a140554be67e83faedf0527d1967506db1a3b884ba75eb96\": container with ID starting with efb32ca916dab390a140554be67e83faedf0527d1967506db1a3b884ba75eb96 not found: ID does not exist" containerID="efb32ca916dab390a140554be67e83faedf0527d1967506db1a3b884ba75eb96" Jul 14 02:38:51.811296 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.811177 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efb32ca916dab390a140554be67e83faedf0527d1967506db1a3b884ba75eb96"} err="failed to get container status \"efb32ca916dab390a140554be67e83faedf0527d1967506db1a3b884ba75eb96\": rpc error: code = NotFound desc = could not find container \"efb32ca916dab390a140554be67e83faedf0527d1967506db1a3b884ba75eb96\": container with ID starting with efb32ca916dab390a140554be67e83faedf0527d1967506db1a3b884ba75eb96 not found: ID does not exist" Jul 14 02:38:51.811296 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.811199 2577 scope.go:117] "RemoveContainer" containerID="94ec8d1faefbb55e6d99e1c7681eb52436dc6b6f983da7d33cbaa178c985b212" Jul 14 02:38:51.811491 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:38:51.811457 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94ec8d1faefbb55e6d99e1c7681eb52436dc6b6f983da7d33cbaa178c985b212\": container with ID starting with 94ec8d1faefbb55e6d99e1c7681eb52436dc6b6f983da7d33cbaa178c985b212 not found: ID does not exist" containerID="94ec8d1faefbb55e6d99e1c7681eb52436dc6b6f983da7d33cbaa178c985b212" Jul 14 02:38:51.811600 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.811493 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94ec8d1faefbb55e6d99e1c7681eb52436dc6b6f983da7d33cbaa178c985b212"} err="failed to get container status \"94ec8d1faefbb55e6d99e1c7681eb52436dc6b6f983da7d33cbaa178c985b212\": rpc error: code = NotFound desc = could not find container \"94ec8d1faefbb55e6d99e1c7681eb52436dc6b6f983da7d33cbaa178c985b212\": container with ID starting with 94ec8d1faefbb55e6d99e1c7681eb52436dc6b6f983da7d33cbaa178c985b212 not found: ID does not exist" Jul 14 02:38:51.814515 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.814491 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv"] Jul 14 02:38:51.817019 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:51.816987 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-ca6f969a-openshift-default-644f597567-qdfjv"] Jul 14 02:38:52.265148 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:52.265114 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6ef5e39-b413-48b3-ac2f-01fae3fb42b9" path="/var/lib/kubelet/pods/d6ef5e39-b413-48b3-ac2f-01fae3fb42b9/volumes" Jul 14 02:38:55.527368 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:55.527336 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:38:55.586610 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:55.586574 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:38:56.224327 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:38:56.224267 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:39:01.897090 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:01.897011 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:39:03.110978 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:03.110947 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:39:12.608026 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:12.607986 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:39:12.932080 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:12.932050 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:39:33.905116 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:33.905083 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:39:39.506532 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:39.506496 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:39:39.569417 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:39.569390 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:39:39.911046 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:39.911013 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:39:40.149974 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.149935 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq"] Jul 14 02:39:40.150348 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.150334 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="837b90b1-8822-4160-8819-5ae11e0fb475" containerName="main" Jul 14 02:39:40.150407 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.150349 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="837b90b1-8822-4160-8819-5ae11e0fb475" containerName="main" Jul 14 02:39:40.150407 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.150359 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6ef5e39-b413-48b3-ac2f-01fae3fb42b9" containerName="istio-proxy" Jul 14 02:39:40.150407 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.150366 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6ef5e39-b413-48b3-ac2f-01fae3fb42b9" containerName="istio-proxy" Jul 14 02:39:40.150407 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.150375 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6ef5e39-b413-48b3-ac2f-01fae3fb42b9" containerName="istio-proxy" Jul 14 02:39:40.150407 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.150381 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6ef5e39-b413-48b3-ac2f-01fae3fb42b9" containerName="istio-proxy" Jul 14 02:39:40.150603 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.150448 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6ef5e39-b413-48b3-ac2f-01fae3fb42b9" containerName="istio-proxy" Jul 14 02:39:40.150603 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.150456 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="837b90b1-8822-4160-8819-5ae11e0fb475" containerName="main" Jul 14 02:39:40.154287 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.154257 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:39:40.156159 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.156137 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-cc2dba-openshift-default-dockercfg-vnct4\"" Jul 14 02:39:40.162644 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.162595 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq"] Jul 14 02:39:40.168002 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.167980 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:39:40.295557 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.295524 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b94870a-c919-4efb-85a4-55f08a095bff-istio-token\") pod \"e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq\" (UID: \"0b94870a-c919-4efb-85a4-55f08a095bff\") " pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:39:40.295717 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.295587 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b94870a-c919-4efb-85a4-55f08a095bff-istio-data\") pod \"e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq\" (UID: \"0b94870a-c919-4efb-85a4-55f08a095bff\") " pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:39:40.295717 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.295606 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b94870a-c919-4efb-85a4-55f08a095bff-istiod-ca-cert\") pod \"e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq\" (UID: \"0b94870a-c919-4efb-85a4-55f08a095bff\") " pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:39:40.295717 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.295625 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b94870a-c919-4efb-85a4-55f08a095bff-workload-certs\") pod \"e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq\" (UID: \"0b94870a-c919-4efb-85a4-55f08a095bff\") " pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:39:40.295717 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.295648 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b94870a-c919-4efb-85a4-55f08a095bff-credential-socket\") pod \"e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq\" (UID: \"0b94870a-c919-4efb-85a4-55f08a095bff\") " pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:39:40.295858 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.295722 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b94870a-c919-4efb-85a4-55f08a095bff-workload-socket\") pod \"e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq\" (UID: \"0b94870a-c919-4efb-85a4-55f08a095bff\") " pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:39:40.295858 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.295738 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nphb\" (UniqueName: \"kubernetes.io/projected/0b94870a-c919-4efb-85a4-55f08a095bff-kube-api-access-8nphb\") pod \"e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq\" (UID: \"0b94870a-c919-4efb-85a4-55f08a095bff\") " pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:39:40.295858 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.295808 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b94870a-c919-4efb-85a4-55f08a095bff-istio-podinfo\") pod \"e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq\" (UID: \"0b94870a-c919-4efb-85a4-55f08a095bff\") " pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:39:40.295858 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.295841 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b94870a-c919-4efb-85a4-55f08a095bff-istio-envoy\") pod \"e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq\" (UID: \"0b94870a-c919-4efb-85a4-55f08a095bff\") " pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:39:40.396646 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.396608 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b94870a-c919-4efb-85a4-55f08a095bff-istio-data\") pod \"e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq\" (UID: \"0b94870a-c919-4efb-85a4-55f08a095bff\") " pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:39:40.396646 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.396645 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b94870a-c919-4efb-85a4-55f08a095bff-istiod-ca-cert\") pod \"e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq\" (UID: \"0b94870a-c919-4efb-85a4-55f08a095bff\") " pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:39:40.396857 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.396671 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b94870a-c919-4efb-85a4-55f08a095bff-workload-certs\") pod \"e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq\" (UID: \"0b94870a-c919-4efb-85a4-55f08a095bff\") " pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:39:40.396857 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.396705 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b94870a-c919-4efb-85a4-55f08a095bff-credential-socket\") pod \"e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq\" (UID: \"0b94870a-c919-4efb-85a4-55f08a095bff\") " pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:39:40.396857 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.396747 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b94870a-c919-4efb-85a4-55f08a095bff-workload-socket\") pod \"e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq\" (UID: \"0b94870a-c919-4efb-85a4-55f08a095bff\") " pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:39:40.396857 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.396770 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nphb\" (UniqueName: \"kubernetes.io/projected/0b94870a-c919-4efb-85a4-55f08a095bff-kube-api-access-8nphb\") pod \"e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq\" (UID: \"0b94870a-c919-4efb-85a4-55f08a095bff\") " pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:39:40.396857 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.396828 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b94870a-c919-4efb-85a4-55f08a095bff-istio-podinfo\") pod \"e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq\" (UID: \"0b94870a-c919-4efb-85a4-55f08a095bff\") " pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:39:40.396857 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.396857 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b94870a-c919-4efb-85a4-55f08a095bff-istio-envoy\") pod \"e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq\" (UID: \"0b94870a-c919-4efb-85a4-55f08a095bff\") " pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:39:40.397145 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.396902 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b94870a-c919-4efb-85a4-55f08a095bff-istio-token\") pod \"e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq\" (UID: \"0b94870a-c919-4efb-85a4-55f08a095bff\") " pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:39:40.397145 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.397090 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b94870a-c919-4efb-85a4-55f08a095bff-istio-data\") pod \"e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq\" (UID: \"0b94870a-c919-4efb-85a4-55f08a095bff\") " pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:39:40.397261 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.397190 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b94870a-c919-4efb-85a4-55f08a095bff-workload-certs\") pod \"e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq\" (UID: \"0b94870a-c919-4efb-85a4-55f08a095bff\") " pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:39:40.397261 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.397223 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b94870a-c919-4efb-85a4-55f08a095bff-credential-socket\") pod \"e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq\" (UID: \"0b94870a-c919-4efb-85a4-55f08a095bff\") " pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:39:40.397380 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.397300 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b94870a-c919-4efb-85a4-55f08a095bff-workload-socket\") pod \"e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq\" (UID: \"0b94870a-c919-4efb-85a4-55f08a095bff\") " pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:39:40.397484 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.397456 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b94870a-c919-4efb-85a4-55f08a095bff-istiod-ca-cert\") pod \"e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq\" (UID: \"0b94870a-c919-4efb-85a4-55f08a095bff\") " pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:39:40.399339 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.399323 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b94870a-c919-4efb-85a4-55f08a095bff-istio-envoy\") pod \"e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq\" (UID: \"0b94870a-c919-4efb-85a4-55f08a095bff\") " pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:39:40.399504 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.399488 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b94870a-c919-4efb-85a4-55f08a095bff-istio-podinfo\") pod \"e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq\" (UID: \"0b94870a-c919-4efb-85a4-55f08a095bff\") " pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:39:40.404127 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.404104 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b94870a-c919-4efb-85a4-55f08a095bff-istio-token\") pod \"e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq\" (UID: \"0b94870a-c919-4efb-85a4-55f08a095bff\") " pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:39:40.404569 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.404551 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nphb\" (UniqueName: \"kubernetes.io/projected/0b94870a-c919-4efb-85a4-55f08a095bff-kube-api-access-8nphb\") pod \"e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq\" (UID: \"0b94870a-c919-4efb-85a4-55f08a095bff\") " pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:39:40.465830 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.465755 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:39:40.509903 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.509874 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:39:40.607885 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.607776 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq"] Jul 14 02:39:40.610154 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:39:40.610128 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b94870a_c919_4efb_85a4_55f08a095bff.slice/crio-9f37e95c2d24536bb35185fa3bb855c9a8dfd2d0b5399c293c2a237197600e52 WatchSource:0}: Error finding container 9f37e95c2d24536bb35185fa3bb855c9a8dfd2d0b5399c293c2a237197600e52: Status 404 returned error can't find the container with id 9f37e95c2d24536bb35185fa3bb855c9a8dfd2d0b5399c293c2a237197600e52 Jul 14 02:39:40.612396 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.612355 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:39:40.612479 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.612444 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:39:40.612518 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:40.612489 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:39:41.041041 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:41.041002 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" event={"ID":"0b94870a-c919-4efb-85a4-55f08a095bff","Type":"ContainerStarted","Data":"21c43107949da5e8f45365d70d692965b668e2efc8ca67ed364c29913da521b5"} Jul 14 02:39:41.041203 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:41.041066 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" event={"ID":"0b94870a-c919-4efb-85a4-55f08a095bff","Type":"ContainerStarted","Data":"9f37e95c2d24536bb35185fa3bb855c9a8dfd2d0b5399c293c2a237197600e52"} Jul 14 02:39:41.060163 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:41.060110 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" podStartSLOduration=1.060095142 podStartE2EDuration="1.060095142s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:39:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:39:41.058749279 +0000 UTC m=+2283.213585927" watchObservedRunningTime="2026-07-14 02:39:41.060095142 +0000 UTC m=+2283.214931788" Jul 14 02:39:41.466121 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:41.466087 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:39:41.471331 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:41.471304 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:39:42.046847 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:42.046814 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:39:42.047790 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:42.047770 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:39:42.970233 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:42.970194 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:39:43.057532 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:43.057499 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:39:43.117365 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:43.117333 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:39:43.215642 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:43.215608 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:39:43.504559 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:43.504514 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:39:43.607041 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:43.607002 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:39:47.483162 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:47.483122 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:39:47.812319 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:47.812221 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:39:48.098895 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.098816 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b"] Jul 14 02:39:48.099238 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.099224 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6ef5e39-b413-48b3-ac2f-01fae3fb42b9" containerName="istio-proxy" Jul 14 02:39:48.108522 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.108498 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:39:48.111431 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.111405 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-7924bc-openshift-default-dockercfg-rckw7\"" Jul 14 02:39:48.114028 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.114002 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b"] Jul 14 02:39:48.126046 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.126023 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:39:48.164914 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.164882 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9kpv\" (UniqueName: \"kubernetes.io/projected/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-kube-api-access-g9kpv\") pod \"e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b\" (UID: \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\") " pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:39:48.165027 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.164938 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-istio-data\") pod \"e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b\" (UID: \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\") " pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:39:48.165027 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.165015 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-credential-socket\") pod \"e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b\" (UID: \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\") " pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:39:48.165127 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.165045 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-workload-certs\") pod \"e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b\" (UID: \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\") " pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:39:48.165127 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.165095 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-istio-envoy\") pod \"e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b\" (UID: \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\") " pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:39:48.165212 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.165125 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-istio-token\") pod \"e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b\" (UID: \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\") " pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:39:48.165212 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.165146 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-workload-socket\") pod \"e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b\" (UID: \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\") " pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:39:48.165212 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.165168 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-istiod-ca-cert\") pod \"e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b\" (UID: \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\") " pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:39:48.165367 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.165214 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-istio-podinfo\") pod \"e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b\" (UID: \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\") " pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:39:48.266061 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.266028 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-istio-envoy\") pod \"e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b\" (UID: \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\") " pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:39:48.266061 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.266064 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-istio-token\") pod \"e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b\" (UID: \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\") " pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:39:48.266386 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.266088 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-workload-socket\") pod \"e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b\" (UID: \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\") " pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:39:48.266386 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.266113 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-istiod-ca-cert\") pod \"e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b\" (UID: \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\") " pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:39:48.266386 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.266153 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-istio-podinfo\") pod \"e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b\" (UID: \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\") " pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:39:48.266386 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.266204 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9kpv\" (UniqueName: \"kubernetes.io/projected/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-kube-api-access-g9kpv\") pod \"e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b\" (UID: \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\") " pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:39:48.266386 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.266257 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-istio-data\") pod \"e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b\" (UID: \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\") " pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:39:48.266386 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.266335 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-credential-socket\") pod \"e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b\" (UID: \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\") " pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:39:48.266386 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.266386 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-workload-certs\") pod \"e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b\" (UID: \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\") " pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:39:48.266771 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.266560 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-workload-socket\") pod \"e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b\" (UID: \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\") " pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:39:48.266771 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.266697 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-istio-data\") pod \"e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b\" (UID: \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\") " pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:39:48.266771 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.266742 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-workload-certs\") pod \"e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b\" (UID: \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\") " pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:39:48.266887 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.266829 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-credential-socket\") pod \"e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b\" (UID: \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\") " pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:39:48.266927 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.266897 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-istiod-ca-cert\") pod \"e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b\" (UID: \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\") " pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:39:48.268464 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.268443 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-istio-envoy\") pod \"e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b\" (UID: \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\") " pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:39:48.268641 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.268622 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-istio-podinfo\") pod \"e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b\" (UID: \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\") " pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:39:48.277094 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.277069 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9kpv\" (UniqueName: \"kubernetes.io/projected/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-kube-api-access-g9kpv\") pod \"e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b\" (UID: \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\") " pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:39:48.277242 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.277223 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-istio-token\") pod \"e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b\" (UID: \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\") " pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:39:48.422181 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.422153 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:39:48.557506 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.557478 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b"] Jul 14 02:39:48.558683 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:39:48.558661 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bba6bbd_d81f_4872_a8e6_2168e0c9079f.slice/crio-ac02697a2d2209fb96048365bb4662d5262c8084eefae5210648c76254fb5e20 WatchSource:0}: Error finding container ac02697a2d2209fb96048365bb4662d5262c8084eefae5210648c76254fb5e20: Status 404 returned error can't find the container with id ac02697a2d2209fb96048365bb4662d5262c8084eefae5210648c76254fb5e20 Jul 14 02:39:48.560925 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.560891 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:39:48.561019 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.560953 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:39:48.561019 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.560980 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:39:48.809810 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:48.809716 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:39:49.081295 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:49.081191 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" event={"ID":"7bba6bbd-d81f-4872-a8e6-2168e0c9079f","Type":"ContainerStarted","Data":"9b4ce6927eaf9ba540f833d42c4302aac753d9655c5601681db851c7c682064f"} Jul 14 02:39:49.081295 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:49.081240 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" event={"ID":"7bba6bbd-d81f-4872-a8e6-2168e0c9079f","Type":"ContainerStarted","Data":"ac02697a2d2209fb96048365bb4662d5262c8084eefae5210648c76254fb5e20"} Jul 14 02:39:49.097417 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:49.097364 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" podStartSLOduration=1.097348088 podStartE2EDuration="1.097348088s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:39:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:39:49.096385111 +0000 UTC m=+2291.251221759" watchObservedRunningTime="2026-07-14 02:39:49.097348088 +0000 UTC m=+2291.252184735" Jul 14 02:39:49.422553 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:49.422507 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:39:49.427579 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:49.427555 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:39:50.086116 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:50.086083 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:39:50.087123 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:50.087098 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:39:50.999287 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:50.999234 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:39:51.069452 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:51.069406 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:39:51.135719 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:51.135686 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:39:51.238548 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:51.238517 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:39:51.535886 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:51.535841 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:39:51.641351 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:51.641319 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:39:55.509764 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:55.509728 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:39:58.088485 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:58.088446 2577 patch_prober.go:28] interesting pod/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 14 02:39:58.088899 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:58.088525 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" podUID="0b94870a-c919-4efb-85a4-55f08a095bff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 14 02:39:59.692327 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:59.692285 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-cc2dba/test-model-cc2dba-kserve-59f95744f9-5czmn"] Jul 14 02:39:59.695972 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:59.695952 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-cc2dba/test-model-cc2dba-kserve-59f95744f9-5czmn" Jul 14 02:39:59.698399 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:59.698377 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-cc2dba\"/\"default-dockercfg-wd2vs\"" Jul 14 02:39:59.698539 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:59.698408 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-cc2dba\"/\"openshift-service-ca.crt\"" Jul 14 02:39:59.698539 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:59.698379 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-cc2dba\"/\"kube-root-ca.crt\"" Jul 14 02:39:59.698539 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:59.698443 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-cc2dba\"/\"test-model-cc2dba-kserve-self-signed-certs\"" Jul 14 02:39:59.702204 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:59.702181 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-cc2dba/test-model-cc2dba-kserve-59f95744f9-5czmn"] Jul 14 02:39:59.789491 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:59.789458 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:39:59.798715 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:59.798687 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9fc1527-6fcb-4af4-a103-02ca29811bac-model-cache\") pod \"test-model-cc2dba-kserve-59f95744f9-5czmn\" (UID: \"a9fc1527-6fcb-4af4-a103-02ca29811bac\") " pod="ai-tenant-e2e-inf-a-cc2dba/test-model-cc2dba-kserve-59f95744f9-5czmn" Jul 14 02:39:59.798876 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:59.798722 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9fc1527-6fcb-4af4-a103-02ca29811bac-dshm\") pod \"test-model-cc2dba-kserve-59f95744f9-5czmn\" (UID: \"a9fc1527-6fcb-4af4-a103-02ca29811bac\") " pod="ai-tenant-e2e-inf-a-cc2dba/test-model-cc2dba-kserve-59f95744f9-5czmn" Jul 14 02:39:59.798876 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:59.798752 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9fc1527-6fcb-4af4-a103-02ca29811bac-home\") pod \"test-model-cc2dba-kserve-59f95744f9-5czmn\" (UID: \"a9fc1527-6fcb-4af4-a103-02ca29811bac\") " pod="ai-tenant-e2e-inf-a-cc2dba/test-model-cc2dba-kserve-59f95744f9-5czmn" Jul 14 02:39:59.798876 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:59.798820 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdk2w\" (UniqueName: \"kubernetes.io/projected/a9fc1527-6fcb-4af4-a103-02ca29811bac-kube-api-access-fdk2w\") pod \"test-model-cc2dba-kserve-59f95744f9-5czmn\" (UID: \"a9fc1527-6fcb-4af4-a103-02ca29811bac\") " pod="ai-tenant-e2e-inf-a-cc2dba/test-model-cc2dba-kserve-59f95744f9-5czmn" Jul 14 02:39:59.798983 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:59.798893 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9fc1527-6fcb-4af4-a103-02ca29811bac-tmp-dir\") pod \"test-model-cc2dba-kserve-59f95744f9-5czmn\" (UID: \"a9fc1527-6fcb-4af4-a103-02ca29811bac\") " pod="ai-tenant-e2e-inf-a-cc2dba/test-model-cc2dba-kserve-59f95744f9-5czmn" Jul 14 02:39:59.798983 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:59.798938 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9fc1527-6fcb-4af4-a103-02ca29811bac-tls-certs\") pod \"test-model-cc2dba-kserve-59f95744f9-5czmn\" (UID: \"a9fc1527-6fcb-4af4-a103-02ca29811bac\") " pod="ai-tenant-e2e-inf-a-cc2dba/test-model-cc2dba-kserve-59f95744f9-5czmn" Jul 14 02:39:59.899883 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:59.899785 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9fc1527-6fcb-4af4-a103-02ca29811bac-dshm\") pod \"test-model-cc2dba-kserve-59f95744f9-5czmn\" (UID: \"a9fc1527-6fcb-4af4-a103-02ca29811bac\") " pod="ai-tenant-e2e-inf-a-cc2dba/test-model-cc2dba-kserve-59f95744f9-5czmn" Jul 14 02:39:59.899883 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:59.899835 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9fc1527-6fcb-4af4-a103-02ca29811bac-home\") pod \"test-model-cc2dba-kserve-59f95744f9-5czmn\" (UID: \"a9fc1527-6fcb-4af4-a103-02ca29811bac\") " pod="ai-tenant-e2e-inf-a-cc2dba/test-model-cc2dba-kserve-59f95744f9-5czmn" Jul 14 02:39:59.900120 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:59.899900 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdk2w\" (UniqueName: \"kubernetes.io/projected/a9fc1527-6fcb-4af4-a103-02ca29811bac-kube-api-access-fdk2w\") pod \"test-model-cc2dba-kserve-59f95744f9-5czmn\" (UID: \"a9fc1527-6fcb-4af4-a103-02ca29811bac\") " pod="ai-tenant-e2e-inf-a-cc2dba/test-model-cc2dba-kserve-59f95744f9-5czmn" Jul 14 02:39:59.900120 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:59.899927 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9fc1527-6fcb-4af4-a103-02ca29811bac-tmp-dir\") pod \"test-model-cc2dba-kserve-59f95744f9-5czmn\" (UID: \"a9fc1527-6fcb-4af4-a103-02ca29811bac\") " pod="ai-tenant-e2e-inf-a-cc2dba/test-model-cc2dba-kserve-59f95744f9-5czmn" Jul 14 02:39:59.900332 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:59.900304 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9fc1527-6fcb-4af4-a103-02ca29811bac-tls-certs\") pod \"test-model-cc2dba-kserve-59f95744f9-5czmn\" (UID: \"a9fc1527-6fcb-4af4-a103-02ca29811bac\") " pod="ai-tenant-e2e-inf-a-cc2dba/test-model-cc2dba-kserve-59f95744f9-5czmn" Jul 14 02:39:59.900428 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:59.900380 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9fc1527-6fcb-4af4-a103-02ca29811bac-tmp-dir\") pod \"test-model-cc2dba-kserve-59f95744f9-5czmn\" (UID: \"a9fc1527-6fcb-4af4-a103-02ca29811bac\") " pod="ai-tenant-e2e-inf-a-cc2dba/test-model-cc2dba-kserve-59f95744f9-5czmn" Jul 14 02:39:59.900497 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:59.900466 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9fc1527-6fcb-4af4-a103-02ca29811bac-model-cache\") pod \"test-model-cc2dba-kserve-59f95744f9-5czmn\" (UID: \"a9fc1527-6fcb-4af4-a103-02ca29811bac\") " pod="ai-tenant-e2e-inf-a-cc2dba/test-model-cc2dba-kserve-59f95744f9-5czmn" Jul 14 02:39:59.900556 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:59.900536 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9fc1527-6fcb-4af4-a103-02ca29811bac-home\") pod \"test-model-cc2dba-kserve-59f95744f9-5czmn\" (UID: \"a9fc1527-6fcb-4af4-a103-02ca29811bac\") " pod="ai-tenant-e2e-inf-a-cc2dba/test-model-cc2dba-kserve-59f95744f9-5czmn" Jul 14 02:39:59.900824 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:59.900785 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9fc1527-6fcb-4af4-a103-02ca29811bac-model-cache\") pod \"test-model-cc2dba-kserve-59f95744f9-5czmn\" (UID: \"a9fc1527-6fcb-4af4-a103-02ca29811bac\") " pod="ai-tenant-e2e-inf-a-cc2dba/test-model-cc2dba-kserve-59f95744f9-5czmn" Jul 14 02:39:59.903544 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:59.903521 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9fc1527-6fcb-4af4-a103-02ca29811bac-tls-certs\") pod \"test-model-cc2dba-kserve-59f95744f9-5czmn\" (UID: \"a9fc1527-6fcb-4af4-a103-02ca29811bac\") " pod="ai-tenant-e2e-inf-a-cc2dba/test-model-cc2dba-kserve-59f95744f9-5czmn" Jul 14 02:39:59.903862 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:59.903771 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9fc1527-6fcb-4af4-a103-02ca29811bac-dshm\") pod \"test-model-cc2dba-kserve-59f95744f9-5czmn\" (UID: \"a9fc1527-6fcb-4af4-a103-02ca29811bac\") " pod="ai-tenant-e2e-inf-a-cc2dba/test-model-cc2dba-kserve-59f95744f9-5czmn" Jul 14 02:39:59.907123 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:39:59.907092 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdk2w\" (UniqueName: \"kubernetes.io/projected/a9fc1527-6fcb-4af4-a103-02ca29811bac-kube-api-access-fdk2w\") pod \"test-model-cc2dba-kserve-59f95744f9-5czmn\" (UID: \"a9fc1527-6fcb-4af4-a103-02ca29811bac\") " pod="ai-tenant-e2e-inf-a-cc2dba/test-model-cc2dba-kserve-59f95744f9-5czmn" Jul 14 02:40:00.008386 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:00.008286 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-cc2dba/test-model-cc2dba-kserve-59f95744f9-5czmn" Jul 14 02:40:00.140175 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:00.140144 2577 generic.go:356] "Generic (PLEG): container finished" podID="0b94870a-c919-4efb-85a4-55f08a095bff" containerID="21c43107949da5e8f45365d70d692965b668e2efc8ca67ed364c29913da521b5" exitCode=0 Jul 14 02:40:00.140360 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:00.140217 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" event={"ID":"0b94870a-c919-4efb-85a4-55f08a095bff","Type":"ContainerDied","Data":"21c43107949da5e8f45365d70d692965b668e2efc8ca67ed364c29913da521b5"} Jul 14 02:40:00.140677 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:00.140661 2577 scope.go:117] "RemoveContainer" containerID="21c43107949da5e8f45365d70d692965b668e2efc8ca67ed364c29913da521b5" Jul 14 02:40:00.141752 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:00.141723 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:40:00.141844 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:00.141785 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:40:00.141844 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:00.141816 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:40:00.145111 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:00.145085 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-cc2dba/test-model-cc2dba-kserve-59f95744f9-5czmn"] Jul 14 02:40:00.149982 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:40:00.149954 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda9fc1527_6fcb_4af4_a103_02ca29811bac.slice/crio-f9662b5cf4eae22d955c39365a6b422c2c55bd506e3b464eb708eddd99ca34bf WatchSource:0}: Error finding container f9662b5cf4eae22d955c39365a6b422c2c55bd506e3b464eb708eddd99ca34bf: Status 404 returned error can't find the container with id f9662b5cf4eae22d955c39365a6b422c2c55bd506e3b464eb708eddd99ca34bf Jul 14 02:40:01.151144 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:01.150253 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-cc2dba/test-model-cc2dba-kserve-59f95744f9-5czmn" event={"ID":"a9fc1527-6fcb-4af4-a103-02ca29811bac","Type":"ContainerStarted","Data":"0de0244bebabc5314520a0442d5faa0a16d9fe13dfe2d00f784533d5d6f2705e"} Jul 14 02:40:01.151144 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:01.150321 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-cc2dba/test-model-cc2dba-kserve-59f95744f9-5czmn" event={"ID":"a9fc1527-6fcb-4af4-a103-02ca29811bac","Type":"ContainerStarted","Data":"f9662b5cf4eae22d955c39365a6b422c2c55bd506e3b464eb708eddd99ca34bf"} Jul 14 02:40:01.156197 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:01.155488 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" event={"ID":"0b94870a-c919-4efb-85a4-55f08a095bff","Type":"ContainerStarted","Data":"3fa5b538ff7be5482cda4188be444bbcba44072373193d9e3443e67dd826b2e8"} Jul 14 02:40:01.165466 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:01.165390 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-cc2dba/test-model-cc2dba-kserve-59f95744f9-5czmn" podStartSLOduration=2.165368307 podStartE2EDuration="2.165368307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:39:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:40:01.16512942 +0000 UTC m=+2303.319966067" watchObservedRunningTime="2026-07-14 02:40:01.165368307 +0000 UTC m=+2303.320204956" Jul 14 02:40:01.466816 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:01.466641 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:40:01.466816 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:01.466677 2577 patch_prober.go:28] interesting pod/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 14 02:40:01.466816 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:01.466731 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" podUID="0b94870a-c919-4efb-85a4-55f08a095bff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 14 02:40:02.466727 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:02.466683 2577 patch_prober.go:28] interesting pod/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 14 02:40:02.467246 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:02.466756 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" podUID="0b94870a-c919-4efb-85a4-55f08a095bff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 14 02:40:03.466977 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:03.466935 2577 patch_prober.go:28] interesting pod/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 14 02:40:03.467603 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:03.467569 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" podUID="0b94870a-c919-4efb-85a4-55f08a095bff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 14 02:40:04.466720 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:04.466679 2577 patch_prober.go:28] interesting pod/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 14 02:40:04.467076 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:04.467046 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" podUID="0b94870a-c919-4efb-85a4-55f08a095bff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 14 02:40:04.922673 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:04.922644 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:40:05.022481 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:05.022448 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:40:05.469586 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:05.469554 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:40:05.469945 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:05.469898 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:40:05.470700 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:05.470683 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:40:08.200732 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:08.200648 2577 generic.go:356] "Generic (PLEG): container finished" podID="7bba6bbd-d81f-4872-a8e6-2168e0c9079f" containerID="9b4ce6927eaf9ba540f833d42c4302aac753d9655c5601681db851c7c682064f" exitCode=0 Jul 14 02:40:08.200732 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:08.200723 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" event={"ID":"7bba6bbd-d81f-4872-a8e6-2168e0c9079f","Type":"ContainerDied","Data":"9b4ce6927eaf9ba540f833d42c4302aac753d9655c5601681db851c7c682064f"} Jul 14 02:40:08.201175 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:08.201109 2577 scope.go:117] "RemoveContainer" containerID="9b4ce6927eaf9ba540f833d42c4302aac753d9655c5601681db851c7c682064f" Jul 14 02:40:08.202084 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:08.202061 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:40:08.202133 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:08.202124 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:40:08.202177 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:08.202153 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:40:09.207899 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:09.207861 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" event={"ID":"7bba6bbd-d81f-4872-a8e6-2168e0c9079f","Type":"ContainerStarted","Data":"4df6abc04a838272d30454bd3a486ee35e717d9139ea8abb7c639d4a8146efb0"} Jul 14 02:40:09.422734 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:09.422690 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:40:09.423019 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:09.422995 2577 patch_prober.go:28] interesting pod/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 14 02:40:09.423125 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:09.423060 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" podUID="7bba6bbd-d81f-4872-a8e6-2168e0c9079f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 14 02:40:10.008855 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:10.008813 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-cc2dba/test-model-cc2dba-kserve-59f95744f9-5czmn" Jul 14 02:40:10.008855 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:10.008861 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-cc2dba/test-model-cc2dba-kserve-59f95744f9-5czmn" Jul 14 02:40:10.021179 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:10.021152 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-cc2dba/test-model-cc2dba-kserve-59f95744f9-5czmn" Jul 14 02:40:10.224412 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:10.224379 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-cc2dba/test-model-cc2dba-kserve-59f95744f9-5czmn" Jul 14 02:40:10.320810 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:10.320720 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:40:10.423446 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:10.423411 2577 patch_prober.go:28] interesting pod/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 14 02:40:10.423610 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:10.423461 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" podUID="7bba6bbd-d81f-4872-a8e6-2168e0c9079f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 14 02:40:11.423080 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:11.423041 2577 patch_prober.go:28] interesting pod/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 14 02:40:11.423495 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:11.423097 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" podUID="7bba6bbd-d81f-4872-a8e6-2168e0c9079f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 14 02:40:12.317586 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:12.317552 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:40:12.416182 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:12.416146 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:40:12.423296 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:12.423237 2577 patch_prober.go:28] interesting pod/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 14 02:40:12.423666 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:12.423319 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" podUID="7bba6bbd-d81f-4872-a8e6-2168e0c9079f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 14 02:40:13.426676 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:13.426644 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:40:13.427132 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:13.426984 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:40:13.427637 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:13.427618 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:40:29.511166 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:29.511130 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:40:48.851471 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:48.851436 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-cc2dba/test-model-cc2dba-kserve-59f95744f9-5czmn"] Jul 14 02:40:48.860961 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:48.860921 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:40:49.840444 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:49.840416 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:40:50.498825 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:50.498793 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:40:50.599153 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:40:50.599126 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:41:00.292584 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:00.292536 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:41:03.874415 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:03.874369 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-cc2dba/test-model-cc2dba-kserve-59f95744f9-5czmn" podUID="a9fc1527-6fcb-4af4-a103-02ca29811bac" containerName="main" containerID="cri-o://0de0244bebabc5314520a0442d5faa0a16d9fe13dfe2d00f784533d5d6f2705e" gracePeriod=45 Jul 14 02:41:04.121388 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:04.121361 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-cc2dba/test-model-cc2dba-kserve-59f95744f9-5czmn" Jul 14 02:41:04.263232 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:04.263144 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9fc1527-6fcb-4af4-a103-02ca29811bac-dshm\") pod \"a9fc1527-6fcb-4af4-a103-02ca29811bac\" (UID: \"a9fc1527-6fcb-4af4-a103-02ca29811bac\") " Jul 14 02:41:04.263232 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:04.263213 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdk2w\" (UniqueName: \"kubernetes.io/projected/a9fc1527-6fcb-4af4-a103-02ca29811bac-kube-api-access-fdk2w\") pod \"a9fc1527-6fcb-4af4-a103-02ca29811bac\" (UID: \"a9fc1527-6fcb-4af4-a103-02ca29811bac\") " Jul 14 02:41:04.263483 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:04.263235 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9fc1527-6fcb-4af4-a103-02ca29811bac-tmp-dir\") pod \"a9fc1527-6fcb-4af4-a103-02ca29811bac\" (UID: \"a9fc1527-6fcb-4af4-a103-02ca29811bac\") " Jul 14 02:41:04.263483 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:04.263256 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9fc1527-6fcb-4af4-a103-02ca29811bac-model-cache\") pod \"a9fc1527-6fcb-4af4-a103-02ca29811bac\" (UID: \"a9fc1527-6fcb-4af4-a103-02ca29811bac\") " Jul 14 02:41:04.263483 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:04.263373 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9fc1527-6fcb-4af4-a103-02ca29811bac-home\") pod \"a9fc1527-6fcb-4af4-a103-02ca29811bac\" (UID: \"a9fc1527-6fcb-4af4-a103-02ca29811bac\") " Jul 14 02:41:04.263483 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:04.263421 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9fc1527-6fcb-4af4-a103-02ca29811bac-tls-certs\") pod \"a9fc1527-6fcb-4af4-a103-02ca29811bac\" (UID: \"a9fc1527-6fcb-4af4-a103-02ca29811bac\") " Jul 14 02:41:04.263692 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:04.263511 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9fc1527-6fcb-4af4-a103-02ca29811bac-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a9fc1527-6fcb-4af4-a103-02ca29811bac" (UID: "a9fc1527-6fcb-4af4-a103-02ca29811bac"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:41:04.263692 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:04.263677 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9fc1527-6fcb-4af4-a103-02ca29811bac-model-cache" (OuterVolumeSpecName: "model-cache") pod "a9fc1527-6fcb-4af4-a103-02ca29811bac" (UID: "a9fc1527-6fcb-4af4-a103-02ca29811bac"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:41:04.263795 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:04.263773 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9fc1527-6fcb-4af4-a103-02ca29811bac-home" (OuterVolumeSpecName: "home") pod "a9fc1527-6fcb-4af4-a103-02ca29811bac" (UID: "a9fc1527-6fcb-4af4-a103-02ca29811bac"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:41:04.263869 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:04.263851 2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9fc1527-6fcb-4af4-a103-02ca29811bac-tmp-dir\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:41:04.263924 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:04.263874 2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9fc1527-6fcb-4af4-a103-02ca29811bac-model-cache\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:41:04.265605 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:04.265580 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9fc1527-6fcb-4af4-a103-02ca29811bac-dshm" (OuterVolumeSpecName: "dshm") pod "a9fc1527-6fcb-4af4-a103-02ca29811bac" (UID: "a9fc1527-6fcb-4af4-a103-02ca29811bac"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:41:04.265605 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:04.265586 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9fc1527-6fcb-4af4-a103-02ca29811bac-kube-api-access-fdk2w" (OuterVolumeSpecName: "kube-api-access-fdk2w") pod "a9fc1527-6fcb-4af4-a103-02ca29811bac" (UID: "a9fc1527-6fcb-4af4-a103-02ca29811bac"). InnerVolumeSpecName "kube-api-access-fdk2w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:41:04.265894 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:04.265875 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9fc1527-6fcb-4af4-a103-02ca29811bac-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a9fc1527-6fcb-4af4-a103-02ca29811bac" (UID: "a9fc1527-6fcb-4af4-a103-02ca29811bac"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 02:41:04.364817 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:04.364780 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdk2w\" (UniqueName: \"kubernetes.io/projected/a9fc1527-6fcb-4af4-a103-02ca29811bac-kube-api-access-fdk2w\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:41:04.364817 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:04.364815 2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9fc1527-6fcb-4af4-a103-02ca29811bac-home\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:41:04.364817 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:04.364826 2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9fc1527-6fcb-4af4-a103-02ca29811bac-tls-certs\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:41:04.365055 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:04.364834 2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9fc1527-6fcb-4af4-a103-02ca29811bac-dshm\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:41:04.491983 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:04.491946 2577 generic.go:356] "Generic (PLEG): container finished" podID="a9fc1527-6fcb-4af4-a103-02ca29811bac" containerID="0de0244bebabc5314520a0442d5faa0a16d9fe13dfe2d00f784533d5d6f2705e" exitCode=0 Jul 14 02:41:04.492156 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:04.491990 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-cc2dba/test-model-cc2dba-kserve-59f95744f9-5czmn" Jul 14 02:41:04.492156 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:04.492030 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-cc2dba/test-model-cc2dba-kserve-59f95744f9-5czmn" event={"ID":"a9fc1527-6fcb-4af4-a103-02ca29811bac","Type":"ContainerDied","Data":"0de0244bebabc5314520a0442d5faa0a16d9fe13dfe2d00f784533d5d6f2705e"} Jul 14 02:41:04.492156 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:04.492079 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-cc2dba/test-model-cc2dba-kserve-59f95744f9-5czmn" event={"ID":"a9fc1527-6fcb-4af4-a103-02ca29811bac","Type":"ContainerDied","Data":"f9662b5cf4eae22d955c39365a6b422c2c55bd506e3b464eb708eddd99ca34bf"} Jul 14 02:41:04.492156 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:04.492106 2577 scope.go:117] "RemoveContainer" containerID="0de0244bebabc5314520a0442d5faa0a16d9fe13dfe2d00f784533d5d6f2705e" Jul 14 02:41:04.500944 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:04.500924 2577 scope.go:117] "RemoveContainer" containerID="0de0244bebabc5314520a0442d5faa0a16d9fe13dfe2d00f784533d5d6f2705e" Jul 14 02:41:04.501214 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:41:04.501195 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0de0244bebabc5314520a0442d5faa0a16d9fe13dfe2d00f784533d5d6f2705e\": container with ID starting with 0de0244bebabc5314520a0442d5faa0a16d9fe13dfe2d00f784533d5d6f2705e not found: ID does not exist" containerID="0de0244bebabc5314520a0442d5faa0a16d9fe13dfe2d00f784533d5d6f2705e" Jul 14 02:41:04.501294 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:04.501223 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0de0244bebabc5314520a0442d5faa0a16d9fe13dfe2d00f784533d5d6f2705e"} err="failed to get container status \"0de0244bebabc5314520a0442d5faa0a16d9fe13dfe2d00f784533d5d6f2705e\": rpc error: code = NotFound desc = could not find container \"0de0244bebabc5314520a0442d5faa0a16d9fe13dfe2d00f784533d5d6f2705e\": container with ID starting with 0de0244bebabc5314520a0442d5faa0a16d9fe13dfe2d00f784533d5d6f2705e not found: ID does not exist" Jul 14 02:41:04.511375 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:04.511350 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-cc2dba/test-model-cc2dba-kserve-59f95744f9-5czmn"] Jul 14 02:41:04.518919 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:04.517791 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-cc2dba/test-model-cc2dba-kserve-59f95744f9-5czmn"] Jul 14 02:41:05.025265 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:05.025222 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:41:06.264937 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:06.264903 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9fc1527-6fcb-4af4-a103-02ca29811bac" path="/var/lib/kubelet/pods/a9fc1527-6fcb-4af4-a103-02ca29811bac/volumes" Jul 14 02:41:10.769503 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:10.769463 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:41:10.829165 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:10.829129 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:41:15.705512 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:15.705475 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq"] Jul 14 02:41:15.706011 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:15.705710 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" podUID="0b94870a-c919-4efb-85a4-55f08a095bff" containerName="istio-proxy" containerID="cri-o://3fa5b538ff7be5482cda4188be444bbcba44072373193d9e3443e67dd826b2e8" gracePeriod=30 Jul 14 02:41:15.721320 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:15.721261 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:41:15.953855 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:15.953818 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:41:16.616525 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:16.616497 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:41:16.718128 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:16.718032 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:41:20.472894 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:20.472856 2577 patch_prober.go:28] interesting pod/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 02:41:20.473342 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:20.472925 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" podUID="0b94870a-c919-4efb-85a4-55f08a095bff" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 02:41:20.972777 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:20.972755 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:41:21.032918 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.032848 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b94870a-c919-4efb-85a4-55f08a095bff-istio-token\") pod \"0b94870a-c919-4efb-85a4-55f08a095bff\" (UID: \"0b94870a-c919-4efb-85a4-55f08a095bff\") " Jul 14 02:41:21.032918 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.032879 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b94870a-c919-4efb-85a4-55f08a095bff-workload-socket\") pod \"0b94870a-c919-4efb-85a4-55f08a095bff\" (UID: \"0b94870a-c919-4efb-85a4-55f08a095bff\") " Jul 14 02:41:21.032918 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.032912 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b94870a-c919-4efb-85a4-55f08a095bff-istio-envoy\") pod \"0b94870a-c919-4efb-85a4-55f08a095bff\" (UID: \"0b94870a-c919-4efb-85a4-55f08a095bff\") " Jul 14 02:41:21.033149 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.032931 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b94870a-c919-4efb-85a4-55f08a095bff-istio-podinfo\") pod \"0b94870a-c919-4efb-85a4-55f08a095bff\" (UID: \"0b94870a-c919-4efb-85a4-55f08a095bff\") " Jul 14 02:41:21.033149 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.032952 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b94870a-c919-4efb-85a4-55f08a095bff-istio-data\") pod \"0b94870a-c919-4efb-85a4-55f08a095bff\" (UID: \"0b94870a-c919-4efb-85a4-55f08a095bff\") " Jul 14 02:41:21.033149 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.032970 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b94870a-c919-4efb-85a4-55f08a095bff-workload-certs\") pod \"0b94870a-c919-4efb-85a4-55f08a095bff\" (UID: \"0b94870a-c919-4efb-85a4-55f08a095bff\") " Jul 14 02:41:21.033149 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.032988 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b94870a-c919-4efb-85a4-55f08a095bff-credential-socket\") pod \"0b94870a-c919-4efb-85a4-55f08a095bff\" (UID: \"0b94870a-c919-4efb-85a4-55f08a095bff\") " Jul 14 02:41:21.033149 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.033013 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8nphb\" (UniqueName: \"kubernetes.io/projected/0b94870a-c919-4efb-85a4-55f08a095bff-kube-api-access-8nphb\") pod \"0b94870a-c919-4efb-85a4-55f08a095bff\" (UID: \"0b94870a-c919-4efb-85a4-55f08a095bff\") " Jul 14 02:41:21.033149 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.033050 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b94870a-c919-4efb-85a4-55f08a095bff-istiod-ca-cert\") pod \"0b94870a-c919-4efb-85a4-55f08a095bff\" (UID: \"0b94870a-c919-4efb-85a4-55f08a095bff\") " Jul 14 02:41:21.033508 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.033157 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b94870a-c919-4efb-85a4-55f08a095bff-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0b94870a-c919-4efb-85a4-55f08a095bff" (UID: "0b94870a-c919-4efb-85a4-55f08a095bff"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:41:21.033508 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.033423 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b94870a-c919-4efb-85a4-55f08a095bff-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0b94870a-c919-4efb-85a4-55f08a095bff" (UID: "0b94870a-c919-4efb-85a4-55f08a095bff"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:41:21.033612 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.033516 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b94870a-c919-4efb-85a4-55f08a095bff-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0b94870a-c919-4efb-85a4-55f08a095bff" (UID: "0b94870a-c919-4efb-85a4-55f08a095bff"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 02:41:21.033612 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.033552 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b94870a-c919-4efb-85a4-55f08a095bff-workload-socket\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:41:21.033612 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.033569 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b94870a-c919-4efb-85a4-55f08a095bff-workload-certs\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:41:21.033759 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.033603 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b94870a-c919-4efb-85a4-55f08a095bff-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0b94870a-c919-4efb-85a4-55f08a095bff" (UID: "0b94870a-c919-4efb-85a4-55f08a095bff"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:41:21.033985 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.033963 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b94870a-c919-4efb-85a4-55f08a095bff-istio-data" (OuterVolumeSpecName: "istio-data") pod "0b94870a-c919-4efb-85a4-55f08a095bff" (UID: "0b94870a-c919-4efb-85a4-55f08a095bff"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:41:21.035133 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.035108 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b94870a-c919-4efb-85a4-55f08a095bff-istio-token" (OuterVolumeSpecName: "istio-token") pod "0b94870a-c919-4efb-85a4-55f08a095bff" (UID: "0b94870a-c919-4efb-85a4-55f08a095bff"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:41:21.035344 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.035316 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0b94870a-c919-4efb-85a4-55f08a095bff-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0b94870a-c919-4efb-85a4-55f08a095bff" (UID: "0b94870a-c919-4efb-85a4-55f08a095bff"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 02:41:21.035524 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.035505 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b94870a-c919-4efb-85a4-55f08a095bff-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0b94870a-c919-4efb-85a4-55f08a095bff" (UID: "0b94870a-c919-4efb-85a4-55f08a095bff"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:41:21.035611 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.035591 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b94870a-c919-4efb-85a4-55f08a095bff-kube-api-access-8nphb" (OuterVolumeSpecName: "kube-api-access-8nphb") pod "0b94870a-c919-4efb-85a4-55f08a095bff" (UID: "0b94870a-c919-4efb-85a4-55f08a095bff"). InnerVolumeSpecName "kube-api-access-8nphb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:41:21.134499 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.134457 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b94870a-c919-4efb-85a4-55f08a095bff-istio-token\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:41:21.134499 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.134494 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b94870a-c919-4efb-85a4-55f08a095bff-istio-envoy\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:41:21.134499 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.134503 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b94870a-c919-4efb-85a4-55f08a095bff-istio-podinfo\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:41:21.134720 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.134512 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b94870a-c919-4efb-85a4-55f08a095bff-istio-data\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:41:21.134720 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.134520 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b94870a-c919-4efb-85a4-55f08a095bff-credential-socket\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:41:21.134720 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.134528 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8nphb\" (UniqueName: \"kubernetes.io/projected/0b94870a-c919-4efb-85a4-55f08a095bff-kube-api-access-8nphb\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:41:21.134720 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.134536 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b94870a-c919-4efb-85a4-55f08a095bff-istiod-ca-cert\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:41:21.575648 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.575613 2577 generic.go:356] "Generic (PLEG): container finished" podID="0b94870a-c919-4efb-85a4-55f08a095bff" containerID="3fa5b538ff7be5482cda4188be444bbcba44072373193d9e3443e67dd826b2e8" exitCode=0 Jul 14 02:41:21.576042 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.575661 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" Jul 14 02:41:21.576042 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.575696 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" event={"ID":"0b94870a-c919-4efb-85a4-55f08a095bff","Type":"ContainerDied","Data":"3fa5b538ff7be5482cda4188be444bbcba44072373193d9e3443e67dd826b2e8"} Jul 14 02:41:21.576042 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.575753 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq" event={"ID":"0b94870a-c919-4efb-85a4-55f08a095bff","Type":"ContainerDied","Data":"9f37e95c2d24536bb35185fa3bb855c9a8dfd2d0b5399c293c2a237197600e52"} Jul 14 02:41:21.576042 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.575777 2577 scope.go:117] "RemoveContainer" containerID="3fa5b538ff7be5482cda4188be444bbcba44072373193d9e3443e67dd826b2e8" Jul 14 02:41:21.584964 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.584949 2577 scope.go:117] "RemoveContainer" containerID="21c43107949da5e8f45365d70d692965b668e2efc8ca67ed364c29913da521b5" Jul 14 02:41:21.592560 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.592537 2577 scope.go:117] "RemoveContainer" containerID="3fa5b538ff7be5482cda4188be444bbcba44072373193d9e3443e67dd826b2e8" Jul 14 02:41:21.592860 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:41:21.592830 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fa5b538ff7be5482cda4188be444bbcba44072373193d9e3443e67dd826b2e8\": container with ID starting with 3fa5b538ff7be5482cda4188be444bbcba44072373193d9e3443e67dd826b2e8 not found: ID does not exist" containerID="3fa5b538ff7be5482cda4188be444bbcba44072373193d9e3443e67dd826b2e8" Jul 14 02:41:21.592975 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.592869 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fa5b538ff7be5482cda4188be444bbcba44072373193d9e3443e67dd826b2e8"} err="failed to get container status \"3fa5b538ff7be5482cda4188be444bbcba44072373193d9e3443e67dd826b2e8\": rpc error: code = NotFound desc = could not find container \"3fa5b538ff7be5482cda4188be444bbcba44072373193d9e3443e67dd826b2e8\": container with ID starting with 3fa5b538ff7be5482cda4188be444bbcba44072373193d9e3443e67dd826b2e8 not found: ID does not exist" Jul 14 02:41:21.592975 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.592893 2577 scope.go:117] "RemoveContainer" containerID="21c43107949da5e8f45365d70d692965b668e2efc8ca67ed364c29913da521b5" Jul 14 02:41:21.593607 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:41:21.593476 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21c43107949da5e8f45365d70d692965b668e2efc8ca67ed364c29913da521b5\": container with ID starting with 21c43107949da5e8f45365d70d692965b668e2efc8ca67ed364c29913da521b5 not found: ID does not exist" containerID="21c43107949da5e8f45365d70d692965b668e2efc8ca67ed364c29913da521b5" Jul 14 02:41:21.593607 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.593507 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21c43107949da5e8f45365d70d692965b668e2efc8ca67ed364c29913da521b5"} err="failed to get container status \"21c43107949da5e8f45365d70d692965b668e2efc8ca67ed364c29913da521b5\": rpc error: code = NotFound desc = could not find container \"21c43107949da5e8f45365d70d692965b668e2efc8ca67ed364c29913da521b5\": container with ID starting with 21c43107949da5e8f45365d70d692965b668e2efc8ca67ed364c29913da521b5 not found: ID does not exist" Jul 14 02:41:21.595802 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.595782 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq"] Jul 14 02:41:21.598597 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:21.598580 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-cc2dba-openshift-default-547ff7b55-gqmlq"] Jul 14 02:41:22.266628 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:22.266592 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b94870a-c919-4efb-85a4-55f08a095bff" path="/var/lib/kubelet/pods/0b94870a-c919-4efb-85a4-55f08a095bff/volumes" Jul 14 02:41:26.422661 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:26.422625 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:41:31.155259 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:31.155222 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:41:36.702601 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:36.702566 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:41:36.770773 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:36.770737 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:41:41.799213 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:41.799174 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b"] Jul 14 02:41:41.799703 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:41.799440 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" podUID="7bba6bbd-d81f-4872-a8e6-2168e0c9079f" containerName="istio-proxy" containerID="cri-o://4df6abc04a838272d30454bd3a486ee35e717d9139ea8abb7c639d4a8146efb0" gracePeriod=30 Jul 14 02:41:41.819474 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:41.819444 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:41:42.049064 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.049033 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:41:42.379136 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.379097 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:41:42.657389 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.657302 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4"] Jul 14 02:41:42.657691 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.657670 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b94870a-c919-4efb-85a4-55f08a095bff" containerName="istio-proxy" Jul 14 02:41:42.657691 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.657684 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b94870a-c919-4efb-85a4-55f08a095bff" containerName="istio-proxy" Jul 14 02:41:42.657789 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.657703 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9fc1527-6fcb-4af4-a103-02ca29811bac" containerName="main" Jul 14 02:41:42.657789 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.657708 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9fc1527-6fcb-4af4-a103-02ca29811bac" containerName="main" Jul 14 02:41:42.657789 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.657720 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b94870a-c919-4efb-85a4-55f08a095bff" containerName="istio-proxy" Jul 14 02:41:42.657789 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.657726 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b94870a-c919-4efb-85a4-55f08a095bff" containerName="istio-proxy" Jul 14 02:41:42.657789 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.657784 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b94870a-c919-4efb-85a4-55f08a095bff" containerName="istio-proxy" Jul 14 02:41:42.657789 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.657791 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b94870a-c919-4efb-85a4-55f08a095bff" containerName="istio-proxy" Jul 14 02:41:42.657966 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.657796 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9fc1527-6fcb-4af4-a103-02ca29811bac" containerName="main" Jul 14 02:41:42.662406 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.662386 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:41:42.664341 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.664321 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-9794b6-openshift-default-dockercfg-k6lqh\"" Jul 14 02:41:42.671070 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.670633 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4"] Jul 14 02:41:42.685223 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.685189 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:41:42.744567 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.744512 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/befee5ec-3cd2-416c-9d76-a02adfceceb6-istio-token\") pod \"e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4\" (UID: \"befee5ec-3cd2-416c-9d76-a02adfceceb6\") " pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:41:42.744755 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.744580 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf2c8\" (UniqueName: \"kubernetes.io/projected/befee5ec-3cd2-416c-9d76-a02adfceceb6-kube-api-access-gf2c8\") pod \"e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4\" (UID: \"befee5ec-3cd2-416c-9d76-a02adfceceb6\") " pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:41:42.744755 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.744613 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/befee5ec-3cd2-416c-9d76-a02adfceceb6-workload-socket\") pod \"e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4\" (UID: \"befee5ec-3cd2-416c-9d76-a02adfceceb6\") " pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:41:42.744755 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.744664 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/befee5ec-3cd2-416c-9d76-a02adfceceb6-istio-data\") pod \"e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4\" (UID: \"befee5ec-3cd2-416c-9d76-a02adfceceb6\") " pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:41:42.744755 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.744702 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/befee5ec-3cd2-416c-9d76-a02adfceceb6-credential-socket\") pod \"e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4\" (UID: \"befee5ec-3cd2-416c-9d76-a02adfceceb6\") " pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:41:42.744755 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.744731 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/befee5ec-3cd2-416c-9d76-a02adfceceb6-istio-podinfo\") pod \"e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4\" (UID: \"befee5ec-3cd2-416c-9d76-a02adfceceb6\") " pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:41:42.745353 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.744794 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/befee5ec-3cd2-416c-9d76-a02adfceceb6-istio-envoy\") pod \"e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4\" (UID: \"befee5ec-3cd2-416c-9d76-a02adfceceb6\") " pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:41:42.745484 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.745400 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/befee5ec-3cd2-416c-9d76-a02adfceceb6-workload-certs\") pod \"e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4\" (UID: \"befee5ec-3cd2-416c-9d76-a02adfceceb6\") " pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:41:42.745484 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.745432 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/befee5ec-3cd2-416c-9d76-a02adfceceb6-istiod-ca-cert\") pod \"e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4\" (UID: \"befee5ec-3cd2-416c-9d76-a02adfceceb6\") " pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:41:42.846084 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.846054 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/befee5ec-3cd2-416c-9d76-a02adfceceb6-istio-token\") pod \"e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4\" (UID: \"befee5ec-3cd2-416c-9d76-a02adfceceb6\") " pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:41:42.846588 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.846094 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf2c8\" (UniqueName: \"kubernetes.io/projected/befee5ec-3cd2-416c-9d76-a02adfceceb6-kube-api-access-gf2c8\") pod \"e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4\" (UID: \"befee5ec-3cd2-416c-9d76-a02adfceceb6\") " pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:41:42.846588 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.846118 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/befee5ec-3cd2-416c-9d76-a02adfceceb6-workload-socket\") pod \"e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4\" (UID: \"befee5ec-3cd2-416c-9d76-a02adfceceb6\") " pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:41:42.846588 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.846156 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/befee5ec-3cd2-416c-9d76-a02adfceceb6-istio-data\") pod \"e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4\" (UID: \"befee5ec-3cd2-416c-9d76-a02adfceceb6\") " pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:41:42.846588 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.846182 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/befee5ec-3cd2-416c-9d76-a02adfceceb6-credential-socket\") pod \"e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4\" (UID: \"befee5ec-3cd2-416c-9d76-a02adfceceb6\") " pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:41:42.846588 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.846202 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/befee5ec-3cd2-416c-9d76-a02adfceceb6-istio-podinfo\") pod \"e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4\" (UID: \"befee5ec-3cd2-416c-9d76-a02adfceceb6\") " pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:41:42.846588 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.846254 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/befee5ec-3cd2-416c-9d76-a02adfceceb6-istio-envoy\") pod \"e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4\" (UID: \"befee5ec-3cd2-416c-9d76-a02adfceceb6\") " pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:41:42.846588 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.846312 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/befee5ec-3cd2-416c-9d76-a02adfceceb6-workload-certs\") pod \"e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4\" (UID: \"befee5ec-3cd2-416c-9d76-a02adfceceb6\") " pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:41:42.846588 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.846342 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/befee5ec-3cd2-416c-9d76-a02adfceceb6-istiod-ca-cert\") pod \"e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4\" (UID: \"befee5ec-3cd2-416c-9d76-a02adfceceb6\") " pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:41:42.846988 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.846710 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/befee5ec-3cd2-416c-9d76-a02adfceceb6-workload-socket\") pod \"e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4\" (UID: \"befee5ec-3cd2-416c-9d76-a02adfceceb6\") " pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:41:42.846988 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.846806 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/befee5ec-3cd2-416c-9d76-a02adfceceb6-credential-socket\") pod \"e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4\" (UID: \"befee5ec-3cd2-416c-9d76-a02adfceceb6\") " pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:41:42.846988 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.846810 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/befee5ec-3cd2-416c-9d76-a02adfceceb6-istio-data\") pod \"e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4\" (UID: \"befee5ec-3cd2-416c-9d76-a02adfceceb6\") " pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:41:42.846988 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.846937 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/befee5ec-3cd2-416c-9d76-a02adfceceb6-workload-certs\") pod \"e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4\" (UID: \"befee5ec-3cd2-416c-9d76-a02adfceceb6\") " pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:41:42.847133 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.847069 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/befee5ec-3cd2-416c-9d76-a02adfceceb6-istiod-ca-cert\") pod \"e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4\" (UID: \"befee5ec-3cd2-416c-9d76-a02adfceceb6\") " pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:41:42.848585 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.848564 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/befee5ec-3cd2-416c-9d76-a02adfceceb6-istio-envoy\") pod \"e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4\" (UID: \"befee5ec-3cd2-416c-9d76-a02adfceceb6\") " pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:41:42.849022 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.849000 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/befee5ec-3cd2-416c-9d76-a02adfceceb6-istio-podinfo\") pod \"e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4\" (UID: \"befee5ec-3cd2-416c-9d76-a02adfceceb6\") " pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:41:42.855020 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.854997 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf2c8\" (UniqueName: \"kubernetes.io/projected/befee5ec-3cd2-416c-9d76-a02adfceceb6-kube-api-access-gf2c8\") pod \"e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4\" (UID: \"befee5ec-3cd2-416c-9d76-a02adfceceb6\") " pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:41:42.855196 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.855021 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/befee5ec-3cd2-416c-9d76-a02adfceceb6-istio-token\") pod \"e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4\" (UID: \"befee5ec-3cd2-416c-9d76-a02adfceceb6\") " pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:41:42.892655 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.892625 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:41:42.986068 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:42.985976 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:41:43.122615 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:43.122590 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4"] Jul 14 02:41:43.124170 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:41:43.124140 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbefee5ec_3cd2_416c_9d76_a02adfceceb6.slice/crio-2d56382e9c34f22d7495a5849d499dec214f843a904e167826a044c2cb0a0d5d WatchSource:0}: Error finding container 2d56382e9c34f22d7495a5849d499dec214f843a904e167826a044c2cb0a0d5d: Status 404 returned error can't find the container with id 2d56382e9c34f22d7495a5849d499dec214f843a904e167826a044c2cb0a0d5d Jul 14 02:41:43.126333 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:43.126265 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:41:43.126435 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:43.126386 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:41:43.126476 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:43.126430 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:41:43.429990 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:43.429954 2577 patch_prober.go:28] interesting pod/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 02:41:43.430188 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:43.430015 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" podUID="7bba6bbd-d81f-4872-a8e6-2168e0c9079f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 02:41:43.691479 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:43.691385 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" event={"ID":"befee5ec-3cd2-416c-9d76-a02adfceceb6","Type":"ContainerStarted","Data":"a3f430b7547fe7ccad750c0e35677782fe24f4c7a8b0ec5ccf143920ebdac4bb"} Jul 14 02:41:43.691479 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:43.691436 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" event={"ID":"befee5ec-3cd2-416c-9d76-a02adfceceb6","Type":"ContainerStarted","Data":"2d56382e9c34f22d7495a5849d499dec214f843a904e167826a044c2cb0a0d5d"} Jul 14 02:41:43.709481 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:43.709417 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" podStartSLOduration=1.7093961640000002 podStartE2EDuration="1.709396164s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:41:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:41:43.707978922 +0000 UTC m=+2405.862815571" watchObservedRunningTime="2026-07-14 02:41:43.709396164 +0000 UTC m=+2405.864232813" Jul 14 02:41:43.986513 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:43.986419 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:41:44.991362 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:44.991331 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:41:45.585188 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:45.585148 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:41:45.701130 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:45.701104 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:41:45.701993 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:45.701975 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:41:45.703483 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:45.703462 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:41:45.833909 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:45.833876 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:41:46.015142 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:46.015106 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:41:46.193390 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:46.193344 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:41:46.293939 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:46.293859 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:41:47.064139 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.064078 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:41:47.188090 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.188054 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-istio-data\") pod \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\" (UID: \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\") " Jul 14 02:41:47.188310 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.188109 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-workload-certs\") pod \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\" (UID: \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\") " Jul 14 02:41:47.188310 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.188134 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g9kpv\" (UniqueName: \"kubernetes.io/projected/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-kube-api-access-g9kpv\") pod \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\" (UID: \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\") " Jul 14 02:41:47.188310 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.188157 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-istio-token\") pod \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\" (UID: \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\") " Jul 14 02:41:47.188310 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.188178 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-workload-socket\") pod \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\" (UID: \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\") " Jul 14 02:41:47.188310 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.188207 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-istiod-ca-cert\") pod \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\" (UID: \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\") " Jul 14 02:41:47.188589 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.188370 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-credential-socket\") pod \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\" (UID: \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\") " Jul 14 02:41:47.188589 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.188451 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-istio-podinfo\") pod \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\" (UID: \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\") " Jul 14 02:41:47.188589 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.188517 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7bba6bbd-d81f-4872-a8e6-2168e0c9079f" (UID: "7bba6bbd-d81f-4872-a8e6-2168e0c9079f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:41:47.188589 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.188529 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7bba6bbd-d81f-4872-a8e6-2168e0c9079f" (UID: "7bba6bbd-d81f-4872-a8e6-2168e0c9079f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:41:47.188589 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.188528 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-istio-envoy\") pod \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\" (UID: \"7bba6bbd-d81f-4872-a8e6-2168e0c9079f\") " Jul 14 02:41:47.188821 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.188648 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7bba6bbd-d81f-4872-a8e6-2168e0c9079f" (UID: "7bba6bbd-d81f-4872-a8e6-2168e0c9079f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:41:47.188821 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.188686 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7bba6bbd-d81f-4872-a8e6-2168e0c9079f" (UID: "7bba6bbd-d81f-4872-a8e6-2168e0c9079f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 02:41:47.188969 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.188951 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-workload-certs\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:41:47.189026 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.188971 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-istio-data" (OuterVolumeSpecName: "istio-data") pod "7bba6bbd-d81f-4872-a8e6-2168e0c9079f" (UID: "7bba6bbd-d81f-4872-a8e6-2168e0c9079f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:41:47.189026 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.188976 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-workload-socket\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:41:47.189026 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.189012 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-istiod-ca-cert\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:41:47.189026 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.189024 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-credential-socket\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:41:47.190461 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.190432 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-kube-api-access-g9kpv" (OuterVolumeSpecName: "kube-api-access-g9kpv") pod "7bba6bbd-d81f-4872-a8e6-2168e0c9079f" (UID: "7bba6bbd-d81f-4872-a8e6-2168e0c9079f"). InnerVolumeSpecName "kube-api-access-g9kpv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:41:47.190683 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.190665 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-istio-token" (OuterVolumeSpecName: "istio-token") pod "7bba6bbd-d81f-4872-a8e6-2168e0c9079f" (UID: "7bba6bbd-d81f-4872-a8e6-2168e0c9079f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:41:47.190958 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.190941 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7bba6bbd-d81f-4872-a8e6-2168e0c9079f" (UID: "7bba6bbd-d81f-4872-a8e6-2168e0c9079f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:41:47.191036 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.190964 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7bba6bbd-d81f-4872-a8e6-2168e0c9079f" (UID: "7bba6bbd-d81f-4872-a8e6-2168e0c9079f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 02:41:47.289798 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.289764 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-istio-data\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:41:47.289798 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.289791 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9kpv\" (UniqueName: \"kubernetes.io/projected/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-kube-api-access-g9kpv\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:41:47.289798 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.289801 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-istio-token\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:41:47.289798 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.289809 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-istio-podinfo\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:41:47.290077 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.289817 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bba6bbd-d81f-4872-a8e6-2168e0c9079f-istio-envoy\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:41:47.712292 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.712241 2577 generic.go:356] "Generic (PLEG): container finished" podID="7bba6bbd-d81f-4872-a8e6-2168e0c9079f" containerID="4df6abc04a838272d30454bd3a486ee35e717d9139ea8abb7c639d4a8146efb0" exitCode=0 Jul 14 02:41:47.712474 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.712310 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" Jul 14 02:41:47.712474 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.712326 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" event={"ID":"7bba6bbd-d81f-4872-a8e6-2168e0c9079f","Type":"ContainerDied","Data":"4df6abc04a838272d30454bd3a486ee35e717d9139ea8abb7c639d4a8146efb0"} Jul 14 02:41:47.712474 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.712380 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b" event={"ID":"7bba6bbd-d81f-4872-a8e6-2168e0c9079f","Type":"ContainerDied","Data":"ac02697a2d2209fb96048365bb4662d5262c8084eefae5210648c76254fb5e20"} Jul 14 02:41:47.712474 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.712401 2577 scope.go:117] "RemoveContainer" containerID="4df6abc04a838272d30454bd3a486ee35e717d9139ea8abb7c639d4a8146efb0" Jul 14 02:41:47.721728 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.721711 2577 scope.go:117] "RemoveContainer" containerID="9b4ce6927eaf9ba540f833d42c4302aac753d9655c5601681db851c7c682064f" Jul 14 02:41:47.729418 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.729398 2577 scope.go:117] "RemoveContainer" containerID="4df6abc04a838272d30454bd3a486ee35e717d9139ea8abb7c639d4a8146efb0" Jul 14 02:41:47.729692 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:41:47.729666 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4df6abc04a838272d30454bd3a486ee35e717d9139ea8abb7c639d4a8146efb0\": container with ID starting with 4df6abc04a838272d30454bd3a486ee35e717d9139ea8abb7c639d4a8146efb0 not found: ID does not exist" containerID="4df6abc04a838272d30454bd3a486ee35e717d9139ea8abb7c639d4a8146efb0" Jul 14 02:41:47.729822 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.729703 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4df6abc04a838272d30454bd3a486ee35e717d9139ea8abb7c639d4a8146efb0"} err="failed to get container status \"4df6abc04a838272d30454bd3a486ee35e717d9139ea8abb7c639d4a8146efb0\": rpc error: code = NotFound desc = could not find container \"4df6abc04a838272d30454bd3a486ee35e717d9139ea8abb7c639d4a8146efb0\": container with ID starting with 4df6abc04a838272d30454bd3a486ee35e717d9139ea8abb7c639d4a8146efb0 not found: ID does not exist" Jul 14 02:41:47.729822 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.729727 2577 scope.go:117] "RemoveContainer" containerID="9b4ce6927eaf9ba540f833d42c4302aac753d9655c5601681db851c7c682064f" Jul 14 02:41:47.730152 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:41:47.730125 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b4ce6927eaf9ba540f833d42c4302aac753d9655c5601681db851c7c682064f\": container with ID starting with 9b4ce6927eaf9ba540f833d42c4302aac753d9655c5601681db851c7c682064f not found: ID does not exist" containerID="9b4ce6927eaf9ba540f833d42c4302aac753d9655c5601681db851c7c682064f" Jul 14 02:41:47.730243 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.730160 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b4ce6927eaf9ba540f833d42c4302aac753d9655c5601681db851c7c682064f"} err="failed to get container status \"9b4ce6927eaf9ba540f833d42c4302aac753d9655c5601681db851c7c682064f\": rpc error: code = NotFound desc = could not find container \"9b4ce6927eaf9ba540f833d42c4302aac753d9655c5601681db851c7c682064f\": container with ID starting with 9b4ce6927eaf9ba540f833d42c4302aac753d9655c5601681db851c7c682064f not found: ID does not exist" Jul 14 02:41:47.732355 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.732335 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b"] Jul 14 02:41:47.735959 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:47.735936 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-7924bc-openshift-default-58c69d59bf-7h59b"] Jul 14 02:41:48.265592 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:48.265559 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bba6bbd-d81f-4872-a8e6-2168e0c9079f" path="/var/lib/kubelet/pods/7bba6bbd-d81f-4872-a8e6-2168e0c9079f/volumes" Jul 14 02:41:50.087054 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:50.087018 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:41:50.554855 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:50.554814 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:41:50.807056 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:50.806968 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq"] Jul 14 02:41:50.807359 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:50.807346 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bba6bbd-d81f-4872-a8e6-2168e0c9079f" containerName="istio-proxy" Jul 14 02:41:50.807426 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:50.807360 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bba6bbd-d81f-4872-a8e6-2168e0c9079f" containerName="istio-proxy" Jul 14 02:41:50.807426 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:50.807369 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bba6bbd-d81f-4872-a8e6-2168e0c9079f" containerName="istio-proxy" Jul 14 02:41:50.807426 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:50.807375 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bba6bbd-d81f-4872-a8e6-2168e0c9079f" containerName="istio-proxy" Jul 14 02:41:50.807533 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:50.807442 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bba6bbd-d81f-4872-a8e6-2168e0c9079f" containerName="istio-proxy" Jul 14 02:41:50.810447 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:50.810428 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:41:50.812465 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:50.812446 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-f83e72-openshift-default-dockercfg-lqsf9\"" Jul 14 02:41:50.820287 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:50.820252 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq"] Jul 14 02:41:50.837426 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:50.837393 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:41:50.925133 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:50.925103 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpj6t\" (UniqueName: \"kubernetes.io/projected/ea69d5a7-16a9-4661-9aac-c2288e7a867b-kube-api-access-tpj6t\") pod \"e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq\" (UID: \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\") " pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:41:50.925389 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:50.925148 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea69d5a7-16a9-4661-9aac-c2288e7a867b-credential-socket\") pod \"e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq\" (UID: \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\") " pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:41:50.925389 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:50.925233 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea69d5a7-16a9-4661-9aac-c2288e7a867b-workload-certs\") pod \"e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq\" (UID: \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\") " pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:41:50.925389 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:50.925341 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea69d5a7-16a9-4661-9aac-c2288e7a867b-istio-data\") pod \"e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq\" (UID: \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\") " pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:41:50.925389 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:50.925376 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea69d5a7-16a9-4661-9aac-c2288e7a867b-istiod-ca-cert\") pod \"e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq\" (UID: \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\") " pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:41:50.925629 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:50.925415 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea69d5a7-16a9-4661-9aac-c2288e7a867b-workload-socket\") pod \"e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq\" (UID: \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\") " pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:41:50.925629 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:50.925439 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea69d5a7-16a9-4661-9aac-c2288e7a867b-istio-token\") pod \"e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq\" (UID: \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\") " pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:41:50.925629 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:50.925469 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea69d5a7-16a9-4661-9aac-c2288e7a867b-istio-envoy\") pod \"e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq\" (UID: \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\") " pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:41:50.925629 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:50.925551 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea69d5a7-16a9-4661-9aac-c2288e7a867b-istio-podinfo\") pod \"e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq\" (UID: \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\") " pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:41:51.026540 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:51.026510 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpj6t\" (UniqueName: \"kubernetes.io/projected/ea69d5a7-16a9-4661-9aac-c2288e7a867b-kube-api-access-tpj6t\") pod \"e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq\" (UID: \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\") " pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:41:51.026712 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:51.026552 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea69d5a7-16a9-4661-9aac-c2288e7a867b-credential-socket\") pod \"e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq\" (UID: \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\") " pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:41:51.026712 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:51.026612 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea69d5a7-16a9-4661-9aac-c2288e7a867b-workload-certs\") pod \"e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq\" (UID: \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\") " pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:41:51.026712 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:51.026643 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea69d5a7-16a9-4661-9aac-c2288e7a867b-istio-data\") pod \"e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq\" (UID: \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\") " pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:41:51.026712 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:51.026668 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea69d5a7-16a9-4661-9aac-c2288e7a867b-istiod-ca-cert\") pod \"e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq\" (UID: \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\") " pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:41:51.026890 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:51.026726 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea69d5a7-16a9-4661-9aac-c2288e7a867b-workload-socket\") pod \"e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq\" (UID: \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\") " pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:41:51.026890 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:51.026762 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea69d5a7-16a9-4661-9aac-c2288e7a867b-istio-token\") pod \"e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq\" (UID: \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\") " pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:41:51.026890 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:51.026795 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea69d5a7-16a9-4661-9aac-c2288e7a867b-istio-envoy\") pod \"e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq\" (UID: \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\") " pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:41:51.026890 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:51.026850 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea69d5a7-16a9-4661-9aac-c2288e7a867b-istio-podinfo\") pod \"e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq\" (UID: \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\") " pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:41:51.027094 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:51.027045 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea69d5a7-16a9-4661-9aac-c2288e7a867b-workload-certs\") pod \"e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq\" (UID: \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\") " pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:41:51.027094 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:51.027066 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea69d5a7-16a9-4661-9aac-c2288e7a867b-workload-socket\") pod \"e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq\" (UID: \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\") " pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:41:51.027193 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:51.027124 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea69d5a7-16a9-4661-9aac-c2288e7a867b-credential-socket\") pod \"e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq\" (UID: \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\") " pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:41:51.027252 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:51.027236 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea69d5a7-16a9-4661-9aac-c2288e7a867b-istio-data\") pod \"e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq\" (UID: \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\") " pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:41:51.027513 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:51.027490 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea69d5a7-16a9-4661-9aac-c2288e7a867b-istiod-ca-cert\") pod \"e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq\" (UID: \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\") " pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:41:51.029081 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:51.029060 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea69d5a7-16a9-4661-9aac-c2288e7a867b-istio-envoy\") pod \"e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq\" (UID: \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\") " pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:41:51.029373 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:51.029357 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea69d5a7-16a9-4661-9aac-c2288e7a867b-istio-podinfo\") pod \"e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq\" (UID: \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\") " pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:41:51.033937 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:51.033902 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpj6t\" (UniqueName: \"kubernetes.io/projected/ea69d5a7-16a9-4661-9aac-c2288e7a867b-kube-api-access-tpj6t\") pod \"e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq\" (UID: \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\") " pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:41:51.037108 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:51.037082 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea69d5a7-16a9-4661-9aac-c2288e7a867b-istio-token\") pod \"e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq\" (UID: \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\") " pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:41:51.121090 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:51.121065 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:41:51.252296 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:51.252255 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq"] Jul 14 02:41:51.253973 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:41:51.253945 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea69d5a7_16a9_4661_9aac_c2288e7a867b.slice/crio-c38af05ec4fb598185443691e7e79d619887490861835e567110ebbe5715412b WatchSource:0}: Error finding container c38af05ec4fb598185443691e7e79d619887490861835e567110ebbe5715412b: Status 404 returned error can't find the container with id c38af05ec4fb598185443691e7e79d619887490861835e567110ebbe5715412b Jul 14 02:41:51.256192 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:51.256159 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:41:51.256263 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:51.256224 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:41:51.256319 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:51.256252 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:41:51.295636 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:51.295613 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:41:51.734785 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:51.734748 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" event={"ID":"ea69d5a7-16a9-4661-9aac-c2288e7a867b","Type":"ContainerStarted","Data":"3601419e9f5d46de2a4af0f3e45f2084082db3758561bdc81d1cd8a6072e5321"} Jul 14 02:41:51.734985 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:51.734851 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" event={"ID":"ea69d5a7-16a9-4661-9aac-c2288e7a867b","Type":"ContainerStarted","Data":"c38af05ec4fb598185443691e7e79d619887490861835e567110ebbe5715412b"} Jul 14 02:41:51.754929 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:51.754875 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" podStartSLOduration=1.7548603379999999 podStartE2EDuration="1.754860338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:41:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:41:51.753877398 +0000 UTC m=+2413.908714061" watchObservedRunningTime="2026-07-14 02:41:51.754860338 +0000 UTC m=+2413.909697050" Jul 14 02:41:51.950511 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:51.950428 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:41:52.121849 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:52.121820 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:41:52.126789 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:52.126764 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:41:52.751890 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:52.751853 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:41:52.760294 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:52.759491 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:41:53.648779 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:53.648744 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:41:53.725043 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:53.725004 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:41:53.788835 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:53.788804 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:41:54.052315 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:54.052157 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:41:54.175890 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:54.175856 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:41:54.273537 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:54.273506 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:41:58.144557 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:58.144521 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:41:58.796032 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:58.796002 2577 generic.go:356] "Generic (PLEG): container finished" podID="befee5ec-3cd2-416c-9d76-a02adfceceb6" containerID="a3f430b7547fe7ccad750c0e35677782fe24f4c7a8b0ec5ccf143920ebdac4bb" exitCode=0 Jul 14 02:41:58.796203 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:58.796070 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" event={"ID":"befee5ec-3cd2-416c-9d76-a02adfceceb6","Type":"ContainerDied","Data":"a3f430b7547fe7ccad750c0e35677782fe24f4c7a8b0ec5ccf143920ebdac4bb"} Jul 14 02:41:58.796554 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:58.796534 2577 scope.go:117] "RemoveContainer" containerID="a3f430b7547fe7ccad750c0e35677782fe24f4c7a8b0ec5ccf143920ebdac4bb" Jul 14 02:41:58.797724 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:58.797694 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:41:58.797805 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:58.797781 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:41:58.797846 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:58.797829 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:41:59.803382 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:59.803290 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" event={"ID":"befee5ec-3cd2-416c-9d76-a02adfceceb6","Type":"ContainerStarted","Data":"3dea8f6a39a0bdcb47c3ab594376f85b45747188f8e1d4e54c4953bb0e898090"} Jul 14 02:41:59.986933 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:59.986893 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:41:59.988080 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:59.988053 2577 patch_prober.go:28] interesting pod/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 14 02:41:59.988309 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:41:59.988265 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" podUID="befee5ec-3cd2-416c-9d76-a02adfceceb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 14 02:42:00.701948 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:00.701907 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:42:00.987936 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:00.987846 2577 patch_prober.go:28] interesting pod/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 14 02:42:00.988515 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:00.988486 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" podUID="befee5ec-3cd2-416c-9d76-a02adfceceb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 14 02:42:01.987013 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:01.986969 2577 patch_prober.go:28] interesting pod/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 14 02:42:01.987231 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:01.987031 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" podUID="befee5ec-3cd2-416c-9d76-a02adfceceb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 14 02:42:02.398762 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:02.398733 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:42:02.498728 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:02.498700 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:42:02.987793 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:02.987754 2577 patch_prober.go:28] interesting pod/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 14 02:42:02.988011 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:02.987823 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" podUID="befee5ec-3cd2-416c-9d76-a02adfceceb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 14 02:42:03.991050 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:03.991015 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:42:03.992210 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:03.992188 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:42:06.540252 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:06.540074 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:42:06.640920 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:06.640881 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:42:06.843417 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:06.843328 2577 generic.go:356] "Generic (PLEG): container finished" podID="ea69d5a7-16a9-4661-9aac-c2288e7a867b" containerID="3601419e9f5d46de2a4af0f3e45f2084082db3758561bdc81d1cd8a6072e5321" exitCode=0 Jul 14 02:42:06.843417 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:06.843387 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" event={"ID":"ea69d5a7-16a9-4661-9aac-c2288e7a867b","Type":"ContainerDied","Data":"3601419e9f5d46de2a4af0f3e45f2084082db3758561bdc81d1cd8a6072e5321"} Jul 14 02:42:06.843820 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:06.843804 2577 scope.go:117] "RemoveContainer" containerID="3601419e9f5d46de2a4af0f3e45f2084082db3758561bdc81d1cd8a6072e5321" Jul 14 02:42:06.844878 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:06.844852 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:42:06.844960 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:06.844942 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:42:06.845000 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:06.844986 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:42:07.752177 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:07.752136 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:42:07.853741 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:07.853702 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" event={"ID":"ea69d5a7-16a9-4661-9aac-c2288e7a867b","Type":"ContainerStarted","Data":"ace4ee61869edf61288db96d20d638a3711176e5e62a6661a67c70b4ff688a7b"} Jul 14 02:42:08.121570 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:08.121536 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:42:08.122750 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:08.122725 2577 patch_prober.go:28] interesting pod/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 14 02:42:08.122887 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:08.122790 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" podUID="ea69d5a7-16a9-4661-9aac-c2288e7a867b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 14 02:42:09.121779 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:09.121746 2577 patch_prober.go:28] interesting pod/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 14 02:42:09.122152 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:09.121814 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" podUID="ea69d5a7-16a9-4661-9aac-c2288e7a867b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 14 02:42:10.122040 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:10.122005 2577 patch_prober.go:28] interesting pod/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 14 02:42:10.122492 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:10.122061 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" podUID="ea69d5a7-16a9-4661-9aac-c2288e7a867b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 14 02:42:11.121922 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:11.121890 2577 patch_prober.go:28] interesting pod/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 14 02:42:11.122100 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:11.121941 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" podUID="ea69d5a7-16a9-4661-9aac-c2288e7a867b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 14 02:42:12.124953 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:12.124925 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:42:12.125373 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:12.125186 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:42:12.125937 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:12.125920 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:42:14.840547 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:14.840505 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:42:14.937186 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:14.937143 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:42:16.592310 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:16.592257 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:42:16.693680 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:16.693648 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:42:16.827103 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:16.827073 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:42:21.178243 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:21.178209 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:42:26.828086 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:26.828049 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:42:26.929380 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:26.929344 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:42:31.897109 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:31.897077 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4"] Jul 14 02:42:31.897609 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:31.897342 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" podUID="befee5ec-3cd2-416c-9d76-a02adfceceb6" containerName="istio-proxy" containerID="cri-o://3dea8f6a39a0bdcb47c3ab594376f85b45747188f8e1d4e54c4953bb0e898090" gracePeriod=30 Jul 14 02:42:31.910988 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:31.910959 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:42:32.137626 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:32.137595 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:42:32.702697 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:32.702665 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:42:32.804427 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:32.804392 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:42:33.994314 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:33.994264 2577 patch_prober.go:28] interesting pod/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 02:42:33.994696 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:33.994351 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" podUID="befee5ec-3cd2-416c-9d76-a02adfceceb6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 02:42:37.157678 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:37.157612 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:42:37.227173 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:37.227138 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/befee5ec-3cd2-416c-9d76-a02adfceceb6-istio-token\") pod \"befee5ec-3cd2-416c-9d76-a02adfceceb6\" (UID: \"befee5ec-3cd2-416c-9d76-a02adfceceb6\") " Jul 14 02:42:37.227397 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:37.227194 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/befee5ec-3cd2-416c-9d76-a02adfceceb6-istio-data\") pod \"befee5ec-3cd2-416c-9d76-a02adfceceb6\" (UID: \"befee5ec-3cd2-416c-9d76-a02adfceceb6\") " Jul 14 02:42:37.227397 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:37.227330 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/befee5ec-3cd2-416c-9d76-a02adfceceb6-credential-socket\") pod \"befee5ec-3cd2-416c-9d76-a02adfceceb6\" (UID: \"befee5ec-3cd2-416c-9d76-a02adfceceb6\") " Jul 14 02:42:37.227397 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:37.227377 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/befee5ec-3cd2-416c-9d76-a02adfceceb6-workload-socket\") pod \"befee5ec-3cd2-416c-9d76-a02adfceceb6\" (UID: \"befee5ec-3cd2-416c-9d76-a02adfceceb6\") " Jul 14 02:42:37.227579 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:37.227415 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/befee5ec-3cd2-416c-9d76-a02adfceceb6-istio-podinfo\") pod \"befee5ec-3cd2-416c-9d76-a02adfceceb6\" (UID: \"befee5ec-3cd2-416c-9d76-a02adfceceb6\") " Jul 14 02:42:37.227579 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:37.227438 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/befee5ec-3cd2-416c-9d76-a02adfceceb6-workload-certs\") pod \"befee5ec-3cd2-416c-9d76-a02adfceceb6\" (UID: \"befee5ec-3cd2-416c-9d76-a02adfceceb6\") " Jul 14 02:42:37.227579 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:37.227471 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/befee5ec-3cd2-416c-9d76-a02adfceceb6-istiod-ca-cert\") pod \"befee5ec-3cd2-416c-9d76-a02adfceceb6\" (UID: \"befee5ec-3cd2-416c-9d76-a02adfceceb6\") " Jul 14 02:42:37.227579 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:37.227566 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/befee5ec-3cd2-416c-9d76-a02adfceceb6-istio-envoy\") pod \"befee5ec-3cd2-416c-9d76-a02adfceceb6\" (UID: \"befee5ec-3cd2-416c-9d76-a02adfceceb6\") " Jul 14 02:42:37.227774 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:37.227594 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gf2c8\" (UniqueName: \"kubernetes.io/projected/befee5ec-3cd2-416c-9d76-a02adfceceb6-kube-api-access-gf2c8\") pod \"befee5ec-3cd2-416c-9d76-a02adfceceb6\" (UID: \"befee5ec-3cd2-416c-9d76-a02adfceceb6\") " Jul 14 02:42:37.227774 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:37.227612 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/befee5ec-3cd2-416c-9d76-a02adfceceb6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "befee5ec-3cd2-416c-9d76-a02adfceceb6" (UID: "befee5ec-3cd2-416c-9d76-a02adfceceb6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:42:37.227774 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:37.227636 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/befee5ec-3cd2-416c-9d76-a02adfceceb6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "befee5ec-3cd2-416c-9d76-a02adfceceb6" (UID: "befee5ec-3cd2-416c-9d76-a02adfceceb6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:42:37.227774 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:37.227757 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/befee5ec-3cd2-416c-9d76-a02adfceceb6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "befee5ec-3cd2-416c-9d76-a02adfceceb6" (UID: "befee5ec-3cd2-416c-9d76-a02adfceceb6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:42:37.227979 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:37.227958 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/befee5ec-3cd2-416c-9d76-a02adfceceb6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "befee5ec-3cd2-416c-9d76-a02adfceceb6" (UID: "befee5ec-3cd2-416c-9d76-a02adfceceb6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 02:42:37.228031 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:37.227995 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/befee5ec-3cd2-416c-9d76-a02adfceceb6-credential-socket\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:42:37.228031 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:37.228018 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/befee5ec-3cd2-416c-9d76-a02adfceceb6-workload-socket\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:42:37.228127 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:37.228033 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/befee5ec-3cd2-416c-9d76-a02adfceceb6-workload-certs\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:42:37.228341 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:37.228319 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/befee5ec-3cd2-416c-9d76-a02adfceceb6-istio-data" (OuterVolumeSpecName: "istio-data") pod "befee5ec-3cd2-416c-9d76-a02adfceceb6" (UID: "befee5ec-3cd2-416c-9d76-a02adfceceb6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:42:37.229542 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:37.229524 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/befee5ec-3cd2-416c-9d76-a02adfceceb6-istio-token" (OuterVolumeSpecName: "istio-token") pod "befee5ec-3cd2-416c-9d76-a02adfceceb6" (UID: "befee5ec-3cd2-416c-9d76-a02adfceceb6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:42:37.229833 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:37.229809 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/befee5ec-3cd2-416c-9d76-a02adfceceb6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "befee5ec-3cd2-416c-9d76-a02adfceceb6" (UID: "befee5ec-3cd2-416c-9d76-a02adfceceb6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:42:37.229938 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:37.229921 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/befee5ec-3cd2-416c-9d76-a02adfceceb6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "befee5ec-3cd2-416c-9d76-a02adfceceb6" (UID: "befee5ec-3cd2-416c-9d76-a02adfceceb6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 02:42:37.230040 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:37.230026 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/befee5ec-3cd2-416c-9d76-a02adfceceb6-kube-api-access-gf2c8" (OuterVolumeSpecName: "kube-api-access-gf2c8") pod "befee5ec-3cd2-416c-9d76-a02adfceceb6" (UID: "befee5ec-3cd2-416c-9d76-a02adfceceb6"). InnerVolumeSpecName "kube-api-access-gf2c8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:42:37.328640 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:37.328595 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/befee5ec-3cd2-416c-9d76-a02adfceceb6-istio-podinfo\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:42:37.328640 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:37.328630 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/befee5ec-3cd2-416c-9d76-a02adfceceb6-istiod-ca-cert\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:42:37.328640 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:37.328640 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/befee5ec-3cd2-416c-9d76-a02adfceceb6-istio-envoy\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:42:37.328640 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:37.328650 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gf2c8\" (UniqueName: \"kubernetes.io/projected/befee5ec-3cd2-416c-9d76-a02adfceceb6-kube-api-access-gf2c8\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:42:37.328919 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:37.328661 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/befee5ec-3cd2-416c-9d76-a02adfceceb6-istio-token\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:42:37.328919 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:37.328669 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/befee5ec-3cd2-416c-9d76-a02adfceceb6-istio-data\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:42:38.003339 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:38.003303 2577 generic.go:356] "Generic (PLEG): container finished" podID="befee5ec-3cd2-416c-9d76-a02adfceceb6" containerID="3dea8f6a39a0bdcb47c3ab594376f85b45747188f8e1d4e54c4953bb0e898090" exitCode=0 Jul 14 02:42:38.003516 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:38.003358 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" Jul 14 02:42:38.003516 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:38.003384 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" event={"ID":"befee5ec-3cd2-416c-9d76-a02adfceceb6","Type":"ContainerDied","Data":"3dea8f6a39a0bdcb47c3ab594376f85b45747188f8e1d4e54c4953bb0e898090"} Jul 14 02:42:38.003516 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:38.003464 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4" event={"ID":"befee5ec-3cd2-416c-9d76-a02adfceceb6","Type":"ContainerDied","Data":"2d56382e9c34f22d7495a5849d499dec214f843a904e167826a044c2cb0a0d5d"} Jul 14 02:42:38.003516 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:38.003488 2577 scope.go:117] "RemoveContainer" containerID="3dea8f6a39a0bdcb47c3ab594376f85b45747188f8e1d4e54c4953bb0e898090" Jul 14 02:42:38.013705 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:38.013675 2577 scope.go:117] "RemoveContainer" containerID="a3f430b7547fe7ccad750c0e35677782fe24f4c7a8b0ec5ccf143920ebdac4bb" Jul 14 02:42:38.022167 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:38.022146 2577 scope.go:117] "RemoveContainer" containerID="3dea8f6a39a0bdcb47c3ab594376f85b45747188f8e1d4e54c4953bb0e898090" Jul 14 02:42:38.022486 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:42:38.022456 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3dea8f6a39a0bdcb47c3ab594376f85b45747188f8e1d4e54c4953bb0e898090\": container with ID starting with 3dea8f6a39a0bdcb47c3ab594376f85b45747188f8e1d4e54c4953bb0e898090 not found: ID does not exist" containerID="3dea8f6a39a0bdcb47c3ab594376f85b45747188f8e1d4e54c4953bb0e898090" Jul 14 02:42:38.022575 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:38.022490 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3dea8f6a39a0bdcb47c3ab594376f85b45747188f8e1d4e54c4953bb0e898090"} err="failed to get container status \"3dea8f6a39a0bdcb47c3ab594376f85b45747188f8e1d4e54c4953bb0e898090\": rpc error: code = NotFound desc = could not find container \"3dea8f6a39a0bdcb47c3ab594376f85b45747188f8e1d4e54c4953bb0e898090\": container with ID starting with 3dea8f6a39a0bdcb47c3ab594376f85b45747188f8e1d4e54c4953bb0e898090 not found: ID does not exist" Jul 14 02:42:38.022575 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:38.022507 2577 scope.go:117] "RemoveContainer" containerID="a3f430b7547fe7ccad750c0e35677782fe24f4c7a8b0ec5ccf143920ebdac4bb" Jul 14 02:42:38.022791 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:42:38.022771 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3f430b7547fe7ccad750c0e35677782fe24f4c7a8b0ec5ccf143920ebdac4bb\": container with ID starting with a3f430b7547fe7ccad750c0e35677782fe24f4c7a8b0ec5ccf143920ebdac4bb not found: ID does not exist" containerID="a3f430b7547fe7ccad750c0e35677782fe24f4c7a8b0ec5ccf143920ebdac4bb" Jul 14 02:42:38.022837 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:38.022798 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3f430b7547fe7ccad750c0e35677782fe24f4c7a8b0ec5ccf143920ebdac4bb"} err="failed to get container status \"a3f430b7547fe7ccad750c0e35677782fe24f4c7a8b0ec5ccf143920ebdac4bb\": rpc error: code = NotFound desc = could not find container \"a3f430b7547fe7ccad750c0e35677782fe24f4c7a8b0ec5ccf143920ebdac4bb\": container with ID starting with a3f430b7547fe7ccad750c0e35677782fe24f4c7a8b0ec5ccf143920ebdac4bb not found: ID does not exist" Jul 14 02:42:38.025835 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:38.025814 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4"] Jul 14 02:42:38.030789 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:38.030770 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-9794b6-openshift-default-5579744dbf-p8md4"] Jul 14 02:42:38.266148 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:38.266071 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="befee5ec-3cd2-416c-9d76-a02adfceceb6" path="/var/lib/kubelet/pods/befee5ec-3cd2-416c-9d76-a02adfceceb6/volumes" Jul 14 02:42:42.605282 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:42.605231 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:42:47.325859 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:47.325825 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:42:52.925098 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:52.925062 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:42:52.982463 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:52.982428 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:42:57.987386 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:57.987347 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq"] Jul 14 02:42:57.987946 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:57.987916 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" podUID="ea69d5a7-16a9-4661-9aac-c2288e7a867b" containerName="istio-proxy" containerID="cri-o://ace4ee61869edf61288db96d20d638a3711176e5e62a6661a67c70b4ff688a7b" gracePeriod=30 Jul 14 02:42:58.007936 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:58.007909 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:42:58.243307 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:58.243203 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:42:58.576334 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:58.576226 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:42:58.862123 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:58.862090 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg"] Jul 14 02:42:58.862648 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:58.862621 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="befee5ec-3cd2-416c-9d76-a02adfceceb6" containerName="istio-proxy" Jul 14 02:42:58.862648 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:58.862647 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="befee5ec-3cd2-416c-9d76-a02adfceceb6" containerName="istio-proxy" Jul 14 02:42:58.862853 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:58.862770 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="befee5ec-3cd2-416c-9d76-a02adfceceb6" containerName="istio-proxy" Jul 14 02:42:58.862853 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:58.862782 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="befee5ec-3cd2-416c-9d76-a02adfceceb6" containerName="istio-proxy" Jul 14 02:42:58.862853 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:58.862793 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bba6bbd-d81f-4872-a8e6-2168e0c9079f" containerName="istio-proxy" Jul 14 02:42:58.866104 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:58.866086 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:42:58.868914 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:58.868888 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-a148c7-openshift-default-dockercfg-fp9z7\"" Jul 14 02:42:58.878758 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:58.878736 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg"] Jul 14 02:42:58.887978 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:58.887932 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:42:58.935116 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:58.935082 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8fb4874-8a24-4377-9707-589577b5e290-istio-token\") pod \"e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg\" (UID: \"e8fb4874-8a24-4377-9707-589577b5e290\") " pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:42:58.935353 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:58.935166 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8fb4874-8a24-4377-9707-589577b5e290-credential-socket\") pod \"e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg\" (UID: \"e8fb4874-8a24-4377-9707-589577b5e290\") " pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:42:58.935353 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:58.935192 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8fb4874-8a24-4377-9707-589577b5e290-istiod-ca-cert\") pod \"e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg\" (UID: \"e8fb4874-8a24-4377-9707-589577b5e290\") " pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:42:58.935353 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:58.935217 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8fb4874-8a24-4377-9707-589577b5e290-istio-envoy\") pod \"e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg\" (UID: \"e8fb4874-8a24-4377-9707-589577b5e290\") " pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:42:58.935353 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:58.935293 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8fb4874-8a24-4377-9707-589577b5e290-workload-certs\") pod \"e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg\" (UID: \"e8fb4874-8a24-4377-9707-589577b5e290\") " pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:42:58.935353 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:58.935348 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8fb4874-8a24-4377-9707-589577b5e290-workload-socket\") pod \"e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg\" (UID: \"e8fb4874-8a24-4377-9707-589577b5e290\") " pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:42:58.935640 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:58.935376 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8fb4874-8a24-4377-9707-589577b5e290-istio-data\") pod \"e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg\" (UID: \"e8fb4874-8a24-4377-9707-589577b5e290\") " pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:42:58.935640 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:58.935398 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8fb4874-8a24-4377-9707-589577b5e290-istio-podinfo\") pod \"e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg\" (UID: \"e8fb4874-8a24-4377-9707-589577b5e290\") " pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:42:58.935640 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:58.935503 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn8fm\" (UniqueName: \"kubernetes.io/projected/e8fb4874-8a24-4377-9707-589577b5e290-kube-api-access-gn8fm\") pod \"e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg\" (UID: \"e8fb4874-8a24-4377-9707-589577b5e290\") " pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:42:59.036704 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:59.036659 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8fb4874-8a24-4377-9707-589577b5e290-istio-token\") pod \"e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg\" (UID: \"e8fb4874-8a24-4377-9707-589577b5e290\") " pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:42:59.037174 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:59.036756 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8fb4874-8a24-4377-9707-589577b5e290-credential-socket\") pod \"e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg\" (UID: \"e8fb4874-8a24-4377-9707-589577b5e290\") " pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:42:59.037174 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:59.036777 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8fb4874-8a24-4377-9707-589577b5e290-istiod-ca-cert\") pod \"e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg\" (UID: \"e8fb4874-8a24-4377-9707-589577b5e290\") " pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:42:59.037174 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:59.036817 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8fb4874-8a24-4377-9707-589577b5e290-istio-envoy\") pod \"e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg\" (UID: \"e8fb4874-8a24-4377-9707-589577b5e290\") " pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:42:59.037174 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:59.036859 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8fb4874-8a24-4377-9707-589577b5e290-workload-certs\") pod \"e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg\" (UID: \"e8fb4874-8a24-4377-9707-589577b5e290\") " pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:42:59.037174 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:59.036897 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8fb4874-8a24-4377-9707-589577b5e290-workload-socket\") pod \"e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg\" (UID: \"e8fb4874-8a24-4377-9707-589577b5e290\") " pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:42:59.037174 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:59.036930 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8fb4874-8a24-4377-9707-589577b5e290-istio-data\") pod \"e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg\" (UID: \"e8fb4874-8a24-4377-9707-589577b5e290\") " pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:42:59.037174 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:59.036956 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8fb4874-8a24-4377-9707-589577b5e290-istio-podinfo\") pod \"e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg\" (UID: \"e8fb4874-8a24-4377-9707-589577b5e290\") " pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:42:59.037174 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:59.036995 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn8fm\" (UniqueName: \"kubernetes.io/projected/e8fb4874-8a24-4377-9707-589577b5e290-kube-api-access-gn8fm\") pod \"e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg\" (UID: \"e8fb4874-8a24-4377-9707-589577b5e290\") " pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:42:59.037707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:59.037203 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8fb4874-8a24-4377-9707-589577b5e290-credential-socket\") pod \"e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg\" (UID: \"e8fb4874-8a24-4377-9707-589577b5e290\") " pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:42:59.037707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:59.037328 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8fb4874-8a24-4377-9707-589577b5e290-workload-certs\") pod \"e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg\" (UID: \"e8fb4874-8a24-4377-9707-589577b5e290\") " pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:42:59.037707 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:59.037545 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8fb4874-8a24-4377-9707-589577b5e290-workload-socket\") pod \"e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg\" (UID: \"e8fb4874-8a24-4377-9707-589577b5e290\") " pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:42:59.037870 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:59.037779 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8fb4874-8a24-4377-9707-589577b5e290-istio-data\") pod \"e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg\" (UID: \"e8fb4874-8a24-4377-9707-589577b5e290\") " pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:42:59.037927 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:59.037889 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8fb4874-8a24-4377-9707-589577b5e290-istiod-ca-cert\") pod \"e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg\" (UID: \"e8fb4874-8a24-4377-9707-589577b5e290\") " pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:42:59.041943 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:59.040252 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8fb4874-8a24-4377-9707-589577b5e290-istio-podinfo\") pod \"e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg\" (UID: \"e8fb4874-8a24-4377-9707-589577b5e290\") " pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:42:59.042086 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:59.042021 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8fb4874-8a24-4377-9707-589577b5e290-istio-envoy\") pod \"e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg\" (UID: \"e8fb4874-8a24-4377-9707-589577b5e290\") " pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:42:59.044030 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:59.043999 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8fb4874-8a24-4377-9707-589577b5e290-istio-token\") pod \"e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg\" (UID: \"e8fb4874-8a24-4377-9707-589577b5e290\") " pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:42:59.044298 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:59.044257 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn8fm\" (UniqueName: \"kubernetes.io/projected/e8fb4874-8a24-4377-9707-589577b5e290-kube-api-access-gn8fm\") pod \"e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg\" (UID: \"e8fb4874-8a24-4377-9707-589577b5e290\") " pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:42:59.068644 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:59.068620 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:42:59.177685 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:59.177583 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:42:59.316694 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:59.316669 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg"] Jul 14 02:42:59.318336 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:42:59.318308 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8fb4874_8a24_4377_9707_589577b5e290.slice/crio-dce622d782388ea4bbbd1d7f6181ceda4d34065323a36f37830671a9cdac8f5c WatchSource:0}: Error finding container dce622d782388ea4bbbd1d7f6181ceda4d34065323a36f37830671a9cdac8f5c: Status 404 returned error can't find the container with id dce622d782388ea4bbbd1d7f6181ceda4d34065323a36f37830671a9cdac8f5c Jul 14 02:42:59.320476 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:59.320444 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:42:59.320586 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:59.320526 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:42:59.320586 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:42:59.320571 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:43:00.111362 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:00.111321 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" event={"ID":"e8fb4874-8a24-4377-9707-589577b5e290","Type":"ContainerStarted","Data":"2e67c5b66203aea5b4d249138499140ecc228c959849624c5b6ff34434ba50b6"} Jul 14 02:43:00.111764 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:00.111373 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" event={"ID":"e8fb4874-8a24-4377-9707-589577b5e290","Type":"ContainerStarted","Data":"dce622d782388ea4bbbd1d7f6181ceda4d34065323a36f37830671a9cdac8f5c"} Jul 14 02:43:00.129840 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:00.129786 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" podStartSLOduration=2.129769808 podStartE2EDuration="2.129769808s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:42:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:43:00.128422145 +0000 UTC m=+2482.283258871" watchObservedRunningTime="2026-07-14 02:43:00.129769808 +0000 UTC m=+2482.284606455" Jul 14 02:43:00.177802 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:00.177768 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:43:00.182629 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:00.182602 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:43:01.116410 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:01.116379 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:43:01.117488 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:01.117471 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:43:02.025525 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:02.025488 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:43:02.103725 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:02.103694 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:43:02.162660 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:02.162629 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:43:02.262646 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:02.262615 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:43:02.700820 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:02.700789 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:43:02.785104 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:02.785076 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:43:03.127220 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:03.127188 2577 generic.go:356] "Generic (PLEG): container finished" podID="ea69d5a7-16a9-4661-9aac-c2288e7a867b" containerID="ace4ee61869edf61288db96d20d638a3711176e5e62a6661a67c70b4ff688a7b" exitCode=0 Jul 14 02:43:03.127391 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:03.127258 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" event={"ID":"ea69d5a7-16a9-4661-9aac-c2288e7a867b","Type":"ContainerDied","Data":"ace4ee61869edf61288db96d20d638a3711176e5e62a6661a67c70b4ff688a7b"} Jul 14 02:43:03.127391 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:03.127335 2577 scope.go:117] "RemoveContainer" containerID="3601419e9f5d46de2a4af0f3e45f2084082db3758561bdc81d1cd8a6072e5321" Jul 14 02:43:03.268051 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:03.267990 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:43:03.278730 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:03.278704 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea69d5a7-16a9-4661-9aac-c2288e7a867b-workload-certs\") pod \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\" (UID: \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\") " Jul 14 02:43:03.278888 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:03.278756 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea69d5a7-16a9-4661-9aac-c2288e7a867b-workload-socket\") pod \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\" (UID: \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\") " Jul 14 02:43:03.278888 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:03.278798 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea69d5a7-16a9-4661-9aac-c2288e7a867b-istio-envoy\") pod \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\" (UID: \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\") " Jul 14 02:43:03.278888 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:03.278824 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea69d5a7-16a9-4661-9aac-c2288e7a867b-istio-podinfo\") pod \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\" (UID: \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\") " Jul 14 02:43:03.278888 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:03.278858 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea69d5a7-16a9-4661-9aac-c2288e7a867b-credential-socket\") pod \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\" (UID: \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\") " Jul 14 02:43:03.278888 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:03.278885 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea69d5a7-16a9-4661-9aac-c2288e7a867b-istiod-ca-cert\") pod \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\" (UID: \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\") " Jul 14 02:43:03.279142 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:03.278936 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea69d5a7-16a9-4661-9aac-c2288e7a867b-istio-token\") pod \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\" (UID: \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\") " Jul 14 02:43:03.279142 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:03.278963 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea69d5a7-16a9-4661-9aac-c2288e7a867b-istio-data\") pod \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\" (UID: \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\") " Jul 14 02:43:03.279142 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:03.278990 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tpj6t\" (UniqueName: \"kubernetes.io/projected/ea69d5a7-16a9-4661-9aac-c2288e7a867b-kube-api-access-tpj6t\") pod \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\" (UID: \"ea69d5a7-16a9-4661-9aac-c2288e7a867b\") " Jul 14 02:43:03.279142 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:03.278998 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea69d5a7-16a9-4661-9aac-c2288e7a867b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ea69d5a7-16a9-4661-9aac-c2288e7a867b" (UID: "ea69d5a7-16a9-4661-9aac-c2288e7a867b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:43:03.279142 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:03.279109 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea69d5a7-16a9-4661-9aac-c2288e7a867b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ea69d5a7-16a9-4661-9aac-c2288e7a867b" (UID: "ea69d5a7-16a9-4661-9aac-c2288e7a867b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:43:03.279427 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:03.279198 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea69d5a7-16a9-4661-9aac-c2288e7a867b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ea69d5a7-16a9-4661-9aac-c2288e7a867b" (UID: "ea69d5a7-16a9-4661-9aac-c2288e7a867b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:43:03.279427 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:03.279419 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea69d5a7-16a9-4661-9aac-c2288e7a867b-workload-certs\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:43:03.279547 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:03.279438 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea69d5a7-16a9-4661-9aac-c2288e7a867b-workload-socket\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:43:03.279547 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:03.279452 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea69d5a7-16a9-4661-9aac-c2288e7a867b-credential-socket\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:43:03.279651 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:03.279601 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea69d5a7-16a9-4661-9aac-c2288e7a867b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ea69d5a7-16a9-4661-9aac-c2288e7a867b" (UID: "ea69d5a7-16a9-4661-9aac-c2288e7a867b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 02:43:03.279933 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:03.279911 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea69d5a7-16a9-4661-9aac-c2288e7a867b-istio-data" (OuterVolumeSpecName: "istio-data") pod "ea69d5a7-16a9-4661-9aac-c2288e7a867b" (UID: "ea69d5a7-16a9-4661-9aac-c2288e7a867b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:43:03.280999 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:03.280977 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ea69d5a7-16a9-4661-9aac-c2288e7a867b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ea69d5a7-16a9-4661-9aac-c2288e7a867b" (UID: "ea69d5a7-16a9-4661-9aac-c2288e7a867b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 02:43:03.281227 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:03.281208 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea69d5a7-16a9-4661-9aac-c2288e7a867b-kube-api-access-tpj6t" (OuterVolumeSpecName: "kube-api-access-tpj6t") pod "ea69d5a7-16a9-4661-9aac-c2288e7a867b" (UID: "ea69d5a7-16a9-4661-9aac-c2288e7a867b"). InnerVolumeSpecName "kube-api-access-tpj6t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:43:03.281455 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:03.281419 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea69d5a7-16a9-4661-9aac-c2288e7a867b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ea69d5a7-16a9-4661-9aac-c2288e7a867b" (UID: "ea69d5a7-16a9-4661-9aac-c2288e7a867b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:43:03.281547 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:03.281452 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea69d5a7-16a9-4661-9aac-c2288e7a867b-istio-token" (OuterVolumeSpecName: "istio-token") pod "ea69d5a7-16a9-4661-9aac-c2288e7a867b" (UID: "ea69d5a7-16a9-4661-9aac-c2288e7a867b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:43:03.380357 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:03.380320 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea69d5a7-16a9-4661-9aac-c2288e7a867b-istio-envoy\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:43:03.380357 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:03.380350 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea69d5a7-16a9-4661-9aac-c2288e7a867b-istio-podinfo\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:43:03.380357 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:03.380359 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea69d5a7-16a9-4661-9aac-c2288e7a867b-istiod-ca-cert\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:43:03.380357 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:03.380366 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea69d5a7-16a9-4661-9aac-c2288e7a867b-istio-token\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:43:03.380630 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:03.380374 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea69d5a7-16a9-4661-9aac-c2288e7a867b-istio-data\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:43:03.380630 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:03.380382 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tpj6t\" (UniqueName: \"kubernetes.io/projected/ea69d5a7-16a9-4661-9aac-c2288e7a867b-kube-api-access-tpj6t\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:43:04.134198 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:04.134159 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" event={"ID":"ea69d5a7-16a9-4661-9aac-c2288e7a867b","Type":"ContainerDied","Data":"c38af05ec4fb598185443691e7e79d619887490861835e567110ebbe5715412b"} Jul 14 02:43:04.134398 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:04.134223 2577 scope.go:117] "RemoveContainer" containerID="ace4ee61869edf61288db96d20d638a3711176e5e62a6661a67c70b4ff688a7b" Jul 14 02:43:04.134398 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:04.134181 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq" Jul 14 02:43:04.154716 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:04.154666 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq"] Jul 14 02:43:04.156859 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:04.156837 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-f83e72-openshift-default-8567fb4675-sn2wq"] Jul 14 02:43:04.264652 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:04.264614 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea69d5a7-16a9-4661-9aac-c2288e7a867b" path="/var/lib/kubelet/pods/ea69d5a7-16a9-4661-9aac-c2288e7a867b/volumes" Jul 14 02:43:06.529010 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:06.528974 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:43:06.857491 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:06.857366 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:43:07.124886 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.124856 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft"] Jul 14 02:43:07.125260 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.125240 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="befee5ec-3cd2-416c-9d76-a02adfceceb6" containerName="istio-proxy" Jul 14 02:43:07.125333 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.125262 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="befee5ec-3cd2-416c-9d76-a02adfceceb6" containerName="istio-proxy" Jul 14 02:43:07.125333 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.125287 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea69d5a7-16a9-4661-9aac-c2288e7a867b" containerName="istio-proxy" Jul 14 02:43:07.125333 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.125293 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea69d5a7-16a9-4661-9aac-c2288e7a867b" containerName="istio-proxy" Jul 14 02:43:07.125437 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.125363 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea69d5a7-16a9-4661-9aac-c2288e7a867b" containerName="istio-proxy" Jul 14 02:43:07.129737 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.129716 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:43:07.131758 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.131732 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-c47335-openshift-default-dockercfg-bpccg\"" Jul 14 02:43:07.137985 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.137945 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft"] Jul 14 02:43:07.146911 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.146888 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:43:07.216956 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.216920 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/163d1a57-7acb-4502-868f-dc4518a5d995-workload-certs\") pod \"e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft\" (UID: \"163d1a57-7acb-4502-868f-dc4518a5d995\") " pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:43:07.217136 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.217027 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/163d1a57-7acb-4502-868f-dc4518a5d995-istio-podinfo\") pod \"e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft\" (UID: \"163d1a57-7acb-4502-868f-dc4518a5d995\") " pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:43:07.217136 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.217066 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/163d1a57-7acb-4502-868f-dc4518a5d995-istio-token\") pod \"e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft\" (UID: \"163d1a57-7acb-4502-868f-dc4518a5d995\") " pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:43:07.217136 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.217114 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t5mq\" (UniqueName: \"kubernetes.io/projected/163d1a57-7acb-4502-868f-dc4518a5d995-kube-api-access-5t5mq\") pod \"e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft\" (UID: \"163d1a57-7acb-4502-868f-dc4518a5d995\") " pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:43:07.217362 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.217141 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/163d1a57-7acb-4502-868f-dc4518a5d995-credential-socket\") pod \"e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft\" (UID: \"163d1a57-7acb-4502-868f-dc4518a5d995\") " pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:43:07.217362 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.217292 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/163d1a57-7acb-4502-868f-dc4518a5d995-istio-envoy\") pod \"e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft\" (UID: \"163d1a57-7acb-4502-868f-dc4518a5d995\") " pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:43:07.217362 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.217353 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/163d1a57-7acb-4502-868f-dc4518a5d995-istiod-ca-cert\") pod \"e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft\" (UID: \"163d1a57-7acb-4502-868f-dc4518a5d995\") " pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:43:07.217510 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.217428 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/163d1a57-7acb-4502-868f-dc4518a5d995-istio-data\") pod \"e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft\" (UID: \"163d1a57-7acb-4502-868f-dc4518a5d995\") " pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:43:07.217510 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.217476 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/163d1a57-7acb-4502-868f-dc4518a5d995-workload-socket\") pod \"e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft\" (UID: \"163d1a57-7acb-4502-868f-dc4518a5d995\") " pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:43:07.318166 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.318128 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/163d1a57-7acb-4502-868f-dc4518a5d995-credential-socket\") pod \"e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft\" (UID: \"163d1a57-7acb-4502-868f-dc4518a5d995\") " pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:43:07.318369 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.318200 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/163d1a57-7acb-4502-868f-dc4518a5d995-istio-envoy\") pod \"e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft\" (UID: \"163d1a57-7acb-4502-868f-dc4518a5d995\") " pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:43:07.318369 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.318238 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/163d1a57-7acb-4502-868f-dc4518a5d995-istiod-ca-cert\") pod \"e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft\" (UID: \"163d1a57-7acb-4502-868f-dc4518a5d995\") " pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:43:07.318369 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.318267 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/163d1a57-7acb-4502-868f-dc4518a5d995-istio-data\") pod \"e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft\" (UID: \"163d1a57-7acb-4502-868f-dc4518a5d995\") " pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:43:07.318369 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.318333 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/163d1a57-7acb-4502-868f-dc4518a5d995-workload-socket\") pod \"e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft\" (UID: \"163d1a57-7acb-4502-868f-dc4518a5d995\") " pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:43:07.318606 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.318378 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/163d1a57-7acb-4502-868f-dc4518a5d995-workload-certs\") pod \"e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft\" (UID: \"163d1a57-7acb-4502-868f-dc4518a5d995\") " pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:43:07.318606 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.318410 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/163d1a57-7acb-4502-868f-dc4518a5d995-istio-podinfo\") pod \"e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft\" (UID: \"163d1a57-7acb-4502-868f-dc4518a5d995\") " pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:43:07.318606 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.318443 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/163d1a57-7acb-4502-868f-dc4518a5d995-istio-token\") pod \"e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft\" (UID: \"163d1a57-7acb-4502-868f-dc4518a5d995\") " pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:43:07.318606 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.318495 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t5mq\" (UniqueName: \"kubernetes.io/projected/163d1a57-7acb-4502-868f-dc4518a5d995-kube-api-access-5t5mq\") pod \"e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft\" (UID: \"163d1a57-7acb-4502-868f-dc4518a5d995\") " pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:43:07.318606 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.318584 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/163d1a57-7acb-4502-868f-dc4518a5d995-credential-socket\") pod \"e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft\" (UID: \"163d1a57-7acb-4502-868f-dc4518a5d995\") " pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:43:07.318831 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.318681 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/163d1a57-7acb-4502-868f-dc4518a5d995-istio-data\") pod \"e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft\" (UID: \"163d1a57-7acb-4502-868f-dc4518a5d995\") " pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:43:07.318831 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.318764 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/163d1a57-7acb-4502-868f-dc4518a5d995-workload-socket\") pod \"e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft\" (UID: \"163d1a57-7acb-4502-868f-dc4518a5d995\") " pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:43:07.318951 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.318823 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/163d1a57-7acb-4502-868f-dc4518a5d995-workload-certs\") pod \"e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft\" (UID: \"163d1a57-7acb-4502-868f-dc4518a5d995\") " pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:43:07.318951 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.318916 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/163d1a57-7acb-4502-868f-dc4518a5d995-istiod-ca-cert\") pod \"e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft\" (UID: \"163d1a57-7acb-4502-868f-dc4518a5d995\") " pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:43:07.320528 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.320501 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/163d1a57-7acb-4502-868f-dc4518a5d995-istio-envoy\") pod \"e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft\" (UID: \"163d1a57-7acb-4502-868f-dc4518a5d995\") " pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:43:07.320924 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.320897 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/163d1a57-7acb-4502-868f-dc4518a5d995-istio-podinfo\") pod \"e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft\" (UID: \"163d1a57-7acb-4502-868f-dc4518a5d995\") " pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:43:07.326639 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.326605 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/163d1a57-7acb-4502-868f-dc4518a5d995-istio-token\") pod \"e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft\" (UID: \"163d1a57-7acb-4502-868f-dc4518a5d995\") " pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:43:07.326770 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.326720 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t5mq\" (UniqueName: \"kubernetes.io/projected/163d1a57-7acb-4502-868f-dc4518a5d995-kube-api-access-5t5mq\") pod \"e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft\" (UID: \"163d1a57-7acb-4502-868f-dc4518a5d995\") " pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:43:07.358174 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.358132 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:43:07.442250 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.442144 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:43:07.605238 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.605213 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft"] Jul 14 02:43:07.607464 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:43:07.607434 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod163d1a57_7acb_4502_868f_dc4518a5d995.slice/crio-ecbbcda3801fef388ef17ea3617ff1152c9b620cd786f3fc701174082af994d4 WatchSource:0}: Error finding container ecbbcda3801fef388ef17ea3617ff1152c9b620cd786f3fc701174082af994d4: Status 404 returned error can't find the container with id ecbbcda3801fef388ef17ea3617ff1152c9b620cd786f3fc701174082af994d4 Jul 14 02:43:07.610048 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.610015 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:43:07.610160 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.610088 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:43:07.610160 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:07.610131 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:43:08.160311 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:08.160254 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" event={"ID":"163d1a57-7acb-4502-868f-dc4518a5d995","Type":"ContainerStarted","Data":"14a7859301f75abf963207cd62fe2eaae2a34c3d93e19381818fd04c894bdb58"} Jul 14 02:43:08.160502 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:08.160346 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" event={"ID":"163d1a57-7acb-4502-868f-dc4518a5d995","Type":"ContainerStarted","Data":"ecbbcda3801fef388ef17ea3617ff1152c9b620cd786f3fc701174082af994d4"} Jul 14 02:43:08.184517 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:08.184463 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" podStartSLOduration=1.184448246 podStartE2EDuration="1.184448246s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:43:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:43:08.182095614 +0000 UTC m=+2490.336932263" watchObservedRunningTime="2026-07-14 02:43:08.184448246 +0000 UTC m=+2490.339284893" Jul 14 02:43:08.443159 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:08.443059 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:43:08.461107 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:08.461060 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:43:09.167260 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:09.167223 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:43:09.168410 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:09.168388 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:43:10.077706 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:10.077671 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:43:10.177449 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:10.177417 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:43:10.282570 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:10.282530 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:43:10.776627 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:10.776592 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:43:10.878389 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:10.878355 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:43:10.934643 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:10.934605 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:43:14.579090 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:14.579045 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:43:15.197414 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:15.197327 2577 generic.go:356] "Generic (PLEG): container finished" podID="e8fb4874-8a24-4377-9707-589577b5e290" containerID="2e67c5b66203aea5b4d249138499140ecc228c959849624c5b6ff34434ba50b6" exitCode=0 Jul 14 02:43:15.197414 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:15.197366 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" event={"ID":"e8fb4874-8a24-4377-9707-589577b5e290","Type":"ContainerDied","Data":"2e67c5b66203aea5b4d249138499140ecc228c959849624c5b6ff34434ba50b6"} Jul 14 02:43:15.197792 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:15.197765 2577 scope.go:117] "RemoveContainer" containerID="2e67c5b66203aea5b4d249138499140ecc228c959849624c5b6ff34434ba50b6" Jul 14 02:43:15.198805 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:15.198774 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:43:15.198924 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:15.198882 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:43:15.198993 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:15.198933 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:43:16.116908 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:16.116877 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:43:16.203896 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:16.203855 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" event={"ID":"e8fb4874-8a24-4377-9707-589577b5e290","Type":"ContainerStarted","Data":"e99dea84b256cd15a7a49c87f178219c8473dc742d7d2c8a76c86b36966d4d1c"} Jul 14 02:43:17.179222 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:17.179181 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:43:17.179936 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:17.179900 2577 patch_prober.go:28] interesting pod/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 14 02:43:17.180046 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:17.179993 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" podUID="e8fb4874-8a24-4377-9707-589577b5e290" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 14 02:43:18.178879 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:18.178841 2577 patch_prober.go:28] interesting pod/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 14 02:43:18.179177 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:18.179148 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" podUID="e8fb4874-8a24-4377-9707-589577b5e290" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 14 02:43:19.178161 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:19.178124 2577 patch_prober.go:28] interesting pod/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 14 02:43:19.178577 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:19.178181 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" podUID="e8fb4874-8a24-4377-9707-589577b5e290" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 14 02:43:20.178008 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:20.177965 2577 patch_prober.go:28] interesting pod/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 14 02:43:20.178228 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:20.178044 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" podUID="e8fb4874-8a24-4377-9707-589577b5e290" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 14 02:43:21.182668 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:21.182636 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:43:21.183063 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:21.183036 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:43:21.183698 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:21.183676 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:43:22.238573 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:22.238479 2577 generic.go:356] "Generic (PLEG): container finished" podID="163d1a57-7acb-4502-868f-dc4518a5d995" containerID="14a7859301f75abf963207cd62fe2eaae2a34c3d93e19381818fd04c894bdb58" exitCode=0 Jul 14 02:43:22.238573 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:22.238519 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" event={"ID":"163d1a57-7acb-4502-868f-dc4518a5d995","Type":"ContainerDied","Data":"14a7859301f75abf963207cd62fe2eaae2a34c3d93e19381818fd04c894bdb58"} Jul 14 02:43:22.239144 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:22.239128 2577 scope.go:117] "RemoveContainer" containerID="14a7859301f75abf963207cd62fe2eaae2a34c3d93e19381818fd04c894bdb58" Jul 14 02:43:22.240223 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:22.240197 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:43:22.240318 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:22.240301 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:43:22.240373 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:22.240342 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 02:43:23.249964 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:23.249913 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" event={"ID":"163d1a57-7acb-4502-868f-dc4518a5d995","Type":"ContainerStarted","Data":"66eb3181cba5790a8e8b8849ac8c0487bafdb03a9c805a71c92050fe4a0ac343"} Jul 14 02:43:23.366985 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:23.366949 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:43:23.444248 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:23.444148 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:43:23.444611 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:23.444578 2577 patch_prober.go:28] interesting pod/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 14 02:43:23.444852 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:23.444822 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" podUID="163d1a57-7acb-4502-868f-dc4518a5d995" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 14 02:43:23.467578 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:23.467545 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:43:24.167565 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:24.167530 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:43:24.443627 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:24.443531 2577 patch_prober.go:28] interesting pod/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 14 02:43:24.444188 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:24.444158 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" podUID="163d1a57-7acb-4502-868f-dc4518a5d995" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 14 02:43:25.443621 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:25.443141 2577 patch_prober.go:28] interesting pod/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 14 02:43:25.443621 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:25.443211 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" podUID="163d1a57-7acb-4502-868f-dc4518a5d995" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 14 02:43:26.443052 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:26.443012 2577 patch_prober.go:28] interesting pod/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 14 02:43:26.443405 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:26.443372 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" podUID="163d1a57-7acb-4502-868f-dc4518a5d995" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 14 02:43:27.446712 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:27.446685 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:43:27.447818 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:27.447800 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:43:32.383166 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:32.383081 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:43:32.480815 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:32.480780 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:43:35.083198 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:35.083160 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-a148c7/auth-test-model-a148c7-kserve-59869c9747-ffsxx"] Jul 14 02:43:35.083589 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:35.083562 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea69d5a7-16a9-4661-9aac-c2288e7a867b" containerName="istio-proxy" Jul 14 02:43:35.083589 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:35.083573 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea69d5a7-16a9-4661-9aac-c2288e7a867b" containerName="istio-proxy" Jul 14 02:43:35.083704 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:35.083649 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea69d5a7-16a9-4661-9aac-c2288e7a867b" containerName="istio-proxy" Jul 14 02:43:35.088259 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:35.088238 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a148c7/auth-test-model-a148c7-kserve-59869c9747-ffsxx" Jul 14 02:43:35.090969 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:35.090948 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a148c7\"/\"default-dockercfg-888g6\"" Jul 14 02:43:35.090969 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:35.090962 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a148c7\"/\"openshift-service-ca.crt\"" Jul 14 02:43:35.091143 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:35.090950 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a148c7\"/\"kube-root-ca.crt\"" Jul 14 02:43:35.091143 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:35.090950 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a148c7\"/\"auth-test-model-a148c7-kserve-self-signed-certs\"" Jul 14 02:43:35.098170 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:35.098142 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a148c7/auth-test-model-a148c7-kserve-59869c9747-ffsxx"] Jul 14 02:43:35.106507 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:35.106474 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:43:35.114773 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:35.114752 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a587527c-d6ed-4b66-802a-40bde985384d-tls-certs\") pod \"auth-test-model-a148c7-kserve-59869c9747-ffsxx\" (UID: \"a587527c-d6ed-4b66-802a-40bde985384d\") " pod="ai-tenant-e2e-shared-a-a148c7/auth-test-model-a148c7-kserve-59869c9747-ffsxx" Jul 14 02:43:35.114879 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:35.114788 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a587527c-d6ed-4b66-802a-40bde985384d-model-cache\") pod \"auth-test-model-a148c7-kserve-59869c9747-ffsxx\" (UID: \"a587527c-d6ed-4b66-802a-40bde985384d\") " pod="ai-tenant-e2e-shared-a-a148c7/auth-test-model-a148c7-kserve-59869c9747-ffsxx" Jul 14 02:43:35.114879 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:35.114857 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgmm9\" (UniqueName: \"kubernetes.io/projected/a587527c-d6ed-4b66-802a-40bde985384d-kube-api-access-bgmm9\") pod \"auth-test-model-a148c7-kserve-59869c9747-ffsxx\" (UID: \"a587527c-d6ed-4b66-802a-40bde985384d\") " pod="ai-tenant-e2e-shared-a-a148c7/auth-test-model-a148c7-kserve-59869c9747-ffsxx" Jul 14 02:43:35.114952 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:35.114910 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a587527c-d6ed-4b66-802a-40bde985384d-tmp-dir\") pod \"auth-test-model-a148c7-kserve-59869c9747-ffsxx\" (UID: \"a587527c-d6ed-4b66-802a-40bde985384d\") " pod="ai-tenant-e2e-shared-a-a148c7/auth-test-model-a148c7-kserve-59869c9747-ffsxx" Jul 14 02:43:35.114952 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:35.114931 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a587527c-d6ed-4b66-802a-40bde985384d-home\") pod \"auth-test-model-a148c7-kserve-59869c9747-ffsxx\" (UID: \"a587527c-d6ed-4b66-802a-40bde985384d\") " pod="ai-tenant-e2e-shared-a-a148c7/auth-test-model-a148c7-kserve-59869c9747-ffsxx" Jul 14 02:43:35.114952 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:35.114950 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a587527c-d6ed-4b66-802a-40bde985384d-dshm\") pod \"auth-test-model-a148c7-kserve-59869c9747-ffsxx\" (UID: \"a587527c-d6ed-4b66-802a-40bde985384d\") " pod="ai-tenant-e2e-shared-a-a148c7/auth-test-model-a148c7-kserve-59869c9747-ffsxx" Jul 14 02:43:35.216262 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:35.216213 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgmm9\" (UniqueName: \"kubernetes.io/projected/a587527c-d6ed-4b66-802a-40bde985384d-kube-api-access-bgmm9\") pod \"auth-test-model-a148c7-kserve-59869c9747-ffsxx\" (UID: \"a587527c-d6ed-4b66-802a-40bde985384d\") " pod="ai-tenant-e2e-shared-a-a148c7/auth-test-model-a148c7-kserve-59869c9747-ffsxx" Jul 14 02:43:35.216482 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:35.216295 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a587527c-d6ed-4b66-802a-40bde985384d-tmp-dir\") pod \"auth-test-model-a148c7-kserve-59869c9747-ffsxx\" (UID: \"a587527c-d6ed-4b66-802a-40bde985384d\") " pod="ai-tenant-e2e-shared-a-a148c7/auth-test-model-a148c7-kserve-59869c9747-ffsxx" Jul 14 02:43:35.216482 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:35.216317 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a587527c-d6ed-4b66-802a-40bde985384d-home\") pod \"auth-test-model-a148c7-kserve-59869c9747-ffsxx\" (UID: \"a587527c-d6ed-4b66-802a-40bde985384d\") " pod="ai-tenant-e2e-shared-a-a148c7/auth-test-model-a148c7-kserve-59869c9747-ffsxx" Jul 14 02:43:35.216482 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:35.216336 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a587527c-d6ed-4b66-802a-40bde985384d-dshm\") pod \"auth-test-model-a148c7-kserve-59869c9747-ffsxx\" (UID: \"a587527c-d6ed-4b66-802a-40bde985384d\") " pod="ai-tenant-e2e-shared-a-a148c7/auth-test-model-a148c7-kserve-59869c9747-ffsxx" Jul 14 02:43:35.216482 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:35.216385 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a587527c-d6ed-4b66-802a-40bde985384d-tls-certs\") pod \"auth-test-model-a148c7-kserve-59869c9747-ffsxx\" (UID: \"a587527c-d6ed-4b66-802a-40bde985384d\") " pod="ai-tenant-e2e-shared-a-a148c7/auth-test-model-a148c7-kserve-59869c9747-ffsxx" Jul 14 02:43:35.216482 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:35.216410 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a587527c-d6ed-4b66-802a-40bde985384d-model-cache\") pod \"auth-test-model-a148c7-kserve-59869c9747-ffsxx\" (UID: \"a587527c-d6ed-4b66-802a-40bde985384d\") " pod="ai-tenant-e2e-shared-a-a148c7/auth-test-model-a148c7-kserve-59869c9747-ffsxx" Jul 14 02:43:35.216782 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:35.216764 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a587527c-d6ed-4b66-802a-40bde985384d-tmp-dir\") pod \"auth-test-model-a148c7-kserve-59869c9747-ffsxx\" (UID: \"a587527c-d6ed-4b66-802a-40bde985384d\") " pod="ai-tenant-e2e-shared-a-a148c7/auth-test-model-a148c7-kserve-59869c9747-ffsxx" Jul 14 02:43:35.216848 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:35.216827 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a587527c-d6ed-4b66-802a-40bde985384d-home\") pod \"auth-test-model-a148c7-kserve-59869c9747-ffsxx\" (UID: \"a587527c-d6ed-4b66-802a-40bde985384d\") " pod="ai-tenant-e2e-shared-a-a148c7/auth-test-model-a148c7-kserve-59869c9747-ffsxx" Jul 14 02:43:35.216898 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:35.216856 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a587527c-d6ed-4b66-802a-40bde985384d-model-cache\") pod \"auth-test-model-a148c7-kserve-59869c9747-ffsxx\" (UID: \"a587527c-d6ed-4b66-802a-40bde985384d\") " pod="ai-tenant-e2e-shared-a-a148c7/auth-test-model-a148c7-kserve-59869c9747-ffsxx" Jul 14 02:43:35.218651 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:35.218631 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a587527c-d6ed-4b66-802a-40bde985384d-dshm\") pod \"auth-test-model-a148c7-kserve-59869c9747-ffsxx\" (UID: \"a587527c-d6ed-4b66-802a-40bde985384d\") " pod="ai-tenant-e2e-shared-a-a148c7/auth-test-model-a148c7-kserve-59869c9747-ffsxx" Jul 14 02:43:35.218915 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:35.218897 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a587527c-d6ed-4b66-802a-40bde985384d-tls-certs\") pod \"auth-test-model-a148c7-kserve-59869c9747-ffsxx\" (UID: \"a587527c-d6ed-4b66-802a-40bde985384d\") " pod="ai-tenant-e2e-shared-a-a148c7/auth-test-model-a148c7-kserve-59869c9747-ffsxx" Jul 14 02:43:35.224174 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:35.224154 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgmm9\" (UniqueName: \"kubernetes.io/projected/a587527c-d6ed-4b66-802a-40bde985384d-kube-api-access-bgmm9\") pod \"auth-test-model-a148c7-kserve-59869c9747-ffsxx\" (UID: \"a587527c-d6ed-4b66-802a-40bde985384d\") " pod="ai-tenant-e2e-shared-a-a148c7/auth-test-model-a148c7-kserve-59869c9747-ffsxx" Jul 14 02:43:35.406542 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:35.406510 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a148c7/auth-test-model-a148c7-kserve-59869c9747-ffsxx" Jul 14 02:43:35.555709 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:35.555686 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a148c7/auth-test-model-a148c7-kserve-59869c9747-ffsxx"] Jul 14 02:43:35.557573 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:43:35.557545 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda587527c_d6ed_4b66_802a_40bde985384d.slice/crio-38cd58913607f58e57b74e050e0571d4a824efc7665b6c572bd69f077c8d8c5c WatchSource:0}: Error finding container 38cd58913607f58e57b74e050e0571d4a824efc7665b6c572bd69f077c8d8c5c: Status 404 returned error can't find the container with id 38cd58913607f58e57b74e050e0571d4a824efc7665b6c572bd69f077c8d8c5c Jul 14 02:43:36.317911 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:36.317871 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a148c7/auth-test-model-a148c7-kserve-59869c9747-ffsxx" event={"ID":"a587527c-d6ed-4b66-802a-40bde985384d","Type":"ContainerStarted","Data":"fb11eb512c57eaa2cb795ad6366a7c3c55e31e9d14074f6a6d6e11f722ad9629"} Jul 14 02:43:36.317911 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:36.317916 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a148c7/auth-test-model-a148c7-kserve-59869c9747-ffsxx" event={"ID":"a587527c-d6ed-4b66-802a-40bde985384d","Type":"ContainerStarted","Data":"38cd58913607f58e57b74e050e0571d4a824efc7665b6c572bd69f077c8d8c5c"} Jul 14 02:43:36.332663 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:36.332616 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-a148c7/auth-test-model-a148c7-kserve-59869c9747-ffsxx" podStartSLOduration=1.332600127 podStartE2EDuration="1.332600127s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 02:43:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:43:36.332311196 +0000 UTC m=+2518.487147847" watchObservedRunningTime="2026-07-14 02:43:36.332600127 +0000 UTC m=+2518.487436774" Jul 14 02:43:45.406686 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:45.406655 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-a148c7/auth-test-model-a148c7-kserve-59869c9747-ffsxx" Jul 14 02:43:45.407105 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:45.406710 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-a148c7/auth-test-model-a148c7-kserve-59869c9747-ffsxx" Jul 14 02:43:45.419402 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:45.419378 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-a148c7/auth-test-model-a148c7-kserve-59869c9747-ffsxx" Jul 14 02:43:46.379280 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:43:46.379249 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-a148c7/auth-test-model-a148c7-kserve-59869c9747-ffsxx" Jul 14 02:44:03.936526 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:44:03.936489 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:44:07.913436 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:44:07.913405 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:46:08.236740 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:46:08.236701 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:46:08.289667 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:46:08.289632 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:46:09.574068 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:46:09.574030 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a148c7/auth-test-model-a148c7-kserve-59869c9747-ffsxx"] Jul 14 02:46:09.586344 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:46:09.586307 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:46:10.529621 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:46:10.529587 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:46:24.599765 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:46:24.599696 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-a148c7/auth-test-model-a148c7-kserve-59869c9747-ffsxx" podUID="a587527c-d6ed-4b66-802a-40bde985384d" containerName="main" containerID="cri-o://fb11eb512c57eaa2cb795ad6366a7c3c55e31e9d14074f6a6d6e11f722ad9629" gracePeriod=45 Jul 14 02:46:24.854673 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:46:24.854595 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a148c7/auth-test-model-a148c7-kserve-59869c9747-ffsxx" Jul 14 02:46:24.923258 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:46:24.923217 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a587527c-d6ed-4b66-802a-40bde985384d-dshm\") pod \"a587527c-d6ed-4b66-802a-40bde985384d\" (UID: \"a587527c-d6ed-4b66-802a-40bde985384d\") " Jul 14 02:46:24.923469 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:46:24.923300 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bgmm9\" (UniqueName: \"kubernetes.io/projected/a587527c-d6ed-4b66-802a-40bde985384d-kube-api-access-bgmm9\") pod \"a587527c-d6ed-4b66-802a-40bde985384d\" (UID: \"a587527c-d6ed-4b66-802a-40bde985384d\") " Jul 14 02:46:24.923469 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:46:24.923328 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a587527c-d6ed-4b66-802a-40bde985384d-model-cache\") pod \"a587527c-d6ed-4b66-802a-40bde985384d\" (UID: \"a587527c-d6ed-4b66-802a-40bde985384d\") " Jul 14 02:46:24.923469 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:46:24.923366 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a587527c-d6ed-4b66-802a-40bde985384d-tls-certs\") pod \"a587527c-d6ed-4b66-802a-40bde985384d\" (UID: \"a587527c-d6ed-4b66-802a-40bde985384d\") " Jul 14 02:46:24.923596 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:46:24.923572 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a587527c-d6ed-4b66-802a-40bde985384d-home\") pod \"a587527c-d6ed-4b66-802a-40bde985384d\" (UID: \"a587527c-d6ed-4b66-802a-40bde985384d\") " Jul 14 02:46:24.923642 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:46:24.923625 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a587527c-d6ed-4b66-802a-40bde985384d-tmp-dir\") pod \"a587527c-d6ed-4b66-802a-40bde985384d\" (UID: \"a587527c-d6ed-4b66-802a-40bde985384d\") " Jul 14 02:46:24.923710 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:46:24.923643 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a587527c-d6ed-4b66-802a-40bde985384d-model-cache" (OuterVolumeSpecName: "model-cache") pod "a587527c-d6ed-4b66-802a-40bde985384d" (UID: "a587527c-d6ed-4b66-802a-40bde985384d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:46:24.923851 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:46:24.923804 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a587527c-d6ed-4b66-802a-40bde985384d-home" (OuterVolumeSpecName: "home") pod "a587527c-d6ed-4b66-802a-40bde985384d" (UID: "a587527c-d6ed-4b66-802a-40bde985384d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:46:24.923928 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:46:24.923870 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a587527c-d6ed-4b66-802a-40bde985384d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a587527c-d6ed-4b66-802a-40bde985384d" (UID: "a587527c-d6ed-4b66-802a-40bde985384d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:46:24.923992 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:46:24.923965 2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a587527c-d6ed-4b66-802a-40bde985384d-tmp-dir\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:46:24.923992 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:46:24.923978 2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a587527c-d6ed-4b66-802a-40bde985384d-model-cache\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:46:24.924095 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:46:24.923991 2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a587527c-d6ed-4b66-802a-40bde985384d-home\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:46:24.925662 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:46:24.925637 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a587527c-d6ed-4b66-802a-40bde985384d-dshm" (OuterVolumeSpecName: "dshm") pod "a587527c-d6ed-4b66-802a-40bde985384d" (UID: "a587527c-d6ed-4b66-802a-40bde985384d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:46:24.925967 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:46:24.925944 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a587527c-d6ed-4b66-802a-40bde985384d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a587527c-d6ed-4b66-802a-40bde985384d" (UID: "a587527c-d6ed-4b66-802a-40bde985384d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 02:46:24.925967 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:46:24.925957 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a587527c-d6ed-4b66-802a-40bde985384d-kube-api-access-bgmm9" (OuterVolumeSpecName: "kube-api-access-bgmm9") pod "a587527c-d6ed-4b66-802a-40bde985384d" (UID: "a587527c-d6ed-4b66-802a-40bde985384d"). InnerVolumeSpecName "kube-api-access-bgmm9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:46:25.024747 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:46:25.024704 2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a587527c-d6ed-4b66-802a-40bde985384d-dshm\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:46:25.024747 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:46:25.024747 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgmm9\" (UniqueName: \"kubernetes.io/projected/a587527c-d6ed-4b66-802a-40bde985384d-kube-api-access-bgmm9\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:46:25.024983 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:46:25.024760 2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a587527c-d6ed-4b66-802a-40bde985384d-tls-certs\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:46:25.153531 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:46:25.153493 2577 generic.go:356] "Generic (PLEG): container finished" podID="a587527c-d6ed-4b66-802a-40bde985384d" containerID="fb11eb512c57eaa2cb795ad6366a7c3c55e31e9d14074f6a6d6e11f722ad9629" exitCode=0 Jul 14 02:46:25.153748 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:46:25.153542 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a148c7/auth-test-model-a148c7-kserve-59869c9747-ffsxx" Jul 14 02:46:25.153748 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:46:25.153539 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a148c7/auth-test-model-a148c7-kserve-59869c9747-ffsxx" event={"ID":"a587527c-d6ed-4b66-802a-40bde985384d","Type":"ContainerDied","Data":"fb11eb512c57eaa2cb795ad6366a7c3c55e31e9d14074f6a6d6e11f722ad9629"} Jul 14 02:46:25.153748 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:46:25.153639 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a148c7/auth-test-model-a148c7-kserve-59869c9747-ffsxx" event={"ID":"a587527c-d6ed-4b66-802a-40bde985384d","Type":"ContainerDied","Data":"38cd58913607f58e57b74e050e0571d4a824efc7665b6c572bd69f077c8d8c5c"} Jul 14 02:46:25.153748 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:46:25.153660 2577 scope.go:117] "RemoveContainer" containerID="fb11eb512c57eaa2cb795ad6366a7c3c55e31e9d14074f6a6d6e11f722ad9629" Jul 14 02:46:25.163090 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:46:25.163069 2577 scope.go:117] "RemoveContainer" containerID="fb11eb512c57eaa2cb795ad6366a7c3c55e31e9d14074f6a6d6e11f722ad9629" Jul 14 02:46:25.163383 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:46:25.163361 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb11eb512c57eaa2cb795ad6366a7c3c55e31e9d14074f6a6d6e11f722ad9629\": container with ID starting with fb11eb512c57eaa2cb795ad6366a7c3c55e31e9d14074f6a6d6e11f722ad9629 not found: ID does not exist" containerID="fb11eb512c57eaa2cb795ad6366a7c3c55e31e9d14074f6a6d6e11f722ad9629" Jul 14 02:46:25.163469 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:46:25.163395 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb11eb512c57eaa2cb795ad6366a7c3c55e31e9d14074f6a6d6e11f722ad9629"} err="failed to get container status \"fb11eb512c57eaa2cb795ad6366a7c3c55e31e9d14074f6a6d6e11f722ad9629\": rpc error: code = NotFound desc = could not find container \"fb11eb512c57eaa2cb795ad6366a7c3c55e31e9d14074f6a6d6e11f722ad9629\": container with ID starting with fb11eb512c57eaa2cb795ad6366a7c3c55e31e9d14074f6a6d6e11f722ad9629 not found: ID does not exist" Jul 14 02:46:25.173114 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:46:25.173086 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a148c7/auth-test-model-a148c7-kserve-59869c9747-ffsxx"] Jul 14 02:46:25.174983 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:46:25.174963 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-a148c7/auth-test-model-a148c7-kserve-59869c9747-ffsxx"] Jul 14 02:46:26.265536 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:46:26.265499 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a587527c-d6ed-4b66-802a-40bde985384d" path="/var/lib/kubelet/pods/a587527c-d6ed-4b66-802a-40bde985384d/volumes" Jul 14 02:46:53.920489 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:46:53.920454 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:47:03.195824 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:47:03.195766 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:47:03.296667 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:47:03.296634 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:47:15.706690 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:47:15.706649 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:47:22.897621 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:47:22.897591 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:47:22.945873 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:47:22.945844 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:47:28.222626 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:47:28.222596 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:47:51.955340 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:47:51.955309 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:47:52.993265 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:47:52.993213 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:48:21.115055 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:48:21.115008 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:48:24.038355 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:48:24.038317 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:48:24.136074 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:48:24.136027 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:48:29.629312 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:48:29.629252 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:48:29.730083 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:48:29.730046 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:48:33.179266 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:48:33.179230 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:48:33.277369 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:48:33.277340 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:48:39.482475 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:48:39.482437 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:49:22.400974 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:49:22.400934 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:49:23.774667 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:49:23.774632 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:49:43.387642 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:49:43.387599 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:49:43.487990 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:49:43.487910 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:49:53.185754 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:49:53.185713 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:49:57.883806 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:49:57.883750 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:50:03.507522 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:03.507484 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:50:03.607921 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:03.607885 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:50:08.545105 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:08.545057 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg"] Jul 14 02:50:08.545636 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:08.545419 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" podUID="e8fb4874-8a24-4377-9707-589577b5e290" containerName="istio-proxy" containerID="cri-o://e99dea84b256cd15a7a49c87f178219c8473dc742d7d2c8a76c86b36966d4d1c" gracePeriod=30 Jul 14 02:50:08.558628 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:08.558589 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:50:08.791788 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:08.791752 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:50:09.580757 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:09.580722 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:50:09.679391 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:09.679355 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:50:13.816014 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:13.815947 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:50:13.913715 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:13.913678 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8fb4874-8a24-4377-9707-589577b5e290-credential-socket\") pod \"e8fb4874-8a24-4377-9707-589577b5e290\" (UID: \"e8fb4874-8a24-4377-9707-589577b5e290\") " Jul 14 02:50:13.913715 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:13.913712 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8fb4874-8a24-4377-9707-589577b5e290-istio-envoy\") pod \"e8fb4874-8a24-4377-9707-589577b5e290\" (UID: \"e8fb4874-8a24-4377-9707-589577b5e290\") " Jul 14 02:50:13.913955 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:13.913750 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8fb4874-8a24-4377-9707-589577b5e290-istio-podinfo\") pod \"e8fb4874-8a24-4377-9707-589577b5e290\" (UID: \"e8fb4874-8a24-4377-9707-589577b5e290\") " Jul 14 02:50:13.913955 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:13.913805 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8fb4874-8a24-4377-9707-589577b5e290-workload-certs\") pod \"e8fb4874-8a24-4377-9707-589577b5e290\" (UID: \"e8fb4874-8a24-4377-9707-589577b5e290\") " Jul 14 02:50:13.913955 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:13.913828 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8fb4874-8a24-4377-9707-589577b5e290-istiod-ca-cert\") pod \"e8fb4874-8a24-4377-9707-589577b5e290\" (UID: \"e8fb4874-8a24-4377-9707-589577b5e290\") " Jul 14 02:50:13.913955 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:13.913853 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8fb4874-8a24-4377-9707-589577b5e290-istio-token\") pod \"e8fb4874-8a24-4377-9707-589577b5e290\" (UID: \"e8fb4874-8a24-4377-9707-589577b5e290\") " Jul 14 02:50:13.913955 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:13.913895 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gn8fm\" (UniqueName: \"kubernetes.io/projected/e8fb4874-8a24-4377-9707-589577b5e290-kube-api-access-gn8fm\") pod \"e8fb4874-8a24-4377-9707-589577b5e290\" (UID: \"e8fb4874-8a24-4377-9707-589577b5e290\") " Jul 14 02:50:13.913955 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:13.913946 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8fb4874-8a24-4377-9707-589577b5e290-workload-socket\") pod \"e8fb4874-8a24-4377-9707-589577b5e290\" (UID: \"e8fb4874-8a24-4377-9707-589577b5e290\") " Jul 14 02:50:13.914374 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:13.914017 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8fb4874-8a24-4377-9707-589577b5e290-istio-data\") pod \"e8fb4874-8a24-4377-9707-589577b5e290\" (UID: \"e8fb4874-8a24-4377-9707-589577b5e290\") " Jul 14 02:50:13.914374 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:13.914013 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8fb4874-8a24-4377-9707-589577b5e290-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e8fb4874-8a24-4377-9707-589577b5e290" (UID: "e8fb4874-8a24-4377-9707-589577b5e290"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:50:13.914374 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:13.914199 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8fb4874-8a24-4377-9707-589577b5e290-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e8fb4874-8a24-4377-9707-589577b5e290" (UID: "e8fb4874-8a24-4377-9707-589577b5e290"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:50:13.914374 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:13.914247 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8fb4874-8a24-4377-9707-589577b5e290-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e8fb4874-8a24-4377-9707-589577b5e290" (UID: "e8fb4874-8a24-4377-9707-589577b5e290"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:50:13.914542 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:13.914474 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8fb4874-8a24-4377-9707-589577b5e290-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e8fb4874-8a24-4377-9707-589577b5e290" (UID: "e8fb4874-8a24-4377-9707-589577b5e290"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 02:50:13.915018 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:13.914995 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8fb4874-8a24-4377-9707-589577b5e290-istio-data" (OuterVolumeSpecName: "istio-data") pod "e8fb4874-8a24-4377-9707-589577b5e290" (UID: "e8fb4874-8a24-4377-9707-589577b5e290"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:50:13.916258 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:13.916231 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e8fb4874-8a24-4377-9707-589577b5e290-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e8fb4874-8a24-4377-9707-589577b5e290" (UID: "e8fb4874-8a24-4377-9707-589577b5e290"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 02:50:13.916258 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:13.916246 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8fb4874-8a24-4377-9707-589577b5e290-istio-token" (OuterVolumeSpecName: "istio-token") pod "e8fb4874-8a24-4377-9707-589577b5e290" (UID: "e8fb4874-8a24-4377-9707-589577b5e290"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:50:13.916548 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:13.916518 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8fb4874-8a24-4377-9707-589577b5e290-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e8fb4874-8a24-4377-9707-589577b5e290" (UID: "e8fb4874-8a24-4377-9707-589577b5e290"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:50:13.916616 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:13.916528 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8fb4874-8a24-4377-9707-589577b5e290-kube-api-access-gn8fm" (OuterVolumeSpecName: "kube-api-access-gn8fm") pod "e8fb4874-8a24-4377-9707-589577b5e290" (UID: "e8fb4874-8a24-4377-9707-589577b5e290"). InnerVolumeSpecName "kube-api-access-gn8fm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:50:14.015069 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:14.015024 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8fb4874-8a24-4377-9707-589577b5e290-istio-data\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:50:14.015069 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:14.015055 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8fb4874-8a24-4377-9707-589577b5e290-credential-socket\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:50:14.015069 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:14.015065 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8fb4874-8a24-4377-9707-589577b5e290-istio-envoy\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:50:14.015069 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:14.015074 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8fb4874-8a24-4377-9707-589577b5e290-istio-podinfo\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:50:14.015069 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:14.015082 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8fb4874-8a24-4377-9707-589577b5e290-workload-certs\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:50:14.015406 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:14.015091 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8fb4874-8a24-4377-9707-589577b5e290-istiod-ca-cert\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:50:14.015406 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:14.015098 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8fb4874-8a24-4377-9707-589577b5e290-istio-token\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:50:14.015406 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:14.015105 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gn8fm\" (UniqueName: \"kubernetes.io/projected/e8fb4874-8a24-4377-9707-589577b5e290-kube-api-access-gn8fm\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:50:14.015406 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:14.015116 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8fb4874-8a24-4377-9707-589577b5e290-workload-socket\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:50:14.295456 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:14.295419 2577 generic.go:356] "Generic (PLEG): container finished" podID="e8fb4874-8a24-4377-9707-589577b5e290" containerID="e99dea84b256cd15a7a49c87f178219c8473dc742d7d2c8a76c86b36966d4d1c" exitCode=0 Jul 14 02:50:14.295661 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:14.295480 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" event={"ID":"e8fb4874-8a24-4377-9707-589577b5e290","Type":"ContainerDied","Data":"e99dea84b256cd15a7a49c87f178219c8473dc742d7d2c8a76c86b36966d4d1c"} Jul 14 02:50:14.295661 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:14.295486 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" Jul 14 02:50:14.295661 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:14.295526 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg" event={"ID":"e8fb4874-8a24-4377-9707-589577b5e290","Type":"ContainerDied","Data":"dce622d782388ea4bbbd1d7f6181ceda4d34065323a36f37830671a9cdac8f5c"} Jul 14 02:50:14.295661 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:14.295548 2577 scope.go:117] "RemoveContainer" containerID="e99dea84b256cd15a7a49c87f178219c8473dc742d7d2c8a76c86b36966d4d1c" Jul 14 02:50:14.304344 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:14.304323 2577 scope.go:117] "RemoveContainer" containerID="2e67c5b66203aea5b4d249138499140ecc228c959849624c5b6ff34434ba50b6" Jul 14 02:50:14.311450 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:14.311418 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg"] Jul 14 02:50:14.313692 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:14.313667 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-a148c7-openshift-default-796d78fc8c-t7nkg"] Jul 14 02:50:14.314129 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:14.314113 2577 scope.go:117] "RemoveContainer" containerID="e99dea84b256cd15a7a49c87f178219c8473dc742d7d2c8a76c86b36966d4d1c" Jul 14 02:50:14.314490 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:50:14.314473 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e99dea84b256cd15a7a49c87f178219c8473dc742d7d2c8a76c86b36966d4d1c\": container with ID starting with e99dea84b256cd15a7a49c87f178219c8473dc742d7d2c8a76c86b36966d4d1c not found: ID does not exist" containerID="e99dea84b256cd15a7a49c87f178219c8473dc742d7d2c8a76c86b36966d4d1c" Jul 14 02:50:14.314574 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:14.314497 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e99dea84b256cd15a7a49c87f178219c8473dc742d7d2c8a76c86b36966d4d1c"} err="failed to get container status \"e99dea84b256cd15a7a49c87f178219c8473dc742d7d2c8a76c86b36966d4d1c\": rpc error: code = NotFound desc = could not find container \"e99dea84b256cd15a7a49c87f178219c8473dc742d7d2c8a76c86b36966d4d1c\": container with ID starting with e99dea84b256cd15a7a49c87f178219c8473dc742d7d2c8a76c86b36966d4d1c not found: ID does not exist" Jul 14 02:50:14.314574 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:14.314518 2577 scope.go:117] "RemoveContainer" containerID="2e67c5b66203aea5b4d249138499140ecc228c959849624c5b6ff34434ba50b6" Jul 14 02:50:14.314785 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:50:14.314765 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e67c5b66203aea5b4d249138499140ecc228c959849624c5b6ff34434ba50b6\": container with ID starting with 2e67c5b66203aea5b4d249138499140ecc228c959849624c5b6ff34434ba50b6 not found: ID does not exist" containerID="2e67c5b66203aea5b4d249138499140ecc228c959849624c5b6ff34434ba50b6" Jul 14 02:50:14.314838 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:14.314793 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e67c5b66203aea5b4d249138499140ecc228c959849624c5b6ff34434ba50b6"} err="failed to get container status \"2e67c5b66203aea5b4d249138499140ecc228c959849624c5b6ff34434ba50b6\": rpc error: code = NotFound desc = could not find container \"2e67c5b66203aea5b4d249138499140ecc228c959849624c5b6ff34434ba50b6\": container with ID starting with 2e67c5b66203aea5b4d249138499140ecc228c959849624c5b6ff34434ba50b6 not found: ID does not exist" Jul 14 02:50:16.265510 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:16.265457 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8fb4874-8a24-4377-9707-589577b5e290" path="/var/lib/kubelet/pods/e8fb4874-8a24-4377-9707-589577b5e290/volumes" Jul 14 02:50:19.251847 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:19.251808 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:50:23.982287 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:23.982240 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:50:29.809564 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:29.809525 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:50:29.881112 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:29.881079 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:50:34.663641 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:34.663607 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft"] Jul 14 02:50:34.664071 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:34.663838 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" podUID="163d1a57-7acb-4502-868f-dc4518a5d995" containerName="istio-proxy" containerID="cri-o://66eb3181cba5790a8e8b8849ac8c0487bafdb03a9c805a71c92050fe4a0ac343" gracePeriod=30 Jul 14 02:50:34.682193 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:34.682159 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:50:34.907820 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:34.907779 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:50:39.923051 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:39.922990 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:50:39.961667 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:39.961639 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/163d1a57-7acb-4502-868f-dc4518a5d995-workload-socket\") pod \"163d1a57-7acb-4502-868f-dc4518a5d995\" (UID: \"163d1a57-7acb-4502-868f-dc4518a5d995\") " Jul 14 02:50:39.961849 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:39.961705 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5t5mq\" (UniqueName: \"kubernetes.io/projected/163d1a57-7acb-4502-868f-dc4518a5d995-kube-api-access-5t5mq\") pod \"163d1a57-7acb-4502-868f-dc4518a5d995\" (UID: \"163d1a57-7acb-4502-868f-dc4518a5d995\") " Jul 14 02:50:39.961849 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:39.961762 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/163d1a57-7acb-4502-868f-dc4518a5d995-workload-certs\") pod \"163d1a57-7acb-4502-868f-dc4518a5d995\" (UID: \"163d1a57-7acb-4502-868f-dc4518a5d995\") " Jul 14 02:50:39.961849 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:39.961803 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/163d1a57-7acb-4502-868f-dc4518a5d995-istiod-ca-cert\") pod \"163d1a57-7acb-4502-868f-dc4518a5d995\" (UID: \"163d1a57-7acb-4502-868f-dc4518a5d995\") " Jul 14 02:50:39.961849 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:39.961832 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/163d1a57-7acb-4502-868f-dc4518a5d995-istio-token\") pod \"163d1a57-7acb-4502-868f-dc4518a5d995\" (UID: \"163d1a57-7acb-4502-868f-dc4518a5d995\") " Jul 14 02:50:39.962081 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:39.961861 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/163d1a57-7acb-4502-868f-dc4518a5d995-istio-envoy\") pod \"163d1a57-7acb-4502-868f-dc4518a5d995\" (UID: \"163d1a57-7acb-4502-868f-dc4518a5d995\") " Jul 14 02:50:39.962081 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:39.961911 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/163d1a57-7acb-4502-868f-dc4518a5d995-credential-socket\") pod \"163d1a57-7acb-4502-868f-dc4518a5d995\" (UID: \"163d1a57-7acb-4502-868f-dc4518a5d995\") " Jul 14 02:50:39.962081 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:39.961948 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/163d1a57-7acb-4502-868f-dc4518a5d995-istio-data\") pod \"163d1a57-7acb-4502-868f-dc4518a5d995\" (UID: \"163d1a57-7acb-4502-868f-dc4518a5d995\") " Jul 14 02:50:39.962081 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:39.962004 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/163d1a57-7acb-4502-868f-dc4518a5d995-istio-podinfo\") pod \"163d1a57-7acb-4502-868f-dc4518a5d995\" (UID: \"163d1a57-7acb-4502-868f-dc4518a5d995\") " Jul 14 02:50:39.962285 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:39.961944 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/163d1a57-7acb-4502-868f-dc4518a5d995-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "163d1a57-7acb-4502-868f-dc4518a5d995" (UID: "163d1a57-7acb-4502-868f-dc4518a5d995"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:50:39.962285 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:39.962099 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/163d1a57-7acb-4502-868f-dc4518a5d995-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "163d1a57-7acb-4502-868f-dc4518a5d995" (UID: "163d1a57-7acb-4502-868f-dc4518a5d995"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:50:39.962399 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:39.962361 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/163d1a57-7acb-4502-868f-dc4518a5d995-workload-socket\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:50:39.962399 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:39.962362 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/163d1a57-7acb-4502-868f-dc4518a5d995-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "163d1a57-7acb-4502-868f-dc4518a5d995" (UID: "163d1a57-7acb-4502-868f-dc4518a5d995"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:50:39.962399 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:39.962380 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/163d1a57-7acb-4502-868f-dc4518a5d995-workload-certs\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:50:39.962502 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:39.962466 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/163d1a57-7acb-4502-868f-dc4518a5d995-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "163d1a57-7acb-4502-868f-dc4518a5d995" (UID: "163d1a57-7acb-4502-868f-dc4518a5d995"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 02:50:39.962765 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:39.962734 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/163d1a57-7acb-4502-868f-dc4518a5d995-istio-data" (OuterVolumeSpecName: "istio-data") pod "163d1a57-7acb-4502-868f-dc4518a5d995" (UID: "163d1a57-7acb-4502-868f-dc4518a5d995"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:50:39.964106 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:39.964075 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/163d1a57-7acb-4502-868f-dc4518a5d995-kube-api-access-5t5mq" (OuterVolumeSpecName: "kube-api-access-5t5mq") pod "163d1a57-7acb-4502-868f-dc4518a5d995" (UID: "163d1a57-7acb-4502-868f-dc4518a5d995"). InnerVolumeSpecName "kube-api-access-5t5mq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:50:39.964106 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:39.964088 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/163d1a57-7acb-4502-868f-dc4518a5d995-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "163d1a57-7acb-4502-868f-dc4518a5d995" (UID: "163d1a57-7acb-4502-868f-dc4518a5d995"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 02:50:39.964575 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:39.964558 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/163d1a57-7acb-4502-868f-dc4518a5d995-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "163d1a57-7acb-4502-868f-dc4518a5d995" (UID: "163d1a57-7acb-4502-868f-dc4518a5d995"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 02:50:39.964654 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:39.964575 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/163d1a57-7acb-4502-868f-dc4518a5d995-istio-token" (OuterVolumeSpecName: "istio-token") pod "163d1a57-7acb-4502-868f-dc4518a5d995" (UID: "163d1a57-7acb-4502-868f-dc4518a5d995"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 02:50:40.062896 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:40.062848 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/163d1a57-7acb-4502-868f-dc4518a5d995-istiod-ca-cert\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:50:40.062896 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:40.062887 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/163d1a57-7acb-4502-868f-dc4518a5d995-istio-token\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:50:40.062896 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:40.062899 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/163d1a57-7acb-4502-868f-dc4518a5d995-istio-envoy\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:50:40.063155 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:40.062910 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/163d1a57-7acb-4502-868f-dc4518a5d995-credential-socket\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:50:40.063155 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:40.062923 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/163d1a57-7acb-4502-868f-dc4518a5d995-istio-data\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:50:40.063155 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:40.062933 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/163d1a57-7acb-4502-868f-dc4518a5d995-istio-podinfo\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:50:40.063155 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:40.062944 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5t5mq\" (UniqueName: \"kubernetes.io/projected/163d1a57-7acb-4502-868f-dc4518a5d995-kube-api-access-5t5mq\") on node \"ip-10-0-131-25.ec2.internal\" DevicePath \"\"" Jul 14 02:50:40.150515 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:40.150483 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-k4497_44a4b3f6-f9fb-40d2-ad21-7e63a6a34269/manager/0.log" Jul 14 02:50:40.270749 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:40.270677 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-zf54c_bc75f235-9d07-4c00-8e8e-59043d8f108e/manager/0.log" Jul 14 02:50:40.433951 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:40.433914 2577 generic.go:356] "Generic (PLEG): container finished" podID="163d1a57-7acb-4502-868f-dc4518a5d995" containerID="66eb3181cba5790a8e8b8849ac8c0487bafdb03a9c805a71c92050fe4a0ac343" exitCode=0 Jul 14 02:50:40.434116 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:40.433957 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" event={"ID":"163d1a57-7acb-4502-868f-dc4518a5d995","Type":"ContainerDied","Data":"66eb3181cba5790a8e8b8849ac8c0487bafdb03a9c805a71c92050fe4a0ac343"} Jul 14 02:50:40.434116 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:40.433974 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" Jul 14 02:50:40.434116 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:40.434024 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft" event={"ID":"163d1a57-7acb-4502-868f-dc4518a5d995","Type":"ContainerDied","Data":"ecbbcda3801fef388ef17ea3617ff1152c9b620cd786f3fc701174082af994d4"} Jul 14 02:50:40.434116 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:40.434050 2577 scope.go:117] "RemoveContainer" containerID="66eb3181cba5790a8e8b8849ac8c0487bafdb03a9c805a71c92050fe4a0ac343" Jul 14 02:50:40.442647 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:40.442627 2577 scope.go:117] "RemoveContainer" containerID="14a7859301f75abf963207cd62fe2eaae2a34c3d93e19381818fd04c894bdb58" Jul 14 02:50:40.449227 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:40.449202 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft"] Jul 14 02:50:40.451186 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:40.451171 2577 scope.go:117] "RemoveContainer" containerID="66eb3181cba5790a8e8b8849ac8c0487bafdb03a9c805a71c92050fe4a0ac343" Jul 14 02:50:40.451442 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:50:40.451421 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66eb3181cba5790a8e8b8849ac8c0487bafdb03a9c805a71c92050fe4a0ac343\": container with ID starting with 66eb3181cba5790a8e8b8849ac8c0487bafdb03a9c805a71c92050fe4a0ac343 not found: ID does not exist" containerID="66eb3181cba5790a8e8b8849ac8c0487bafdb03a9c805a71c92050fe4a0ac343" Jul 14 02:50:40.451503 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:40.451450 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66eb3181cba5790a8e8b8849ac8c0487bafdb03a9c805a71c92050fe4a0ac343"} err="failed to get container status \"66eb3181cba5790a8e8b8849ac8c0487bafdb03a9c805a71c92050fe4a0ac343\": rpc error: code = NotFound desc = could not find container \"66eb3181cba5790a8e8b8849ac8c0487bafdb03a9c805a71c92050fe4a0ac343\": container with ID starting with 66eb3181cba5790a8e8b8849ac8c0487bafdb03a9c805a71c92050fe4a0ac343 not found: ID does not exist" Jul 14 02:50:40.451503 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:40.451468 2577 scope.go:117] "RemoveContainer" containerID="14a7859301f75abf963207cd62fe2eaae2a34c3d93e19381818fd04c894bdb58" Jul 14 02:50:40.451688 ip-10-0-131-25 kubenswrapper[2577]: E0714 02:50:40.451670 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14a7859301f75abf963207cd62fe2eaae2a34c3d93e19381818fd04c894bdb58\": container with ID starting with 14a7859301f75abf963207cd62fe2eaae2a34c3d93e19381818fd04c894bdb58 not found: ID does not exist" containerID="14a7859301f75abf963207cd62fe2eaae2a34c3d93e19381818fd04c894bdb58" Jul 14 02:50:40.451753 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:40.451693 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14a7859301f75abf963207cd62fe2eaae2a34c3d93e19381818fd04c894bdb58"} err="failed to get container status \"14a7859301f75abf963207cd62fe2eaae2a34c3d93e19381818fd04c894bdb58\": rpc error: code = NotFound desc = could not find container \"14a7859301f75abf963207cd62fe2eaae2a34c3d93e19381818fd04c894bdb58\": container with ID starting with 14a7859301f75abf963207cd62fe2eaae2a34c3d93e19381818fd04c894bdb58 not found: ID does not exist" Jul 14 02:50:40.454144 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:40.454123 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-c47335-openshift-default-554f6fb775-kxwft"] Jul 14 02:50:40.751781 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:40.751751 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5756559977-b9bqv_1d26deb9-d556-46a9-8c88-43fb9390a06b/copy-manifests/0.log" Jul 14 02:50:40.758856 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:40.758802 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5756559977-b9bqv_1d26deb9-d556-46a9-8c88-43fb9390a06b/manager/0.log" Jul 14 02:50:42.264700 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:42.264671 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="163d1a57-7acb-4502-868f-dc4518a5d995" path="/var/lib/kubelet/pods/163d1a57-7acb-4502-868f-dc4518a5d995/volumes" Jul 14 02:50:44.627997 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:44.627967 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-gkhww_d823fbe9-7068-4f4d-af9d-8fa935a9f485/manager/0.log" Jul 14 02:50:44.979749 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:44.979672 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-9t4xc_98eaba39-c2ba-4cae-87fb-74089ac777d7/manager/0.log" Jul 14 02:50:45.561399 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:45.561371 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf447zmt_83887e08-ddc3-4ae6-a9cd-126107e2dde0/istio-proxy/0.log" Jul 14 02:50:46.027072 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:46.027044 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-fdsj7_8a993e36-54b0-48f9-876d-170e53d0794a/istio-proxy/1.log" Jul 14 02:50:46.374536 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:46.374513 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-8f44474b6-jrxf5_998e452c-dc1f-4e3d-862e-7365d5a216e7/router/0.log" Jul 14 02:50:46.838123 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:46.838043 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-5fgjt_f365efbc-4564-439a-80a3-e4a30903bce1/main/0.log" Jul 14 02:50:46.947379 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:46.947353 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-2vvsp_33de58ad-f50a-45c9-9475-661288f6233d/main/0.log" Jul 14 02:50:47.177896 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:47.177870 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-6vh2d_2c950ba9-5f49-4d76-9608-b9c16356f4a7/main/0.log" Jul 14 02:50:51.805525 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:51.805496 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:50:51.904775 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:51.904741 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" limit=1024 Jul 14 02:50:54.624215 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:54.624180 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wmd5q_45d07b54-1b6f-4f51-bf7c-cc8d287e5789/global-pull-secret-syncer/0.log" Jul 14 02:50:54.722059 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:54.722029 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-bs5bn_f361ee09-a428-4b20-93c3-b9762d14f012/konnectivity-agent/0.log" Jul 14 02:50:54.838134 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:54.838105 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-25.ec2.internal_68e660ef911ade3343c0551cb4b95404/haproxy/0.log" Jul 14 02:50:59.790827 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:59.790742 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-gkhww_d823fbe9-7068-4f4d-af9d-8fa935a9f485/manager/0.log" Jul 14 02:50:59.994944 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:50:59.994909 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-9t4xc_98eaba39-c2ba-4cae-87fb-74089ac777d7/manager/0.log" Jul 14 02:51:01.917415 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:01.917385 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-sfkpv_318d3e98-edc4-40e9-88fb-6f1558d193ad/kube-state-metrics/0.log" Jul 14 02:51:01.932804 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:01.932776 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-sfkpv_318d3e98-edc4-40e9-88fb-6f1558d193ad/kube-rbac-proxy-main/0.log" Jul 14 02:51:01.952127 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:01.952095 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-sfkpv_318d3e98-edc4-40e9-88fb-6f1558d193ad/kube-rbac-proxy-self/0.log" Jul 14 02:51:02.154257 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:02.154229 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r2p5h_d0fd1ddf-e24c-4639-a99f-28cc076262e5/node-exporter/0.log" Jul 14 02:51:02.173700 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:02.173675 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r2p5h_d0fd1ddf-e24c-4639-a99f-28cc076262e5/kube-rbac-proxy/0.log" Jul 14 02:51:02.190838 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:02.190812 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r2p5h_d0fd1ddf-e24c-4639-a99f-28cc076262e5/init-textfile/0.log" Jul 14 02:51:02.304031 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:02.304002 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5ee55730-71d9-410e-9d34-6858e719419f/prometheus/0.log" Jul 14 02:51:02.320894 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:02.320871 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5ee55730-71d9-410e-9d34-6858e719419f/config-reloader/0.log" Jul 14 02:51:02.339831 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:02.339807 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5ee55730-71d9-410e-9d34-6858e719419f/thanos-sidecar/0.log" Jul 14 02:51:02.362210 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:02.362182 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5ee55730-71d9-410e-9d34-6858e719419f/kube-rbac-proxy-web/0.log" Jul 14 02:51:02.381281 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:02.381254 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5ee55730-71d9-410e-9d34-6858e719419f/kube-rbac-proxy/0.log" Jul 14 02:51:02.400787 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:02.400755 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5ee55730-71d9-410e-9d34-6858e719419f/kube-rbac-proxy-thanos/0.log" Jul 14 02:51:02.419650 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:02.419625 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5ee55730-71d9-410e-9d34-6858e719419f/init-config-reloader/0.log" Jul 14 02:51:02.497069 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:02.497040 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-fkbzn_bf9b9caa-ed64-44ce-b36f-4cab77cf304d/prometheus-operator-admission-webhook/0.log" Jul 14 02:51:06.019035 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.018992 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-btqq2/perf-node-gather-daemonset-fl5dc"] Jul 14 02:51:06.019559 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.019537 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8fb4874-8a24-4377-9707-589577b5e290" containerName="istio-proxy" Jul 14 02:51:06.019642 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.019563 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8fb4874-8a24-4377-9707-589577b5e290" containerName="istio-proxy" Jul 14 02:51:06.019642 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.019579 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="163d1a57-7acb-4502-868f-dc4518a5d995" containerName="istio-proxy" Jul 14 02:51:06.019642 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.019589 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="163d1a57-7acb-4502-868f-dc4518a5d995" containerName="istio-proxy" Jul 14 02:51:06.019642 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.019599 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8fb4874-8a24-4377-9707-589577b5e290" containerName="istio-proxy" Jul 14 02:51:06.019642 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.019608 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8fb4874-8a24-4377-9707-589577b5e290" containerName="istio-proxy" Jul 14 02:51:06.019642 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.019618 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="163d1a57-7acb-4502-868f-dc4518a5d995" containerName="istio-proxy" Jul 14 02:51:06.019642 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.019626 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="163d1a57-7acb-4502-868f-dc4518a5d995" containerName="istio-proxy" Jul 14 02:51:06.020106 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.019672 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a587527c-d6ed-4b66-802a-40bde985384d" containerName="main" Jul 14 02:51:06.020106 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.019681 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a587527c-d6ed-4b66-802a-40bde985384d" containerName="main" Jul 14 02:51:06.020106 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.019816 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="163d1a57-7acb-4502-868f-dc4518a5d995" containerName="istio-proxy" Jul 14 02:51:06.020106 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.019829 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8fb4874-8a24-4377-9707-589577b5e290" containerName="istio-proxy" Jul 14 02:51:06.020106 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.019839 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8fb4874-8a24-4377-9707-589577b5e290" containerName="istio-proxy" Jul 14 02:51:06.020106 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.019848 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a587527c-d6ed-4b66-802a-40bde985384d" containerName="main" Jul 14 02:51:06.023243 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.023220 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-btqq2/perf-node-gather-daemonset-fl5dc" Jul 14 02:51:06.025261 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.025243 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-btqq2\"/\"openshift-service-ca.crt\"" Jul 14 02:51:06.025847 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.025835 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-btqq2\"/\"kube-root-ca.crt\"" Jul 14 02:51:06.025897 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.025877 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-btqq2\"/\"default-dockercfg-9lvvj\"" Jul 14 02:51:06.029157 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.029131 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-btqq2/perf-node-gather-daemonset-fl5dc"] Jul 14 02:51:06.110608 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.110571 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8958a66f-256b-41f1-9afe-7c8ecdbeda2f-podres\") pod \"perf-node-gather-daemonset-fl5dc\" (UID: \"8958a66f-256b-41f1-9afe-7c8ecdbeda2f\") " pod="openshift-must-gather-btqq2/perf-node-gather-daemonset-fl5dc" Jul 14 02:51:06.110816 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.110632 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pz5v2\" (UniqueName: \"kubernetes.io/projected/8958a66f-256b-41f1-9afe-7c8ecdbeda2f-kube-api-access-pz5v2\") pod \"perf-node-gather-daemonset-fl5dc\" (UID: \"8958a66f-256b-41f1-9afe-7c8ecdbeda2f\") " pod="openshift-must-gather-btqq2/perf-node-gather-daemonset-fl5dc" Jul 14 02:51:06.110816 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.110679 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8958a66f-256b-41f1-9afe-7c8ecdbeda2f-sys\") pod \"perf-node-gather-daemonset-fl5dc\" (UID: \"8958a66f-256b-41f1-9afe-7c8ecdbeda2f\") " pod="openshift-must-gather-btqq2/perf-node-gather-daemonset-fl5dc" Jul 14 02:51:06.110816 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.110709 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8958a66f-256b-41f1-9afe-7c8ecdbeda2f-lib-modules\") pod \"perf-node-gather-daemonset-fl5dc\" (UID: \"8958a66f-256b-41f1-9afe-7c8ecdbeda2f\") " pod="openshift-must-gather-btqq2/perf-node-gather-daemonset-fl5dc" Jul 14 02:51:06.110816 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.110791 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8958a66f-256b-41f1-9afe-7c8ecdbeda2f-proc\") pod \"perf-node-gather-daemonset-fl5dc\" (UID: \"8958a66f-256b-41f1-9afe-7c8ecdbeda2f\") " pod="openshift-must-gather-btqq2/perf-node-gather-daemonset-fl5dc" Jul 14 02:51:06.211600 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.211560 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8958a66f-256b-41f1-9afe-7c8ecdbeda2f-lib-modules\") pod \"perf-node-gather-daemonset-fl5dc\" (UID: \"8958a66f-256b-41f1-9afe-7c8ecdbeda2f\") " pod="openshift-must-gather-btqq2/perf-node-gather-daemonset-fl5dc" Jul 14 02:51:06.211784 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.211618 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8958a66f-256b-41f1-9afe-7c8ecdbeda2f-proc\") pod \"perf-node-gather-daemonset-fl5dc\" (UID: \"8958a66f-256b-41f1-9afe-7c8ecdbeda2f\") " pod="openshift-must-gather-btqq2/perf-node-gather-daemonset-fl5dc" Jul 14 02:51:06.211784 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.211682 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8958a66f-256b-41f1-9afe-7c8ecdbeda2f-podres\") pod \"perf-node-gather-daemonset-fl5dc\" (UID: \"8958a66f-256b-41f1-9afe-7c8ecdbeda2f\") " pod="openshift-must-gather-btqq2/perf-node-gather-daemonset-fl5dc" Jul 14 02:51:06.211784 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.211685 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8958a66f-256b-41f1-9afe-7c8ecdbeda2f-proc\") pod \"perf-node-gather-daemonset-fl5dc\" (UID: \"8958a66f-256b-41f1-9afe-7c8ecdbeda2f\") " pod="openshift-must-gather-btqq2/perf-node-gather-daemonset-fl5dc" Jul 14 02:51:06.211784 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.211737 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8958a66f-256b-41f1-9afe-7c8ecdbeda2f-lib-modules\") pod \"perf-node-gather-daemonset-fl5dc\" (UID: \"8958a66f-256b-41f1-9afe-7c8ecdbeda2f\") " pod="openshift-must-gather-btqq2/perf-node-gather-daemonset-fl5dc" Jul 14 02:51:06.211784 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.211757 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz5v2\" (UniqueName: \"kubernetes.io/projected/8958a66f-256b-41f1-9afe-7c8ecdbeda2f-kube-api-access-pz5v2\") pod \"perf-node-gather-daemonset-fl5dc\" (UID: \"8958a66f-256b-41f1-9afe-7c8ecdbeda2f\") " pod="openshift-must-gather-btqq2/perf-node-gather-daemonset-fl5dc" Jul 14 02:51:06.211981 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.211792 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8958a66f-256b-41f1-9afe-7c8ecdbeda2f-sys\") pod \"perf-node-gather-daemonset-fl5dc\" (UID: \"8958a66f-256b-41f1-9afe-7c8ecdbeda2f\") " pod="openshift-must-gather-btqq2/perf-node-gather-daemonset-fl5dc" Jul 14 02:51:06.211981 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.211789 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8958a66f-256b-41f1-9afe-7c8ecdbeda2f-podres\") pod \"perf-node-gather-daemonset-fl5dc\" (UID: \"8958a66f-256b-41f1-9afe-7c8ecdbeda2f\") " pod="openshift-must-gather-btqq2/perf-node-gather-daemonset-fl5dc" Jul 14 02:51:06.211981 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.211860 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8958a66f-256b-41f1-9afe-7c8ecdbeda2f-sys\") pod \"perf-node-gather-daemonset-fl5dc\" (UID: \"8958a66f-256b-41f1-9afe-7c8ecdbeda2f\") " pod="openshift-must-gather-btqq2/perf-node-gather-daemonset-fl5dc" Jul 14 02:51:06.218417 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.218391 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pz5v2\" (UniqueName: \"kubernetes.io/projected/8958a66f-256b-41f1-9afe-7c8ecdbeda2f-kube-api-access-pz5v2\") pod \"perf-node-gather-daemonset-fl5dc\" (UID: \"8958a66f-256b-41f1-9afe-7c8ecdbeda2f\") " pod="openshift-must-gather-btqq2/perf-node-gather-daemonset-fl5dc" Jul 14 02:51:06.333575 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.333492 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-btqq2/perf-node-gather-daemonset-fl5dc" Jul 14 02:51:06.466220 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.466192 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-btqq2/perf-node-gather-daemonset-fl5dc"] Jul 14 02:51:06.467982 ip-10-0-131-25 kubenswrapper[2577]: W0714 02:51:06.467946 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8958a66f_256b_41f1_9afe_7c8ecdbeda2f.slice/crio-e1d1196a96c9fcd5320995ffb6a4cfc3f392d5e245fa84b1620182eb8486a5f8 WatchSource:0}: Error finding container e1d1196a96c9fcd5320995ffb6a4cfc3f392d5e245fa84b1620182eb8486a5f8: Status 404 returned error can't find the container with id e1d1196a96c9fcd5320995ffb6a4cfc3f392d5e245fa84b1620182eb8486a5f8 Jul 14 02:51:06.567895 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.567859 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-btqq2/perf-node-gather-daemonset-fl5dc" event={"ID":"8958a66f-256b-41f1-9afe-7c8ecdbeda2f","Type":"ContainerStarted","Data":"edc933a36f57ce8891fb7d374e5182018f410e2d63df1482d8dd647a502cc416"} Jul 14 02:51:06.568112 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.567917 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-btqq2/perf-node-gather-daemonset-fl5dc" event={"ID":"8958a66f-256b-41f1-9afe-7c8ecdbeda2f","Type":"ContainerStarted","Data":"e1d1196a96c9fcd5320995ffb6a4cfc3f392d5e245fa84b1620182eb8486a5f8"} Jul 14 02:51:06.568112 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.568017 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-btqq2/perf-node-gather-daemonset-fl5dc" Jul 14 02:51:06.581310 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.581243 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-btqq2/perf-node-gather-daemonset-fl5dc" podStartSLOduration=0.581229057 podStartE2EDuration="581.229057ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 02:51:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 02:51:06.580351471 +0000 UTC m=+2968.735188116" watchObservedRunningTime="2026-07-14 02:51:06.581229057 +0000 UTC m=+2968.736065682" Jul 14 02:51:06.962012 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.961981 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6pg9z_a9eccb8f-2ee4-4534-a4f0-60a83fc0879b/dns/0.log" Jul 14 02:51:06.988711 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:06.988678 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6pg9z_a9eccb8f-2ee4-4534-a4f0-60a83fc0879b/kube-rbac-proxy/0.log" Jul 14 02:51:07.099791 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:07.099763 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-k4w2h_059e3b07-82d1-475b-a7c2-4e9345579238/dns-node-resolver/0.log" Jul 14 02:51:07.650831 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:07.650799 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7b4cb8c7f6-kn9xv_193772c5-ef88-4b4d-afbe-c14e65e8e81f/registry/0.log" Jul 14 02:51:07.713466 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:07.713435 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rpl6g_94fa401d-46a7-45c2-bc57-88c112508139/node-ca/0.log" Jul 14 02:51:08.583179 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:08.583154 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf447zmt_83887e08-ddc3-4ae6-a9cd-126107e2dde0/istio-proxy/0.log" Jul 14 02:51:08.748483 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:08.748453 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-fdsj7_8a993e36-54b0-48f9-876d-170e53d0794a/istio-proxy/0.log" Jul 14 02:51:08.777643 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:08.777604 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-fdsj7_8a993e36-54b0-48f9-876d-170e53d0794a/istio-proxy/1.log" Jul 14 02:51:08.850644 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:08.850561 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-8f44474b6-jrxf5_998e452c-dc1f-4e3d-862e-7365d5a216e7/router/0.log" Jul 14 02:51:09.507116 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:09.507082 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-j4k5p_29fa5f87-8e2f-4d38-bc0f-2d6584e8016b/serve-healthcheck-canary/0.log" Jul 14 02:51:09.979154 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:09.979119 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-bkdjp_bbc58bc3-8649-42d2-a277-c27b1c6670c9/insights-operator/0.log" Jul 14 02:51:09.979893 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:09.979872 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-bkdjp_bbc58bc3-8649-42d2-a277-c27b1c6670c9/insights-operator/1.log" Jul 14 02:51:09.997961 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:09.997934 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4v7x2_9162ca72-95c1-49d7-9812-c5705faa7ee8/kube-rbac-proxy/0.log" Jul 14 02:51:10.015809 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:10.015778 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4v7x2_9162ca72-95c1-49d7-9812-c5705faa7ee8/exporter/0.log" Jul 14 02:51:10.032846 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:10.032821 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4v7x2_9162ca72-95c1-49d7-9812-c5705faa7ee8/extractor/0.log" Jul 14 02:51:12.192142 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:12.192112 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-k4497_44a4b3f6-f9fb-40d2-ad21-7e63a6a34269/manager/0.log" Jul 14 02:51:12.324368 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:12.324331 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-zf54c_bc75f235-9d07-4c00-8e8e-59043d8f108e/manager/0.log" Jul 14 02:51:12.509758 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:12.509680 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5756559977-b9bqv_1d26deb9-d556-46a9-8c88-43fb9390a06b/manager/0.log" Jul 14 02:51:12.526173 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:12.526145 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5756559977-b9bqv_1d26deb9-d556-46a9-8c88-43fb9390a06b/copy-manifests/0.log" Jul 14 02:51:12.582180 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:12.582145 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-btqq2/perf-node-gather-daemonset-fl5dc" Jul 14 02:51:13.782680 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:13.782650 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7cbf965c66-v7xzk_9426ae77-c6f7-4e20-b782-97af2aef5277/manager/0.log" Jul 14 02:51:13.802916 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:13.802887 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-v88wb_4b990867-e76a-497f-ab31-2bd3e55b51c5/openshift-lws-operator/0.log" Jul 14 02:51:16.014489 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:16.014452 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf447zmt_83887e08-ddc3-4ae6-a9cd-126107e2dde0/istio-proxy/0.log" Jul 14 02:51:16.168783 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:16.168751 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-fdsj7_8a993e36-54b0-48f9-876d-170e53d0794a/istio-proxy/0.log" Jul 14 02:51:16.197450 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:16.197415 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-fdsj7_8a993e36-54b0-48f9-876d-170e53d0794a/istio-proxy/1.log" Jul 14 02:51:16.249888 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:16.249856 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-8f44474b6-jrxf5_998e452c-dc1f-4e3d-862e-7365d5a216e7/router/0.log" Jul 14 02:51:17.518365 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:17.518326 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf447zmt_83887e08-ddc3-4ae6-a9cd-126107e2dde0/istio-proxy/0.log" Jul 14 02:51:17.687465 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:17.687435 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-fdsj7_8a993e36-54b0-48f9-876d-170e53d0794a/istio-proxy/0.log" Jul 14 02:51:17.717432 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:17.717407 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-fdsj7_8a993e36-54b0-48f9-876d-170e53d0794a/istio-proxy/1.log" Jul 14 02:51:17.775743 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:17.775714 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-8f44474b6-jrxf5_998e452c-dc1f-4e3d-862e-7365d5a216e7/router/0.log" Jul 14 02:51:19.377508 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:19.377467 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-rn5p6_78343924-8733-41ae-9863-94ec59df33d5/kube-storage-version-migrator-operator/1.log" Jul 14 02:51:19.378555 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:19.378531 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-rn5p6_78343924-8733-41ae-9863-94ec59df33d5/kube-storage-version-migrator-operator/0.log" Jul 14 02:51:20.439303 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:20.439255 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gvljc_3d396e3c-208e-4489-88ee-334cfd664280/kube-multus-additional-cni-plugins/0.log" Jul 14 02:51:20.456697 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:20.456672 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gvljc_3d396e3c-208e-4489-88ee-334cfd664280/egress-router-binary-copy/0.log" Jul 14 02:51:20.474360 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:20.474333 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gvljc_3d396e3c-208e-4489-88ee-334cfd664280/cni-plugins/0.log" Jul 14 02:51:20.491046 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:20.491015 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gvljc_3d396e3c-208e-4489-88ee-334cfd664280/bond-cni-plugin/0.log" Jul 14 02:51:20.506858 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:20.506830 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gvljc_3d396e3c-208e-4489-88ee-334cfd664280/routeoverride-cni/0.log" Jul 14 02:51:20.525884 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:20.525854 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gvljc_3d396e3c-208e-4489-88ee-334cfd664280/whereabouts-cni-bincopy/0.log" Jul 14 02:51:20.543150 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:20.543123 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gvljc_3d396e3c-208e-4489-88ee-334cfd664280/whereabouts-cni/0.log" Jul 14 02:51:20.734207 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:20.734121 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-zc4lr_62531cce-f69d-4ccd-a6c6-e090b2789022/kube-multus/0.log" Jul 14 02:51:20.788197 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:20.788164 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-t9bnr_8d6a986a-c066-4ce3-a49c-4bab5be2f075/network-metrics-daemon/0.log" Jul 14 02:51:20.803448 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:20.803417 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-t9bnr_8d6a986a-c066-4ce3-a49c-4bab5be2f075/kube-rbac-proxy/0.log" Jul 14 02:51:21.632908 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:21.632878 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bnpgh_13c988f5-6d7a-4d10-87c3-d548e48ac1d6/ovn-controller/0.log" Jul 14 02:51:21.655777 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:21.655752 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bnpgh_13c988f5-6d7a-4d10-87c3-d548e48ac1d6/ovn-acl-logging/0.log" Jul 14 02:51:21.676991 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:21.676961 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bnpgh_13c988f5-6d7a-4d10-87c3-d548e48ac1d6/kube-rbac-proxy-node/0.log" Jul 14 02:51:21.699361 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:21.699336 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bnpgh_13c988f5-6d7a-4d10-87c3-d548e48ac1d6/kube-rbac-proxy-ovn-metrics/0.log" Jul 14 02:51:21.720691 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:21.720669 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bnpgh_13c988f5-6d7a-4d10-87c3-d548e48ac1d6/northd/0.log" Jul 14 02:51:21.739005 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:21.738979 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bnpgh_13c988f5-6d7a-4d10-87c3-d548e48ac1d6/nbdb/0.log" Jul 14 02:51:21.756748 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:21.756722 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bnpgh_13c988f5-6d7a-4d10-87c3-d548e48ac1d6/sbdb/0.log" Jul 14 02:51:21.804106 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:21.804077 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bnpgh_13c988f5-6d7a-4d10-87c3-d548e48ac1d6/ovnkube-controller/0.log" Jul 14 02:51:23.186380 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:23.186353 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-869dcfcdcd-pb8mx_d76325b8-c23d-4872-b1ce-a612cac8ad43/check-endpoints/0.log" Jul 14 02:51:23.207714 ip-10-0-131-25 kubenswrapper[2577]: I0714 02:51:23.207688 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-c6wmg_f018324f-f63f-41e0-9a3f-d6fed83b4f1c/network-check-target-container/0.log"