Jul 22 03:07:39.448253 ip-10-0-133-13 systemd[1]: Starting Kubernetes Kubelet... Jul 22 03:07:39.776691 ip-10-0-133-13 kubenswrapper[2556]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 03:07:39.776691 ip-10-0-133-13 kubenswrapper[2556]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 22 03:07:39.776691 ip-10-0-133-13 kubenswrapper[2556]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 03:07:39.776691 ip-10-0-133-13 kubenswrapper[2556]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 22 03:07:39.776691 ip-10-0-133-13 kubenswrapper[2556]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 03:07:39.778190 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.778075 2556 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 22 03:07:39.782718 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782695 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 03:07:39.782718 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782715 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 03:07:39.782718 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782720 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 03:07:39.782718 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782724 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 03:07:39.782981 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782729 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 03:07:39.782981 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782733 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 03:07:39.782981 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782736 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 03:07:39.782981 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782740 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 03:07:39.782981 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782743 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 03:07:39.782981 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782748 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 03:07:39.782981 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782751 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 03:07:39.782981 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782755 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 03:07:39.782981 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782759 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 03:07:39.782981 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782763 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 03:07:39.782981 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782767 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 03:07:39.782981 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782771 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 03:07:39.782981 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782775 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 03:07:39.782981 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782778 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 03:07:39.782981 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782782 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 03:07:39.782981 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782785 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 03:07:39.782981 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782789 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 03:07:39.782981 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782793 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 03:07:39.782981 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782797 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 03:07:39.783613 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782801 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 03:07:39.783613 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782805 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 03:07:39.783613 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782810 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 03:07:39.783613 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782813 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 03:07:39.783613 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782822 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 03:07:39.783613 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782826 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 03:07:39.783613 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782830 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 03:07:39.783613 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782833 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 03:07:39.783613 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782838 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 03:07:39.783613 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782842 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 03:07:39.783613 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782846 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 03:07:39.783613 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782850 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 03:07:39.783613 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782854 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 03:07:39.783613 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782862 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 03:07:39.783613 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782870 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 03:07:39.783613 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782874 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 03:07:39.783613 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782878 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 03:07:39.783613 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782883 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 03:07:39.783613 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782887 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 03:07:39.783613 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782891 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 03:07:39.784152 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782895 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 03:07:39.784152 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782900 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 03:07:39.784152 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782904 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 03:07:39.784152 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782908 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 03:07:39.784152 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782912 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 03:07:39.784152 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782916 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 03:07:39.784152 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782920 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 03:07:39.784152 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782924 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 03:07:39.784152 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782929 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 03:07:39.784152 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782933 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 03:07:39.784152 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782937 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 03:07:39.784152 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782942 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 03:07:39.784152 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782964 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 03:07:39.784152 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782972 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 22 03:07:39.784152 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782977 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 03:07:39.784152 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782981 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 03:07:39.784152 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782985 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 03:07:39.784152 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782989 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 03:07:39.784152 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.782993 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 03:07:39.784771 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.783000 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 03:07:39.784771 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.783006 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 03:07:39.784771 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.783010 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 03:07:39.784771 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.783016 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 03:07:39.784771 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.783022 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 03:07:39.784771 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.783026 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 03:07:39.784771 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.783031 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 03:07:39.784771 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.783036 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 03:07:39.784771 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.783041 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 03:07:39.784771 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.783055 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 03:07:39.784771 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.783060 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 03:07:39.784771 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.783065 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 03:07:39.784771 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.783070 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 03:07:39.784771 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.783075 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 03:07:39.784771 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.783079 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 03:07:39.784771 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.783084 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 03:07:39.784771 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.783088 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 03:07:39.784771 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.783092 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 03:07:39.784771 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.783096 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 03:07:39.785577 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.783100 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 03:07:39.785577 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.783105 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 03:07:39.785577 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.783111 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 03:07:39.785577 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.783115 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 03:07:39.785577 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.783120 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 03:07:39.785577 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.783124 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 03:07:39.785577 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.783128 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 03:07:39.785577 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.783132 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 03:07:39.785577 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.783137 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 03:07:39.785577 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.783141 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 03:07:39.785577 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.783145 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 03:07:39.785577 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.783149 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 03:07:39.785577 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.783154 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 03:07:39.785577 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.783158 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 03:07:39.785577 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.783162 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 03:07:39.785577 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.783166 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 03:07:39.785577 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.783170 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 03:07:39.785577 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.783174 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 03:07:39.785577 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785566 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 03:07:39.785577 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785576 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 03:07:39.786438 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785581 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 03:07:39.786438 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785586 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 03:07:39.786438 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785591 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 03:07:39.786438 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785596 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 03:07:39.786438 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785601 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 03:07:39.786438 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785606 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 03:07:39.786438 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785610 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 03:07:39.786438 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785615 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 03:07:39.786438 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785619 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 03:07:39.786438 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785623 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 03:07:39.786438 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785627 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 03:07:39.786438 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785631 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 03:07:39.786438 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785635 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 03:07:39.786438 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785639 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 22 03:07:39.786438 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785643 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 03:07:39.786438 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785648 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 03:07:39.786438 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785653 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 03:07:39.786438 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785657 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 03:07:39.786438 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785661 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 03:07:39.787338 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785665 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 03:07:39.787338 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785669 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 03:07:39.787338 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785673 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 03:07:39.787338 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785677 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 03:07:39.787338 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785683 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 03:07:39.787338 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785689 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 03:07:39.787338 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785694 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 03:07:39.787338 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785698 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 03:07:39.787338 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785702 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 03:07:39.787338 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785706 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 03:07:39.787338 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785710 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 03:07:39.787338 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785714 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 03:07:39.787338 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785718 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 03:07:39.787338 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785722 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 03:07:39.787338 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785728 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 03:07:39.787338 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785732 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 03:07:39.787338 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785736 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 03:07:39.787338 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785740 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 03:07:39.787338 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785744 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 03:07:39.787338 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785748 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 03:07:39.788233 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785752 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 03:07:39.788233 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785756 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 03:07:39.788233 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785760 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 03:07:39.788233 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785765 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 03:07:39.788233 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785769 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 03:07:39.788233 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785775 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 03:07:39.788233 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785781 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 03:07:39.788233 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785786 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 03:07:39.788233 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785791 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 03:07:39.788233 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785797 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 03:07:39.788233 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785802 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 03:07:39.788233 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785807 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 03:07:39.788233 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785811 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 03:07:39.788233 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785818 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 03:07:39.788233 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785824 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 03:07:39.788233 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785829 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 03:07:39.788233 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785834 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 03:07:39.788233 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785838 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 03:07:39.788891 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785843 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 03:07:39.788891 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785849 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 03:07:39.788891 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785853 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 03:07:39.788891 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785858 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 03:07:39.788891 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785863 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 03:07:39.788891 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785867 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 03:07:39.788891 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785872 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 03:07:39.788891 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785877 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 03:07:39.788891 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785883 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 03:07:39.788891 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785887 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 03:07:39.788891 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785891 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 03:07:39.788891 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785895 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 03:07:39.788891 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785900 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 03:07:39.788891 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785905 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 03:07:39.788891 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785909 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 03:07:39.788891 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785914 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 03:07:39.788891 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785918 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 03:07:39.788891 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785922 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 03:07:39.788891 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785926 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 03:07:39.788891 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785930 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 03:07:39.789503 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785934 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 03:07:39.789503 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785938 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 03:07:39.789503 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785943 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 03:07:39.789503 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785947 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 03:07:39.789503 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785970 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 03:07:39.789503 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785974 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 03:07:39.789503 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785979 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 03:07:39.789503 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785983 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 03:07:39.789503 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785987 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 03:07:39.789503 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785991 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 03:07:39.789503 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.785995 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 03:07:39.789503 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786000 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 03:07:39.789503 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786005 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 03:07:39.789503 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786009 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 03:07:39.789503 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786014 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 03:07:39.789503 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786018 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 03:07:39.789503 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786022 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 03:07:39.789503 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786026 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 03:07:39.789503 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786030 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 03:07:39.789503 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786035 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 03:07:39.790140 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786646 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 03:07:39.790140 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786655 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 03:07:39.790140 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786660 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 03:07:39.790140 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786665 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 03:07:39.790140 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786670 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 03:07:39.790140 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786677 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 03:07:39.790140 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786683 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 03:07:39.790140 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786687 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 03:07:39.790140 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786692 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 03:07:39.790140 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786696 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 03:07:39.790140 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786700 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 03:07:39.790140 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786705 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 03:07:39.790140 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786710 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 03:07:39.790140 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786714 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 03:07:39.790140 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786718 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 03:07:39.790140 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786722 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 03:07:39.790140 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786728 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 03:07:39.790140 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786732 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 03:07:39.790140 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786736 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 03:07:39.790140 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786741 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 03:07:39.790736 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786745 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 03:07:39.790736 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786749 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 03:07:39.790736 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786753 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 03:07:39.790736 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786757 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 03:07:39.790736 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786761 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 03:07:39.790736 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786766 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 03:07:39.790736 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786770 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 03:07:39.790736 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786774 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 03:07:39.790736 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786778 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 03:07:39.790736 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786782 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 03:07:39.790736 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786786 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 03:07:39.790736 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786790 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 03:07:39.790736 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786797 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 03:07:39.790736 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786802 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 03:07:39.790736 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786806 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 03:07:39.790736 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786811 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 03:07:39.790736 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786814 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 03:07:39.790736 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786818 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 03:07:39.790736 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786822 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 03:07:39.791247 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786835 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 03:07:39.791247 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786840 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 03:07:39.791247 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786844 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 03:07:39.791247 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786849 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 03:07:39.791247 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786853 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 03:07:39.791247 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786857 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 03:07:39.791247 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786860 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 03:07:39.791247 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786865 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 03:07:39.791247 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786891 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 03:07:39.791247 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786897 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 03:07:39.791247 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786903 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 03:07:39.791247 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786908 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 03:07:39.791247 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786913 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 03:07:39.791247 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786917 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 03:07:39.791247 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786921 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 03:07:39.791247 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786925 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 03:07:39.791247 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786930 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 03:07:39.791247 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786934 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 03:07:39.791247 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786938 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 03:07:39.791247 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786943 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 03:07:39.791863 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786947 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 03:07:39.791863 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786968 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 03:07:39.791863 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786973 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 03:07:39.791863 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786977 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 03:07:39.791863 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786981 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 03:07:39.791863 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786987 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 03:07:39.791863 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786991 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 03:07:39.791863 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.786996 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 03:07:39.791863 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.787000 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 03:07:39.791863 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.787009 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 03:07:39.791863 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.787015 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 03:07:39.791863 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.787020 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 22 03:07:39.791863 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.787025 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 03:07:39.791863 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.787030 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 03:07:39.791863 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.787035 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 03:07:39.791863 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.787039 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 03:07:39.791863 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.787052 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 03:07:39.791863 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.787058 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 03:07:39.791863 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.787063 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 03:07:39.792383 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.787067 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 03:07:39.792383 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.787072 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 03:07:39.792383 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.787076 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 03:07:39.792383 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.787080 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 03:07:39.792383 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.787085 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 03:07:39.792383 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.787089 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 03:07:39.792383 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.787093 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 03:07:39.792383 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.787097 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 03:07:39.792383 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.787102 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 03:07:39.792383 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.787106 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 03:07:39.792383 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.787110 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 03:07:39.792383 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.787115 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 03:07:39.792383 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.787119 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 03:07:39.792383 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.787123 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 03:07:39.792383 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.787127 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 03:07:39.792383 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.787132 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 03:07:39.792383 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.787136 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 03:07:39.792383 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.787140 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 03:07:39.792383 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.787147 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 03:07:39.792383 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.787151 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 03:07:39.792864 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.787156 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 03:07:39.792864 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787244 2556 flags.go:64] FLAG: --address="0.0.0.0" Jul 22 03:07:39.792864 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787256 2556 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 22 03:07:39.792864 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787268 2556 flags.go:64] FLAG: --anonymous-auth="true" Jul 22 03:07:39.792864 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787274 2556 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 22 03:07:39.792864 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787280 2556 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 22 03:07:39.792864 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787285 2556 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 22 03:07:39.792864 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787291 2556 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 22 03:07:39.792864 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787298 2556 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 22 03:07:39.792864 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787303 2556 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 22 03:07:39.792864 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787308 2556 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 22 03:07:39.792864 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787314 2556 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 22 03:07:39.792864 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787319 2556 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 22 03:07:39.792864 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787324 2556 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 22 03:07:39.792864 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787328 2556 flags.go:64] FLAG: --cgroup-root="" Jul 22 03:07:39.792864 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787333 2556 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 22 03:07:39.792864 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787337 2556 flags.go:64] FLAG: --client-ca-file="" Jul 22 03:07:39.792864 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787343 2556 flags.go:64] FLAG: --cloud-provider="external" Jul 22 03:07:39.792864 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787347 2556 flags.go:64] FLAG: --cluster-dns="[]" Jul 22 03:07:39.792864 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787352 2556 flags.go:64] FLAG: --cluster-domain="" Jul 22 03:07:39.792864 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787357 2556 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 22 03:07:39.792864 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787362 2556 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 22 03:07:39.792864 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787367 2556 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 22 03:07:39.792864 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787372 2556 flags.go:64] FLAG: --container-log-max-files="5" Jul 22 03:07:39.793493 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787379 2556 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 22 03:07:39.793493 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787384 2556 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 22 03:07:39.793493 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787389 2556 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 22 03:07:39.793493 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787394 2556 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 22 03:07:39.793493 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787399 2556 flags.go:64] FLAG: --contention-profiling="false" Jul 22 03:07:39.793493 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787403 2556 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 22 03:07:39.793493 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787408 2556 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 22 03:07:39.793493 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787413 2556 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 22 03:07:39.793493 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787418 2556 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 22 03:07:39.793493 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787424 2556 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 22 03:07:39.793493 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787429 2556 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 22 03:07:39.793493 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787434 2556 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 22 03:07:39.793493 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787438 2556 flags.go:64] FLAG: --enable-load-reader="false" Jul 22 03:07:39.793493 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787443 2556 flags.go:64] FLAG: --enable-server="true" Jul 22 03:07:39.793493 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787447 2556 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 22 03:07:39.793493 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787456 2556 flags.go:64] FLAG: --event-burst="100" Jul 22 03:07:39.793493 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787460 2556 flags.go:64] FLAG: --event-qps="50" Jul 22 03:07:39.793493 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787465 2556 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 22 03:07:39.793493 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787470 2556 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 22 03:07:39.793493 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787474 2556 flags.go:64] FLAG: --eviction-hard="" Jul 22 03:07:39.793493 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787480 2556 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 22 03:07:39.793493 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787485 2556 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 22 03:07:39.793493 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787489 2556 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 22 03:07:39.793493 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787494 2556 flags.go:64] FLAG: --eviction-soft="" Jul 22 03:07:39.793493 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787499 2556 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 22 03:07:39.794143 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787503 2556 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 22 03:07:39.794143 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787509 2556 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 22 03:07:39.794143 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787513 2556 flags.go:64] FLAG: --experimental-mounter-path="" Jul 22 03:07:39.794143 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787518 2556 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 22 03:07:39.794143 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787522 2556 flags.go:64] FLAG: --fail-swap-on="true" Jul 22 03:07:39.794143 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787527 2556 flags.go:64] FLAG: --feature-gates="" Jul 22 03:07:39.794143 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787533 2556 flags.go:64] FLAG: --file-check-frequency="20s" Jul 22 03:07:39.794143 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787538 2556 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 22 03:07:39.794143 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787543 2556 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 22 03:07:39.794143 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787548 2556 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 22 03:07:39.794143 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787553 2556 flags.go:64] FLAG: --healthz-port="10248" Jul 22 03:07:39.794143 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787558 2556 flags.go:64] FLAG: --help="false" Jul 22 03:07:39.794143 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787562 2556 flags.go:64] FLAG: --hostname-override="ip-10-0-133-13.ec2.internal" Jul 22 03:07:39.794143 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787568 2556 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 22 03:07:39.794143 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787573 2556 flags.go:64] FLAG: --http-check-frequency="20s" Jul 22 03:07:39.794143 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787577 2556 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 22 03:07:39.794143 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787583 2556 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 22 03:07:39.794143 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787589 2556 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 22 03:07:39.794143 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787594 2556 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 22 03:07:39.794143 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787599 2556 flags.go:64] FLAG: --image-service-endpoint="" Jul 22 03:07:39.794143 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787603 2556 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 22 03:07:39.794143 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787608 2556 flags.go:64] FLAG: --kube-api-burst="100" Jul 22 03:07:39.794143 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787612 2556 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 22 03:07:39.794747 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787617 2556 flags.go:64] FLAG: --kube-api-qps="50" Jul 22 03:07:39.794747 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787622 2556 flags.go:64] FLAG: --kube-reserved="" Jul 22 03:07:39.794747 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787626 2556 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 22 03:07:39.794747 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787631 2556 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 22 03:07:39.794747 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787636 2556 flags.go:64] FLAG: --kubelet-cgroups="" Jul 22 03:07:39.794747 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787640 2556 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 22 03:07:39.794747 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787645 2556 flags.go:64] FLAG: --lock-file="" Jul 22 03:07:39.794747 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787650 2556 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 22 03:07:39.794747 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787656 2556 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 22 03:07:39.794747 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787660 2556 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 22 03:07:39.794747 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787668 2556 flags.go:64] FLAG: --log-json-split-stream="false" Jul 22 03:07:39.794747 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787679 2556 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 22 03:07:39.794747 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787685 2556 flags.go:64] FLAG: --log-text-split-stream="false" Jul 22 03:07:39.794747 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787689 2556 flags.go:64] FLAG: --logging-format="text" Jul 22 03:07:39.794747 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787694 2556 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 22 03:07:39.794747 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787699 2556 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 22 03:07:39.794747 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787704 2556 flags.go:64] FLAG: --manifest-url="" Jul 22 03:07:39.794747 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787708 2556 flags.go:64] FLAG: --manifest-url-header="" Jul 22 03:07:39.794747 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787715 2556 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 22 03:07:39.794747 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787720 2556 flags.go:64] FLAG: --max-open-files="1000000" Jul 22 03:07:39.794747 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787726 2556 flags.go:64] FLAG: --max-pods="110" Jul 22 03:07:39.794747 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787731 2556 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 22 03:07:39.794747 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787736 2556 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 22 03:07:39.794747 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787741 2556 flags.go:64] FLAG: --memory-manager-policy="None" Jul 22 03:07:39.794747 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787745 2556 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 22 03:07:39.795385 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787750 2556 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 22 03:07:39.795385 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787754 2556 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 22 03:07:39.795385 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787759 2556 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 22 03:07:39.795385 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787771 2556 flags.go:64] FLAG: --node-status-max-images="50" Jul 22 03:07:39.795385 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787775 2556 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 22 03:07:39.795385 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787780 2556 flags.go:64] FLAG: --oom-score-adj="-999" Jul 22 03:07:39.795385 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787784 2556 flags.go:64] FLAG: --pod-cidr="" Jul 22 03:07:39.795385 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787789 2556 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 22 03:07:39.795385 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787797 2556 flags.go:64] FLAG: --pod-manifest-path="" Jul 22 03:07:39.795385 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787801 2556 flags.go:64] FLAG: --pod-max-pids="-1" Jul 22 03:07:39.795385 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787806 2556 flags.go:64] FLAG: --pods-per-core="0" Jul 22 03:07:39.795385 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787810 2556 flags.go:64] FLAG: --port="10250" Jul 22 03:07:39.795385 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787815 2556 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 22 03:07:39.795385 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787820 2556 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-087a066bc793d0142" Jul 22 03:07:39.795385 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787825 2556 flags.go:64] FLAG: --qos-reserved="" Jul 22 03:07:39.795385 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787829 2556 flags.go:64] FLAG: --read-only-port="10255" Jul 22 03:07:39.795385 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787833 2556 flags.go:64] FLAG: --register-node="true" Jul 22 03:07:39.795385 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787838 2556 flags.go:64] FLAG: --register-with-taints="" Jul 22 03:07:39.795385 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787843 2556 flags.go:64] FLAG: --registry-burst="10" Jul 22 03:07:39.795385 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787856 2556 flags.go:64] FLAG: --registry-qps="5" Jul 22 03:07:39.795385 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787860 2556 flags.go:64] FLAG: --reserved-cpus="" Jul 22 03:07:39.795385 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787867 2556 flags.go:64] FLAG: --reserved-memory="" Jul 22 03:07:39.795385 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787873 2556 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 22 03:07:39.795385 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787877 2556 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 22 03:07:39.795385 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787882 2556 flags.go:64] FLAG: --rotate-certificates="false" Jul 22 03:07:39.796018 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787887 2556 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 22 03:07:39.796018 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787892 2556 flags.go:64] FLAG: --runonce="false" Jul 22 03:07:39.796018 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787896 2556 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 22 03:07:39.796018 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787901 2556 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 22 03:07:39.796018 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787906 2556 flags.go:64] FLAG: --seccomp-default="false" Jul 22 03:07:39.796018 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787911 2556 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 22 03:07:39.796018 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787915 2556 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 22 03:07:39.796018 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787920 2556 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 22 03:07:39.796018 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787925 2556 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 22 03:07:39.796018 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787929 2556 flags.go:64] FLAG: --storage-driver-password="root" Jul 22 03:07:39.796018 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787934 2556 flags.go:64] FLAG: --storage-driver-secure="false" Jul 22 03:07:39.796018 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787938 2556 flags.go:64] FLAG: --storage-driver-table="stats" Jul 22 03:07:39.796018 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787943 2556 flags.go:64] FLAG: --storage-driver-user="root" Jul 22 03:07:39.796018 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787966 2556 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 22 03:07:39.796018 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787971 2556 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 22 03:07:39.796018 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787976 2556 flags.go:64] FLAG: --system-cgroups="" Jul 22 03:07:39.796018 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787980 2556 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 22 03:07:39.796018 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787988 2556 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 22 03:07:39.796018 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787993 2556 flags.go:64] FLAG: --tls-cert-file="" Jul 22 03:07:39.796018 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.787997 2556 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 22 03:07:39.796018 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.788003 2556 flags.go:64] FLAG: --tls-min-version="" Jul 22 03:07:39.796018 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.788008 2556 flags.go:64] FLAG: --tls-private-key-file="" Jul 22 03:07:39.796018 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.788012 2556 flags.go:64] FLAG: --topology-manager-policy="none" Jul 22 03:07:39.796018 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.788017 2556 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 22 03:07:39.796018 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.788021 2556 flags.go:64] FLAG: --topology-manager-scope="container" Jul 22 03:07:39.796624 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.788026 2556 flags.go:64] FLAG: --v="2" Jul 22 03:07:39.796624 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.788032 2556 flags.go:64] FLAG: --version="false" Jul 22 03:07:39.796624 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.788040 2556 flags.go:64] FLAG: --vmodule="" Jul 22 03:07:39.796624 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.788052 2556 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 22 03:07:39.796624 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.788057 2556 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 22 03:07:39.796624 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788234 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 03:07:39.796624 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788242 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 03:07:39.796624 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788246 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 03:07:39.796624 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788251 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 03:07:39.796624 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788255 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 03:07:39.796624 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788261 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 03:07:39.796624 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788267 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 03:07:39.796624 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788274 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 03:07:39.796624 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788278 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 03:07:39.796624 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788283 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 03:07:39.796624 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788287 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 03:07:39.796624 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788291 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 03:07:39.796624 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788296 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 03:07:39.796624 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788300 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 03:07:39.796624 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788304 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 03:07:39.796624 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788309 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 03:07:39.797150 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788313 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 03:07:39.797150 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788317 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 03:07:39.797150 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788321 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 03:07:39.797150 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788326 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 03:07:39.797150 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788330 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 03:07:39.797150 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788334 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 03:07:39.797150 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788338 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 03:07:39.797150 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788342 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 03:07:39.797150 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788346 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 03:07:39.797150 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788352 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 03:07:39.797150 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788355 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 03:07:39.797150 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788360 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 03:07:39.797150 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788366 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 03:07:39.797150 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788372 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 03:07:39.797150 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788379 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 03:07:39.797150 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788384 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 03:07:39.797150 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788388 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 03:07:39.797150 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788393 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 03:07:39.797150 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788397 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 03:07:39.797637 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788402 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 03:07:39.797637 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788406 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 03:07:39.797637 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788410 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 03:07:39.797637 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788414 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 22 03:07:39.797637 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788420 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 03:07:39.797637 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788424 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 03:07:39.797637 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788429 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 03:07:39.797637 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788433 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 03:07:39.797637 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788437 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 03:07:39.797637 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788441 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 03:07:39.797637 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788446 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 03:07:39.797637 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788450 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 03:07:39.797637 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788454 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 03:07:39.797637 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788458 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 03:07:39.797637 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788462 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 03:07:39.797637 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788467 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 03:07:39.797637 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788471 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 03:07:39.797637 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788475 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 03:07:39.797637 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788480 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 03:07:39.798120 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788484 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 03:07:39.798120 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788488 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 03:07:39.798120 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788492 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 03:07:39.798120 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788498 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 03:07:39.798120 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788502 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 03:07:39.798120 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788506 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 03:07:39.798120 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788510 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 03:07:39.798120 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788514 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 03:07:39.798120 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788520 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 03:07:39.798120 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788526 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 03:07:39.798120 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788530 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 03:07:39.798120 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788534 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 03:07:39.798120 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788539 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 03:07:39.798120 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788543 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 03:07:39.798120 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788549 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 03:07:39.798120 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788555 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 03:07:39.798120 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788559 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 03:07:39.798120 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788565 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 03:07:39.798120 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788569 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 03:07:39.798590 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788574 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 03:07:39.798590 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788578 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 03:07:39.798590 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788582 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 03:07:39.798590 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788586 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 03:07:39.798590 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788590 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 03:07:39.798590 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788594 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 03:07:39.798590 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788599 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 03:07:39.798590 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788603 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 03:07:39.798590 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788607 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 03:07:39.798590 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788612 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 03:07:39.798590 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788616 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 03:07:39.798590 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788620 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 03:07:39.798590 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788624 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 03:07:39.798590 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788628 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 03:07:39.798590 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788633 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 03:07:39.798590 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788637 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 03:07:39.798590 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788641 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 03:07:39.798590 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788645 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 03:07:39.798590 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788649 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 03:07:39.798590 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788654 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 03:07:39.799089 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788658 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 03:07:39.799089 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788664 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 03:07:39.799089 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788668 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 03:07:39.799089 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788673 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 03:07:39.799089 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788677 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 03:07:39.799089 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.788681 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 03:07:39.799089 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.789277 2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 03:07:39.799089 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.795997 2556 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 22 03:07:39.799089 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.796019 2556 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 22 03:07:39.799089 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.796038 2556 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 22 03:07:39.799089 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.796044 2556 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 22 03:07:39.799089 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796105 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 03:07:39.799089 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796110 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 03:07:39.799089 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796113 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 03:07:39.799089 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796116 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 03:07:39.799460 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796119 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 03:07:39.799460 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796122 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 03:07:39.799460 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796125 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 03:07:39.799460 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796127 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 03:07:39.799460 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796130 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 03:07:39.799460 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796132 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 03:07:39.799460 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796135 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 03:07:39.799460 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796138 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 22 03:07:39.799460 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796140 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 03:07:39.799460 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796144 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 03:07:39.799460 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796148 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 03:07:39.799460 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796152 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 03:07:39.799460 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796155 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 03:07:39.799460 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796159 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 03:07:39.799460 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796161 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 03:07:39.799460 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796164 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 03:07:39.799460 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796168 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 03:07:39.799460 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796171 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 03:07:39.799890 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796174 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 03:07:39.799890 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796177 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 03:07:39.799890 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796180 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 03:07:39.799890 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796182 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 03:07:39.799890 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796185 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 03:07:39.799890 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796188 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 03:07:39.799890 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796191 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 03:07:39.799890 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796200 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 03:07:39.799890 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796203 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 03:07:39.799890 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796205 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 03:07:39.799890 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796208 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 03:07:39.799890 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796211 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 03:07:39.799890 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796213 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 03:07:39.799890 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796216 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 03:07:39.799890 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796218 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 03:07:39.799890 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796221 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 03:07:39.799890 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796223 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 03:07:39.799890 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796226 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 03:07:39.799890 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796228 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 03:07:39.799890 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796231 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 03:07:39.800393 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796233 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 03:07:39.800393 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796236 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 03:07:39.800393 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796239 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 03:07:39.800393 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796241 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 03:07:39.800393 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796243 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 03:07:39.800393 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796246 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 03:07:39.800393 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796248 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 03:07:39.800393 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796251 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 03:07:39.800393 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796253 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 03:07:39.800393 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796256 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 03:07:39.800393 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796258 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 03:07:39.800393 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796261 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 03:07:39.800393 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796264 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 03:07:39.800393 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796266 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 03:07:39.800393 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796269 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 03:07:39.800393 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796271 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 03:07:39.800393 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796274 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 03:07:39.800393 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796276 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 03:07:39.800393 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796279 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 03:07:39.800393 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796282 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 03:07:39.800868 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796290 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 03:07:39.800868 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796293 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 03:07:39.800868 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796296 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 03:07:39.800868 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796298 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 03:07:39.800868 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796301 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 03:07:39.800868 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796303 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 03:07:39.800868 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796306 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 03:07:39.800868 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796308 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 03:07:39.800868 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796311 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 03:07:39.800868 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796314 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 03:07:39.800868 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796316 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 03:07:39.800868 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796319 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 03:07:39.800868 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796322 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 03:07:39.800868 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796324 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 03:07:39.800868 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796327 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 03:07:39.800868 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796329 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 03:07:39.800868 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796332 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 03:07:39.800868 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796334 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 03:07:39.800868 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796337 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 03:07:39.800868 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796339 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 03:07:39.801433 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796342 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 03:07:39.801433 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796344 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 03:07:39.801433 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796347 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 03:07:39.801433 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796350 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 03:07:39.801433 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796353 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 03:07:39.801433 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796355 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 03:07:39.801433 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796358 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 03:07:39.801433 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796361 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 03:07:39.801433 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796364 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 03:07:39.801433 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796366 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 03:07:39.801433 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796368 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 03:07:39.801433 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796371 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 03:07:39.801433 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796374 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 03:07:39.801433 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796382 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 03:07:39.801433 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796384 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 03:07:39.801433 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796386 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 03:07:39.801433 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796389 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 03:07:39.801841 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.796394 2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 03:07:39.801841 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796534 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 03:07:39.801841 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796538 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 03:07:39.801841 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796541 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 03:07:39.801841 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796544 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 03:07:39.801841 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796547 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 03:07:39.801841 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796550 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 03:07:39.801841 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796553 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 03:07:39.801841 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796555 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 03:07:39.801841 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796558 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 03:07:39.801841 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796561 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 03:07:39.801841 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796564 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 03:07:39.801841 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796566 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 03:07:39.801841 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796569 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 03:07:39.801841 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796572 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 03:07:39.802230 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796574 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 03:07:39.802230 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796576 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 03:07:39.802230 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796579 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 03:07:39.802230 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796581 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 03:07:39.802230 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796584 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 03:07:39.802230 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796587 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 03:07:39.802230 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796590 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 03:07:39.802230 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796592 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 03:07:39.802230 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796595 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 03:07:39.802230 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796597 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 03:07:39.802230 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796600 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 03:07:39.802230 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796602 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 03:07:39.802230 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796605 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 03:07:39.802230 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796607 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 03:07:39.802230 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796616 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 03:07:39.802230 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796618 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 03:07:39.802230 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796621 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 03:07:39.802230 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796624 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 03:07:39.802230 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796626 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 03:07:39.802230 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796629 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 03:07:39.802738 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796631 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 03:07:39.802738 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796634 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 03:07:39.802738 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796637 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 03:07:39.802738 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796639 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 03:07:39.802738 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796642 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 03:07:39.802738 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796644 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 03:07:39.802738 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796647 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 03:07:39.802738 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796649 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 03:07:39.802738 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796652 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 03:07:39.802738 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796654 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 03:07:39.802738 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796656 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 03:07:39.802738 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796659 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 03:07:39.802738 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796661 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 03:07:39.802738 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796664 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 03:07:39.802738 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796666 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 03:07:39.802738 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796668 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 03:07:39.802738 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796672 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 03:07:39.802738 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796675 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 03:07:39.802738 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796677 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 03:07:39.802738 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796680 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 03:07:39.803262 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796682 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 03:07:39.803262 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796685 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 03:07:39.803262 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796687 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 03:07:39.803262 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796689 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 03:07:39.803262 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796692 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 03:07:39.803262 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796694 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 03:07:39.803262 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796696 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 03:07:39.803262 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796704 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 03:07:39.803262 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796707 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 03:07:39.803262 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796711 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 03:07:39.803262 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796714 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 03:07:39.803262 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796717 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 03:07:39.803262 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796719 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 03:07:39.803262 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796722 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 03:07:39.803262 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796725 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 03:07:39.803262 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796729 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 03:07:39.803262 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796732 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 03:07:39.803262 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796735 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 03:07:39.803262 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796738 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 03:07:39.803980 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796741 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 03:07:39.803980 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796743 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 03:07:39.803980 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796746 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 03:07:39.803980 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796749 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 03:07:39.803980 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796751 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 03:07:39.803980 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796753 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 03:07:39.803980 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796756 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 03:07:39.803980 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796758 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 03:07:39.803980 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796761 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 03:07:39.803980 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796764 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 03:07:39.803980 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796766 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 03:07:39.803980 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796769 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 03:07:39.803980 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796771 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 03:07:39.803980 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796774 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 03:07:39.803980 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796776 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 03:07:39.803980 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796778 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 22 03:07:39.803980 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796781 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 03:07:39.803980 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796783 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 03:07:39.803980 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796787 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 03:07:39.803980 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796791 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 03:07:39.804671 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796793 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 03:07:39.804671 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796802 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 03:07:39.804671 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796804 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 03:07:39.804671 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796807 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 03:07:39.804671 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796809 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 03:07:39.804671 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:39.796812 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 03:07:39.804671 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.796816 2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 03:07:39.804671 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.796944 2556 server.go:962] "Client rotation is on, will bootstrap in background" Jul 22 03:07:39.804671 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.798769 2556 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 22 03:07:39.804671 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.799893 2556 server.go:1019] "Starting client certificate rotation" Jul 22 03:07:39.804671 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.800000 2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 03:07:39.804671 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.800029 2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 03:07:39.804671 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.802300 2556 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 03:07:39.805886 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.805868 2556 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 03:07:39.815091 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.815074 2556 log.go:25] "Validated CRI v1 runtime API" Jul 22 03:07:39.816647 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.816631 2556 log.go:25] "Validated CRI v1 image API" Jul 22 03:07:39.817473 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.817456 2556 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 22 03:07:39.819741 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.819717 2556 fs.go:135] Filesystem UUIDs: map[42eb6c39-72cb-426e-81dc-94fea2268883:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 9dfef19a-3c30-4eb0-8b5c-b4984d0a598d:/dev/nvme0n1p3] Jul 22 03:07:39.819824 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.819739 2556 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 22 03:07:39.824996 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.824867 2556 manager.go:216] Machine: {Timestamp:2026-07-22 03:07:39.823597649 +0000 UTC m=+0.272580889 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101514 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ac770c7dc1e3ea04aee468076e560 SystemUUID:ec2ac770-c7dc-1e3e-a04a-ee468076e560 BootID:255e6ca6-1e9c-4905-b64f-9d9f0bd54200 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:4d:73:18:f0:75 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:4d:73:18:f0:75 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:92:2d:f9:b3:2a:a0 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 22 03:07:39.824996 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.824988 2556 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 22 03:07:39.825147 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.825133 2556 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 22 03:07:39.827652 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.827624 2556 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 22 03:07:39.827788 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.827654 2556 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-13.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 22 03:07:39.828278 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.828268 2556 topology_manager.go:138] "Creating topology manager with none policy" Jul 22 03:07:39.828309 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.828281 2556 container_manager_linux.go:309] "Creating device plugin manager" Jul 22 03:07:39.828309 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.828289 2556 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 03:07:39.828857 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.828847 2556 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 03:07:39.829789 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.829779 2556 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 22 03:07:39.833288 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.833276 2556 state_mem.go:36] "Initialized new in-memory state store" Jul 22 03:07:39.833406 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.833396 2556 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 22 03:07:39.833978 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.833969 2556 kubelet.go:486] "Attempting to sync node with API server" Jul 22 03:07:39.834009 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.833982 2556 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 22 03:07:39.834009 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.833992 2556 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 22 03:07:39.834009 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.834001 2556 kubelet.go:398] "Adding apiserver pod source" Jul 22 03:07:39.834129 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.834050 2556 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 22 03:07:39.834360 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.834350 2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 03:07:39.834393 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.834369 2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 03:07:39.837854 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.837832 2556 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 22 03:07:39.839249 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.839235 2556 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 22 03:07:39.839315 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.839264 2556 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 22 03:07:39.840782 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.840770 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 22 03:07:39.840818 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.840787 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 22 03:07:39.840818 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.840796 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 22 03:07:39.840818 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.840805 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 22 03:07:39.840818 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.840817 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 22 03:07:39.840928 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.840823 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 22 03:07:39.840928 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.840831 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 22 03:07:39.840928 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.840837 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 22 03:07:39.840928 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.840844 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 22 03:07:39.840928 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.840849 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 22 03:07:39.840928 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.840863 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 22 03:07:39.840928 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.840869 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 22 03:07:39.842101 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.842090 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 22 03:07:39.842135 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.842101 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 22 03:07:39.845226 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.845214 2556 server.go:1268] "Started kubelet" Jul 22 03:07:39.845394 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.845353 2556 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 22 03:07:39.845489 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.845417 2556 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 22 03:07:39.845609 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.845546 2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 03:07:39.848247 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.848223 2556 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 22 03:07:39.848738 ip-10-0-133-13 systemd[1]: Started Kubernetes Kubelet. Jul 22 03:07:39.848971 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.848787 2556 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 22 03:07:39.850927 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:39.850902 2556 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 22 03:07:39.851024 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.850974 2556 server.go:310] "Adding debug handlers to kubelet server" Jul 22 03:07:39.851024 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.850973 2556 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-13.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 22 03:07:39.852452 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:39.852429 2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-13.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 22 03:07:39.852452 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:39.852431 2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 22 03:07:39.855713 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:39.854980 2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-13.ec2.internal.18c47dca3962acec default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-13.ec2.internal,UID:ip-10-0-133-13.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-13.ec2.internal,},FirstTimestamp:2026-07-22 03:07:39.84519294 +0000 UTC m=+0.294176180,LastTimestamp:2026-07-22 03:07:39.84519294 +0000 UTC m=+0.294176180,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-13.ec2.internal,}" Jul 22 03:07:39.856147 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.856131 2556 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 22 03:07:39.856233 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:39.856218 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-13.ec2.internal\" not found" Jul 22 03:07:39.856292 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.856132 2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 22 03:07:39.856573 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.856534 2556 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 22 03:07:39.856573 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.856534 2556 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 22 03:07:39.856573 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.856565 2556 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 22 03:07:39.856736 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.856668 2556 reconstruct.go:97] "Volume reconstruction finished" Jul 22 03:07:39.856736 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.856677 2556 reconciler.go:29] "Reconciler: start to sync state" Jul 22 03:07:39.857154 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.857141 2556 factory.go:55] Registering systemd factory Jul 22 03:07:39.857154 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.857155 2556 factory.go:223] Registration of the systemd container factory successfully Jul 22 03:07:39.857410 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.857395 2556 factory.go:153] Registering CRI-O factory Jul 22 03:07:39.857455 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.857413 2556 factory.go:223] Registration of the crio container factory successfully Jul 22 03:07:39.857500 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.857474 2556 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 22 03:07:39.857531 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.857499 2556 factory.go:103] Registering Raw factory Jul 22 03:07:39.857531 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.857515 2556 manager.go:1196] Started watching for new ooms in manager Jul 22 03:07:39.858209 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.858196 2556 manager.go:349] Starting recovery of all containers Jul 22 03:07:39.861990 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:39.861967 2556 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 22 03:07:39.862102 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:39.862038 2556 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-13.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 22 03:07:39.868405 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.868390 2556 manager.go:354] Recovery completed Jul 22 03:07:39.873582 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.873565 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 03:07:39.875223 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.875187 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-13.ec2.internal" event="NodeHasSufficientMemory" Jul 22 03:07:39.875317 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.875234 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-13.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 03:07:39.875317 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.875249 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-13.ec2.internal" event="NodeHasSufficientPID" Jul 22 03:07:39.875641 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.875625 2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qnmg6" Jul 22 03:07:39.875711 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.875699 2556 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 22 03:07:39.875760 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.875713 2556 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 22 03:07:39.875760 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.875723 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 03:07:39.875760 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.875729 2556 state_mem.go:36] "Initialized new in-memory state store" Jul 22 03:07:39.876379 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.876362 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-13.ec2.internal" event="NodeHasSufficientMemory" Jul 22 03:07:39.876447 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.876395 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-13.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 03:07:39.876447 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.876409 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-13.ec2.internal" event="NodeHasSufficientPID" Jul 22 03:07:39.877169 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:39.877104 2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-13.ec2.internal.18c47dca3b2ccddd default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-13.ec2.internal,UID:ip-10-0-133-13.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-13.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-13.ec2.internal,},FirstTimestamp:2026-07-22 03:07:39.875216861 +0000 UTC m=+0.324200105,LastTimestamp:2026-07-22 03:07:39.875216861 +0000 UTC m=+0.324200105,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-13.ec2.internal,}" Jul 22 03:07:39.877886 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.877874 2556 policy_none.go:49] "None policy: Start" Jul 22 03:07:39.877886 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.877889 2556 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 22 03:07:39.877994 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.877900 2556 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 22 03:07:39.879986 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.879974 2556 policy_none.go:47] "Start" Jul 22 03:07:39.883323 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.883298 2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qnmg6" Jul 22 03:07:39.887358 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:39.887281 2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-13.ec2.internal.18c47dca3b2d2e00 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-13.ec2.internal,UID:ip-10-0-133-13.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-13.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-13.ec2.internal,},FirstTimestamp:2026-07-22 03:07:39.875241472 +0000 UTC m=+0.324224713,LastTimestamp:2026-07-22 03:07:39.875241472 +0000 UTC m=+0.324224713,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-13.ec2.internal,}" Jul 22 03:07:39.911466 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.911449 2556 manager.go:337] "Starting Device Plugin manager" Jul 22 03:07:39.911521 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:39.911485 2556 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 22 03:07:39.911521 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.911495 2556 server.go:87] "Starting device plugin registration server" Jul 22 03:07:39.911722 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.911712 2556 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 22 03:07:39.911771 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.911723 2556 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 22 03:07:39.911860 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.911829 2556 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 22 03:07:39.911930 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.911921 2556 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 22 03:07:39.912001 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:39.911930 2556 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 22 03:07:39.912278 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:39.912263 2556 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 22 03:07:39.912356 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:39.912308 2556 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-13.ec2.internal\" not found" Jul 22 03:07:40.012415 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.012381 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 03:07:40.015123 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.015098 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-13.ec2.internal" event="NodeHasSufficientMemory" Jul 22 03:07:40.015123 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.015135 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-13.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 03:07:40.015306 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.015150 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-13.ec2.internal" event="NodeHasSufficientPID" Jul 22 03:07:40.015306 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.015183 2556 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-13.ec2.internal" Jul 22 03:07:40.016305 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.016283 2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 22 03:07:40.017405 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.017386 2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 22 03:07:40.017488 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.017426 2556 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 22 03:07:40.017488 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.017452 2556 kubelet.go:2448] "Starting kubelet main sync loop" Jul 22 03:07:40.017488 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:40.017484 2556 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 22 03:07:40.019717 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.019699 2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 03:07:40.025199 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.025184 2556 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-13.ec2.internal" Jul 22 03:07:40.025279 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:40.025205 2556 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-13.ec2.internal\": node \"ip-10-0-133-13.ec2.internal\" not found" Jul 22 03:07:40.073293 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:40.073229 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-13.ec2.internal\" not found" Jul 22 03:07:40.118005 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.117970 2556 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-13.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-13.ec2.internal"] Jul 22 03:07:40.118100 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.118050 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 03:07:40.119617 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.119599 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-13.ec2.internal" event="NodeHasSufficientMemory" Jul 22 03:07:40.119697 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.119631 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-13.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 03:07:40.119697 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.119642 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-13.ec2.internal" event="NodeHasSufficientPID" Jul 22 03:07:40.121878 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.121866 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 03:07:40.122022 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.122007 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-13.ec2.internal" Jul 22 03:07:40.122120 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.122042 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 03:07:40.122636 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.122621 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-13.ec2.internal" event="NodeHasSufficientMemory" Jul 22 03:07:40.122636 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.122645 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-13.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 03:07:40.122736 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.122655 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-13.ec2.internal" event="NodeHasSufficientPID" Jul 22 03:07:40.122736 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.122655 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-13.ec2.internal" event="NodeHasSufficientMemory" Jul 22 03:07:40.122736 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.122680 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-13.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 03:07:40.122736 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.122690 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-13.ec2.internal" event="NodeHasSufficientPID" Jul 22 03:07:40.124887 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.124873 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-13.ec2.internal" Jul 22 03:07:40.124937 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.124899 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 03:07:40.125626 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.125609 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-13.ec2.internal" event="NodeHasSufficientMemory" Jul 22 03:07:40.125672 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.125643 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-13.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 03:07:40.125672 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.125659 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-13.ec2.internal" event="NodeHasSufficientPID" Jul 22 03:07:40.145496 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:40.145479 2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-13.ec2.internal\" not found" node="ip-10-0-133-13.ec2.internal" Jul 22 03:07:40.148903 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:40.148881 2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-13.ec2.internal\" not found" node="ip-10-0-133-13.ec2.internal" Jul 22 03:07:40.158215 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.158198 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d5cf9dba7131053469ef1265ca066507-config\") pod \"kube-apiserver-proxy-ip-10-0-133-13.ec2.internal\" (UID: \"d5cf9dba7131053469ef1265ca066507\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-13.ec2.internal" Jul 22 03:07:40.158281 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.158223 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1ee6828dcab126e50933f520556b028c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-13.ec2.internal\" (UID: \"1ee6828dcab126e50933f520556b028c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-13.ec2.internal" Jul 22 03:07:40.158281 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.158241 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ee6828dcab126e50933f520556b028c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-13.ec2.internal\" (UID: \"1ee6828dcab126e50933f520556b028c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-13.ec2.internal" Jul 22 03:07:40.173917 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:40.173899 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-13.ec2.internal\" not found" Jul 22 03:07:40.258672 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.258643 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1ee6828dcab126e50933f520556b028c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-13.ec2.internal\" (UID: \"1ee6828dcab126e50933f520556b028c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-13.ec2.internal" Jul 22 03:07:40.258672 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.258671 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ee6828dcab126e50933f520556b028c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-13.ec2.internal\" (UID: \"1ee6828dcab126e50933f520556b028c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-13.ec2.internal" Jul 22 03:07:40.258821 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.258690 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d5cf9dba7131053469ef1265ca066507-config\") pod \"kube-apiserver-proxy-ip-10-0-133-13.ec2.internal\" (UID: \"d5cf9dba7131053469ef1265ca066507\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-13.ec2.internal" Jul 22 03:07:40.258821 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.258749 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1ee6828dcab126e50933f520556b028c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-13.ec2.internal\" (UID: \"1ee6828dcab126e50933f520556b028c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-13.ec2.internal" Jul 22 03:07:40.258821 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.258758 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ee6828dcab126e50933f520556b028c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-13.ec2.internal\" (UID: \"1ee6828dcab126e50933f520556b028c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-13.ec2.internal" Jul 22 03:07:40.258821 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.258810 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d5cf9dba7131053469ef1265ca066507-config\") pod \"kube-apiserver-proxy-ip-10-0-133-13.ec2.internal\" (UID: \"d5cf9dba7131053469ef1265ca066507\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-13.ec2.internal" Jul 22 03:07:40.274779 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:40.274757 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-13.ec2.internal\" not found" Jul 22 03:07:40.375564 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:40.375480 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-13.ec2.internal\" not found" Jul 22 03:07:40.448717 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.448694 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-13.ec2.internal" Jul 22 03:07:40.451794 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.451771 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-13.ec2.internal" Jul 22 03:07:40.476144 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:40.475987 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-13.ec2.internal\" not found" Jul 22 03:07:40.576461 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:40.576431 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-13.ec2.internal\" not found" Jul 22 03:07:40.677000 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:40.676911 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-13.ec2.internal\" not found" Jul 22 03:07:40.777521 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:40.777491 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-13.ec2.internal\" not found" Jul 22 03:07:40.799663 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.799636 2556 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 22 03:07:40.799787 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.799770 2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 22 03:07:40.877121 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.877096 2556 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 03:07:40.878097 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:40.878080 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-13.ec2.internal\" not found" Jul 22 03:07:40.889249 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.889225 2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-21 03:02:39 +0000 UTC" deadline="2028-04-18 02:22:42.333696808 +0000 UTC" Jul 22 03:07:40.889249 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:40.889246 2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15263h15m1.444452995s" Jul 22 03:07:40.978982 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:40.978893 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-13.ec2.internal\" not found" Jul 22 03:07:41.004035 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.004011 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 03:07:41.005307 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.005290 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-13.ec2.internal" event="NodeHasSufficientMemory" Jul 22 03:07:41.005401 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.005320 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-13.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 03:07:41.005401 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.005330 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-13.ec2.internal" event="NodeHasSufficientPID" Jul 22 03:07:41.022180 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:41.022150 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5cf9dba7131053469ef1265ca066507.slice/crio-0e0e85664bfd902ec5e0d590affd52a5da653176e77c3a40c820bf21157c7725 WatchSource:0}: Error finding container 0e0e85664bfd902ec5e0d590affd52a5da653176e77c3a40c820bf21157c7725: Status 404 returned error can't find the container with id 0e0e85664bfd902ec5e0d590affd52a5da653176e77c3a40c820bf21157c7725 Jul 22 03:07:41.022769 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:41.022750 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ee6828dcab126e50933f520556b028c.slice/crio-cb9d9e1ded369ec5e063e7054c9a234c14d86ee25627c108198092b95dc6b8b1 WatchSource:0}: Error finding container cb9d9e1ded369ec5e063e7054c9a234c14d86ee25627c108198092b95dc6b8b1: Status 404 returned error can't find the container with id cb9d9e1ded369ec5e063e7054c9a234c14d86ee25627c108198092b95dc6b8b1 Jul 22 03:07:41.028122 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.028107 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 03:07:41.079019 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:41.078994 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-13.ec2.internal\" not found" Jul 22 03:07:41.145280 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.145254 2556 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 03:07:41.179907 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:41.179887 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-13.ec2.internal\" not found" Jul 22 03:07:41.216655 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.216631 2556 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 03:07:41.257271 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.257208 2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-13.ec2.internal" Jul 22 03:07:41.270743 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.270721 2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 03:07:41.271807 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.271794 2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-13.ec2.internal" Jul 22 03:07:41.280581 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.280566 2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 03:07:41.835970 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.835919 2556 apiserver.go:52] "Watching apiserver" Jul 22 03:07:41.844779 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.844751 2556 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 22 03:07:41.845139 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.845111 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-13.ec2.internal","openshift-multus/multus-additional-cni-plugins-wth5q","openshift-multus/network-metrics-daemon-xrbdq","openshift-network-diagnostics/network-check-target-k4rxt","openshift-ovn-kubernetes/ovnkube-node-sscf8","kube-system/global-pull-secret-syncer-qhg45","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w85fv","openshift-cluster-node-tuning-operator/tuned-vr69p","openshift-dns/node-resolver-kkcg5","openshift-multus/multus-fwk5q","openshift-network-operator/iptables-alerter-n94lh","kube-system/konnectivity-agent-68xqw","kube-system/kube-apiserver-proxy-ip-10-0-133-13.ec2.internal","openshift-image-registry/node-ca-rv678"] Jul 22 03:07:41.847744 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.847710 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w85fv" Jul 22 03:07:41.849702 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.849683 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-n94lh" Jul 22 03:07:41.850534 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.850519 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 22 03:07:41.850621 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.850556 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 22 03:07:41.850621 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.850558 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 22 03:07:41.850726 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.850635 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-864pd\"" Jul 22 03:07:41.851718 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.851702 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrbdq" Jul 22 03:07:41.851809 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:41.851785 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrbdq" podUID="bb8b43a6-b50f-41e9-9e68-763231228640" Jul 22 03:07:41.852342 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.852327 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 22 03:07:41.852342 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.852335 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 22 03:07:41.852472 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.852353 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-n27g6\"" Jul 22 03:07:41.852620 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.852606 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 22 03:07:41.853828 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.853812 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4rxt" Jul 22 03:07:41.853915 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:41.853885 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k4rxt" podUID="1c2c0875-8ece-494b-95a1-00fe315db1e4" Jul 22 03:07:41.856257 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.856236 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.856527 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.856498 2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 22 03:07:41.858528 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.858510 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qhg45" Jul 22 03:07:41.858623 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:41.858565 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qhg45" podUID="1297a647-da14-4c7b-bf94-6f4f4e679c31" Jul 22 03:07:41.859175 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.859149 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 22 03:07:41.859279 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.859155 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 22 03:07:41.859279 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.859246 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-ft5h7\"" Jul 22 03:07:41.859396 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.859156 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 22 03:07:41.859396 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.859252 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 22 03:07:41.859396 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.859155 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 22 03:07:41.860257 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.860122 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 22 03:07:41.861053 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.860796 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kkcg5" Jul 22 03:07:41.863558 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.863540 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 22 03:07:41.863638 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.863609 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 22 03:07:41.863702 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.863637 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-l6phf\"" Jul 22 03:07:41.864872 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.864854 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rv678" Jul 22 03:07:41.865851 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.865829 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-run-openvswitch\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.865963 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.865862 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-host-cni-bin\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.865963 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.865885 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a01dfba4-f590-412a-8438-1646c51bb1a1-ovnkube-config\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.865963 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.865907 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/84476432-16cb-4559-bf1f-4fe42ea2e87a-device-dir\") pod \"aws-ebs-csi-driver-node-w85fv\" (UID: \"84476432-16cb-4559-bf1f-4fe42ea2e87a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w85fv" Jul 22 03:07:41.865963 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.865937 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/84476432-16cb-4559-bf1f-4fe42ea2e87a-etc-selinux\") pod \"aws-ebs-csi-driver-node-w85fv\" (UID: \"84476432-16cb-4559-bf1f-4fe42ea2e87a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w85fv" Jul 22 03:07:41.866177 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.866018 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/84476432-16cb-4559-bf1f-4fe42ea2e87a-sys-fs\") pod \"aws-ebs-csi-driver-node-w85fv\" (UID: \"84476432-16cb-4559-bf1f-4fe42ea2e87a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w85fv" Jul 22 03:07:41.866177 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.866054 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a01dfba4-f590-412a-8438-1646c51bb1a1-ovn-node-metrics-cert\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.866177 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.866081 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a01dfba4-f590-412a-8438-1646c51bb1a1-ovnkube-script-lib\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.866177 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.866106 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-host-run-ovn-kubernetes\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.866177 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.866127 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-host-cni-netd\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.866177 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.866147 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqjsf\" (UniqueName: \"kubernetes.io/projected/bb8b43a6-b50f-41e9-9e68-763231228640-kube-api-access-cqjsf\") pod \"network-metrics-daemon-xrbdq\" (UID: \"bb8b43a6-b50f-41e9-9e68-763231228640\") " pod="openshift-multus/network-metrics-daemon-xrbdq" Jul 22 03:07:41.866177 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.866161 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x25qb\" (UniqueName: \"kubernetes.io/projected/1c2c0875-8ece-494b-95a1-00fe315db1e4-kube-api-access-x25qb\") pod \"network-check-target-k4rxt\" (UID: \"1c2c0875-8ece-494b-95a1-00fe315db1e4\") " pod="openshift-network-diagnostics/network-check-target-k4rxt" Jul 22 03:07:41.866177 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.866176 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-run-ovn\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.866536 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.866197 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1297a647-da14-4c7b-bf94-6f4f4e679c31-original-pull-secret\") pod \"global-pull-secret-syncer-qhg45\" (UID: \"1297a647-da14-4c7b-bf94-6f4f4e679c31\") " pod="kube-system/global-pull-secret-syncer-qhg45" Jul 22 03:07:41.866536 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.866213 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7bfceba5-9ca9-4955-abfd-1c029a7e6062-iptables-alerter-script\") pod \"iptables-alerter-n94lh\" (UID: \"7bfceba5-9ca9-4955-abfd-1c029a7e6062\") " pod="openshift-network-operator/iptables-alerter-n94lh" Jul 22 03:07:41.866536 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.866228 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7ld8\" (UniqueName: \"kubernetes.io/projected/7bfceba5-9ca9-4955-abfd-1c029a7e6062-kube-api-access-m7ld8\") pod \"iptables-alerter-n94lh\" (UID: \"7bfceba5-9ca9-4955-abfd-1c029a7e6062\") " pod="openshift-network-operator/iptables-alerter-n94lh" Jul 22 03:07:41.866536 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.866242 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb8b43a6-b50f-41e9-9e68-763231228640-metrics-certs\") pod \"network-metrics-daemon-xrbdq\" (UID: \"bb8b43a6-b50f-41e9-9e68-763231228640\") " pod="openshift-multus/network-metrics-daemon-xrbdq" Jul 22 03:07:41.866536 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.866270 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-host-kubelet\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.866536 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.866285 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a01dfba4-f590-412a-8438-1646c51bb1a1-env-overrides\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.866536 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.866297 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/84476432-16cb-4559-bf1f-4fe42ea2e87a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w85fv\" (UID: \"84476432-16cb-4559-bf1f-4fe42ea2e87a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w85fv" Jul 22 03:07:41.866536 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.866323 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-systemd-units\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.866536 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.866343 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-run-systemd\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.866536 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.866358 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-var-lib-openvswitch\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.866536 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.866372 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1297a647-da14-4c7b-bf94-6f4f4e679c31-kubelet-config\") pod \"global-pull-secret-syncer-qhg45\" (UID: \"1297a647-da14-4c7b-bf94-6f4f4e679c31\") " pod="kube-system/global-pull-secret-syncer-qhg45" Jul 22 03:07:41.866536 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.866388 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1297a647-da14-4c7b-bf94-6f4f4e679c31-dbus\") pod \"global-pull-secret-syncer-qhg45\" (UID: \"1297a647-da14-4c7b-bf94-6f4f4e679c31\") " pod="kube-system/global-pull-secret-syncer-qhg45" Jul 22 03:07:41.866536 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.866416 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-host-slash\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.866536 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.866459 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-host-run-netns\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.866536 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.866488 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-node-log\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.867267 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.866542 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-log-socket\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.867267 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.866570 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.867267 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.866587 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/84476432-16cb-4559-bf1f-4fe42ea2e87a-registration-dir\") pod \"aws-ebs-csi-driver-node-w85fv\" (UID: \"84476432-16cb-4559-bf1f-4fe42ea2e87a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w85fv" Jul 22 03:07:41.867267 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.866602 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kct5z\" (UniqueName: \"kubernetes.io/projected/a01dfba4-f590-412a-8438-1646c51bb1a1-kube-api-access-kct5z\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.867267 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.866617 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/84476432-16cb-4559-bf1f-4fe42ea2e87a-socket-dir\") pod \"aws-ebs-csi-driver-node-w85fv\" (UID: \"84476432-16cb-4559-bf1f-4fe42ea2e87a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w85fv" Jul 22 03:07:41.867267 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.866631 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txdlt\" (UniqueName: \"kubernetes.io/projected/84476432-16cb-4559-bf1f-4fe42ea2e87a-kube-api-access-txdlt\") pod \"aws-ebs-csi-driver-node-w85fv\" (UID: \"84476432-16cb-4559-bf1f-4fe42ea2e87a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w85fv" Jul 22 03:07:41.867267 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.866644 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7bfceba5-9ca9-4955-abfd-1c029a7e6062-host-slash\") pod \"iptables-alerter-n94lh\" (UID: \"7bfceba5-9ca9-4955-abfd-1c029a7e6062\") " pod="openshift-network-operator/iptables-alerter-n94lh" Jul 22 03:07:41.867267 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.866660 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-etc-openvswitch\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.867267 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.867164 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 22 03:07:41.867713 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.867481 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wth5q" Jul 22 03:07:41.867713 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.867707 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 22 03:07:41.868396 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.868376 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 22 03:07:41.869524 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.869501 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-v2wvj\"" Jul 22 03:07:41.870648 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.870629 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 22 03:07:41.870726 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.870681 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 22 03:07:41.870921 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.870901 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qc4g7\"" Jul 22 03:07:41.871695 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.871670 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 22 03:07:41.871799 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.871733 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fwk5q" Jul 22 03:07:41.871908 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.871890 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 22 03:07:41.871999 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.871941 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 22 03:07:41.872088 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.872066 2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 03:07:41.874233 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.874216 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:41.875063 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.875035 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 22 03:07:41.875063 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.875050 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-gwx5k\"" Jul 22 03:07:41.876433 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.876415 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-68xqw" Jul 22 03:07:41.876889 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.876871 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-n49s5\"" Jul 22 03:07:41.876986 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.876896 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 22 03:07:41.877044 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.876899 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 22 03:07:41.879035 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.878857 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 22 03:07:41.879035 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.878999 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 22 03:07:41.879542 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.879523 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-f2gv4\"" Jul 22 03:07:41.897158 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.897133 2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-64lgb" Jul 22 03:07:41.905905 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.905882 2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-64lgb" Jul 22 03:07:41.957530 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.957500 2556 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 22 03:07:41.967057 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.967024 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a01dfba4-f590-412a-8438-1646c51bb1a1-ovn-node-metrics-cert\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.967183 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.967058 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a01dfba4-f590-412a-8438-1646c51bb1a1-ovnkube-script-lib\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.967183 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.967083 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-host-run-ovn-kubernetes\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.967183 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.967102 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-host-cni-netd\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.967183 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.967128 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-var-lib-kubelet\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:41.967183 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.967150 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-etc-tuned\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:41.967380 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.967179 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-host-run-ovn-kubernetes\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.967380 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.967176 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/db34ad96-aec1-457b-8c55-61fea9c7f14a-agent-certs\") pod \"konnectivity-agent-68xqw\" (UID: \"db34ad96-aec1-457b-8c55-61fea9c7f14a\") " pod="kube-system/konnectivity-agent-68xqw" Jul 22 03:07:41.967380 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.967240 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-host-cni-netd\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.967380 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.967251 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1297a647-da14-4c7b-bf94-6f4f4e679c31-original-pull-secret\") pod \"global-pull-secret-syncer-qhg45\" (UID: \"1297a647-da14-4c7b-bf94-6f4f4e679c31\") " pod="kube-system/global-pull-secret-syncer-qhg45" Jul 22 03:07:41.967380 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.967273 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7ld8\" (UniqueName: \"kubernetes.io/projected/7bfceba5-9ca9-4955-abfd-1c029a7e6062-kube-api-access-m7ld8\") pod \"iptables-alerter-n94lh\" (UID: \"7bfceba5-9ca9-4955-abfd-1c029a7e6062\") " pod="openshift-network-operator/iptables-alerter-n94lh" Jul 22 03:07:41.967380 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.967303 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb8b43a6-b50f-41e9-9e68-763231228640-metrics-certs\") pod \"network-metrics-daemon-xrbdq\" (UID: \"bb8b43a6-b50f-41e9-9e68-763231228640\") " pod="openshift-multus/network-metrics-daemon-xrbdq" Jul 22 03:07:41.967380 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.967330 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-lib-modules\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:41.967380 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.967357 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b9f02261-c5da-4a76-aa41-e09c6955ef76-os-release\") pod \"multus-additional-cni-plugins-wth5q\" (UID: \"b9f02261-c5da-4a76-aa41-e09c6955ef76\") " pod="openshift-multus/multus-additional-cni-plugins-wth5q" Jul 22 03:07:41.967690 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.967415 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-hostroot\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:41.967690 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:41.967430 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 03:07:41.967690 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.967438 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-host-run-multus-certs\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:41.967690 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.967465 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-host-kubelet\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.967690 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:41.967499 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb8b43a6-b50f-41e9-9e68-763231228640-metrics-certs podName:bb8b43a6-b50f-41e9-9e68-763231228640 nodeName:}" failed. No retries permitted until 2026-07-22 03:07:42.467477726 +0000 UTC m=+2.916460972 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bb8b43a6-b50f-41e9-9e68-763231228640-metrics-certs") pod "network-metrics-daemon-xrbdq" (UID: "bb8b43a6-b50f-41e9-9e68-763231228640") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 03:07:41.967690 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.967504 2556 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 22 03:07:41.967690 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.967525 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-etc-modprobe-d\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:41.967690 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:41.967571 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 03:07:41.967690 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:41.967619 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1297a647-da14-4c7b-bf94-6f4f4e679c31-original-pull-secret podName:1297a647-da14-4c7b-bf94-6f4f4e679c31 nodeName:}" failed. No retries permitted until 2026-07-22 03:07:42.467605983 +0000 UTC m=+2.916589223 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1297a647-da14-4c7b-bf94-6f4f4e679c31-original-pull-secret") pod "global-pull-secret-syncer-qhg45" (UID: "1297a647-da14-4c7b-bf94-6f4f4e679c31") : object "kube-system"/"original-pull-secret" not registered Jul 22 03:07:41.968178 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.967702 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-host-kubelet\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.968178 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.967748 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-etc-sysctl-conf\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:41.968178 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.967774 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a01dfba4-f590-412a-8438-1646c51bb1a1-ovnkube-script-lib\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.968178 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.967795 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-tmp\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:41.968178 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.967835 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcgrv\" (UniqueName: \"kubernetes.io/projected/590f0095-ecb1-43e4-9c1e-1c62a87a3b8a-kube-api-access-bcgrv\") pod \"node-ca-rv678\" (UID: \"590f0095-ecb1-43e4-9c1e-1c62a87a3b8a\") " pod="openshift-image-registry/node-ca-rv678" Jul 22 03:07:41.968178 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.967864 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-systemd-units\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.968178 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.967926 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-run-systemd\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.968178 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.967993 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-run-systemd\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.968178 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.967998 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-systemd-units\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.968178 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.968004 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-var-lib-openvswitch\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.968178 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.968046 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-etc-kubernetes\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:41.968178 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.968080 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-etc-systemd\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:41.968178 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.968107 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-var-lib-openvswitch\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.968178 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.968153 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-multus-socket-dir-parent\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:41.968178 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.968180 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jws4\" (UniqueName: \"kubernetes.io/projected/734b270b-0f16-4bba-9cd3-7a41b919a010-kube-api-access-5jws4\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:41.968753 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.968199 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-etc-sysctl-d\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:41.968753 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.968224 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-node-log\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.968753 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.968244 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-log-socket\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.968753 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.968282 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-node-log\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.968753 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.968286 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-log-socket\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.968753 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.968333 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.968753 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.968359 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/84476432-16cb-4559-bf1f-4fe42ea2e87a-registration-dir\") pod \"aws-ebs-csi-driver-node-w85fv\" (UID: \"84476432-16cb-4559-bf1f-4fe42ea2e87a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w85fv" Jul 22 03:07:41.968753 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.968377 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7rb9\" (UniqueName: \"kubernetes.io/projected/497ea3ce-7ad8-42f9-a52e-8094ec646cc9-kube-api-access-l7rb9\") pod \"node-resolver-kkcg5\" (UID: \"497ea3ce-7ad8-42f9-a52e-8094ec646cc9\") " pod="openshift-dns/node-resolver-kkcg5" Jul 22 03:07:41.968753 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.968394 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-host-run-netns\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:41.968753 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.968418 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-host-var-lib-kubelet\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:41.968753 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.968424 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.968753 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.968450 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/84476432-16cb-4559-bf1f-4fe42ea2e87a-registration-dir\") pod \"aws-ebs-csi-driver-node-w85fv\" (UID: \"84476432-16cb-4559-bf1f-4fe42ea2e87a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w85fv" Jul 22 03:07:41.968753 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.968453 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kct5z\" (UniqueName: \"kubernetes.io/projected/a01dfba4-f590-412a-8438-1646c51bb1a1-kube-api-access-kct5z\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.968753 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.968515 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txdlt\" (UniqueName: \"kubernetes.io/projected/84476432-16cb-4559-bf1f-4fe42ea2e87a-kube-api-access-txdlt\") pod \"aws-ebs-csi-driver-node-w85fv\" (UID: \"84476432-16cb-4559-bf1f-4fe42ea2e87a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w85fv" Jul 22 03:07:41.968753 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.968557 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-sys\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:41.968753 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.968574 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b9f02261-c5da-4a76-aa41-e09c6955ef76-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wth5q\" (UID: \"b9f02261-c5da-4a76-aa41-e09c6955ef76\") " pod="openshift-multus/multus-additional-cni-plugins-wth5q" Jul 22 03:07:41.968753 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.968590 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-host-var-lib-cni-bin\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:41.969568 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.968607 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-host-var-lib-cni-multus\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:41.969568 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.968630 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/497ea3ce-7ad8-42f9-a52e-8094ec646cc9-tmp-dir\") pod \"node-resolver-kkcg5\" (UID: \"497ea3ce-7ad8-42f9-a52e-8094ec646cc9\") " pod="openshift-dns/node-resolver-kkcg5" Jul 22 03:07:41.969568 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.968647 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/590f0095-ecb1-43e4-9c1e-1c62a87a3b8a-serviceca\") pod \"node-ca-rv678\" (UID: \"590f0095-ecb1-43e4-9c1e-1c62a87a3b8a\") " pod="openshift-image-registry/node-ca-rv678" Jul 22 03:07:41.969568 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.968662 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b9f02261-c5da-4a76-aa41-e09c6955ef76-system-cni-dir\") pod \"multus-additional-cni-plugins-wth5q\" (UID: \"b9f02261-c5da-4a76-aa41-e09c6955ef76\") " pod="openshift-multus/multus-additional-cni-plugins-wth5q" Jul 22 03:07:41.969568 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.968680 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-host-cni-bin\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.969568 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.968702 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a01dfba4-f590-412a-8438-1646c51bb1a1-ovnkube-config\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.969568 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.968724 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/84476432-16cb-4559-bf1f-4fe42ea2e87a-device-dir\") pod \"aws-ebs-csi-driver-node-w85fv\" (UID: \"84476432-16cb-4559-bf1f-4fe42ea2e87a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w85fv" Jul 22 03:07:41.969568 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.968802 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/84476432-16cb-4559-bf1f-4fe42ea2e87a-device-dir\") pod \"aws-ebs-csi-driver-node-w85fv\" (UID: \"84476432-16cb-4559-bf1f-4fe42ea2e87a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w85fv" Jul 22 03:07:41.969568 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.968827 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-host-cni-bin\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.969568 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.968866 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/84476432-16cb-4559-bf1f-4fe42ea2e87a-etc-selinux\") pod \"aws-ebs-csi-driver-node-w85fv\" (UID: \"84476432-16cb-4559-bf1f-4fe42ea2e87a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w85fv" Jul 22 03:07:41.969568 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.968898 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/84476432-16cb-4559-bf1f-4fe42ea2e87a-sys-fs\") pod \"aws-ebs-csi-driver-node-w85fv\" (UID: \"84476432-16cb-4559-bf1f-4fe42ea2e87a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w85fv" Jul 22 03:07:41.969568 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.968927 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-os-release\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:41.969568 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.968967 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/734b270b-0f16-4bba-9cd3-7a41b919a010-cni-binary-copy\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:41.969568 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.968994 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qs25k\" (UniqueName: \"kubernetes.io/projected/b9f02261-c5da-4a76-aa41-e09c6955ef76-kube-api-access-qs25k\") pod \"multus-additional-cni-plugins-wth5q\" (UID: \"b9f02261-c5da-4a76-aa41-e09c6955ef76\") " pod="openshift-multus/multus-additional-cni-plugins-wth5q" Jul 22 03:07:41.969568 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969000 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/84476432-16cb-4559-bf1f-4fe42ea2e87a-etc-selinux\") pod \"aws-ebs-csi-driver-node-w85fv\" (UID: \"84476432-16cb-4559-bf1f-4fe42ea2e87a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w85fv" Jul 22 03:07:41.969568 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969018 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-multus-cni-dir\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:41.969568 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969050 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-cnibin\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:41.970337 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969075 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/734b270b-0f16-4bba-9cd3-7a41b919a010-multus-daemon-config\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:41.970337 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969092 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/84476432-16cb-4559-bf1f-4fe42ea2e87a-sys-fs\") pod \"aws-ebs-csi-driver-node-w85fv\" (UID: \"84476432-16cb-4559-bf1f-4fe42ea2e87a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w85fv" Jul 22 03:07:41.970337 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969102 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x25qb\" (UniqueName: \"kubernetes.io/projected/1c2c0875-8ece-494b-95a1-00fe315db1e4-kube-api-access-x25qb\") pod \"network-check-target-k4rxt\" (UID: \"1c2c0875-8ece-494b-95a1-00fe315db1e4\") " pod="openshift-network-diagnostics/network-check-target-k4rxt" Jul 22 03:07:41.970337 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969127 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-run-ovn\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.970337 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969160 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7bfceba5-9ca9-4955-abfd-1c029a7e6062-iptables-alerter-script\") pod \"iptables-alerter-n94lh\" (UID: \"7bfceba5-9ca9-4955-abfd-1c029a7e6062\") " pod="openshift-network-operator/iptables-alerter-n94lh" Jul 22 03:07:41.970337 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969185 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-system-cni-dir\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:41.970337 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969193 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a01dfba4-f590-412a-8438-1646c51bb1a1-ovnkube-config\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.970337 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969209 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/db34ad96-aec1-457b-8c55-61fea9c7f14a-konnectivity-ca\") pod \"konnectivity-agent-68xqw\" (UID: \"db34ad96-aec1-457b-8c55-61fea9c7f14a\") " pod="kube-system/konnectivity-agent-68xqw" Jul 22 03:07:41.970337 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969235 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a01dfba4-f590-412a-8438-1646c51bb1a1-env-overrides\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.970337 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969257 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-run-ovn\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.970337 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969259 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-etc-sysconfig\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:41.970337 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969297 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xx24f\" (UniqueName: \"kubernetes.io/projected/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-kube-api-access-xx24f\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:41.970337 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969313 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/590f0095-ecb1-43e4-9c1e-1c62a87a3b8a-host\") pod \"node-ca-rv678\" (UID: \"590f0095-ecb1-43e4-9c1e-1c62a87a3b8a\") " pod="openshift-image-registry/node-ca-rv678" Jul 22 03:07:41.970337 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969331 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-etc-kubernetes\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:41.970337 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969351 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/497ea3ce-7ad8-42f9-a52e-8094ec646cc9-hosts-file\") pod \"node-resolver-kkcg5\" (UID: \"497ea3ce-7ad8-42f9-a52e-8094ec646cc9\") " pod="openshift-dns/node-resolver-kkcg5" Jul 22 03:07:41.970337 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969367 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/84476432-16cb-4559-bf1f-4fe42ea2e87a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w85fv\" (UID: \"84476432-16cb-4559-bf1f-4fe42ea2e87a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w85fv" Jul 22 03:07:41.970337 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969385 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-run\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:41.971164 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969406 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-host\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:41.971164 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969427 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b9f02261-c5da-4a76-aa41-e09c6955ef76-cni-binary-copy\") pod \"multus-additional-cni-plugins-wth5q\" (UID: \"b9f02261-c5da-4a76-aa41-e09c6955ef76\") " pod="openshift-multus/multus-additional-cni-plugins-wth5q" Jul 22 03:07:41.971164 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969449 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1297a647-da14-4c7b-bf94-6f4f4e679c31-kubelet-config\") pod \"global-pull-secret-syncer-qhg45\" (UID: \"1297a647-da14-4c7b-bf94-6f4f4e679c31\") " pod="kube-system/global-pull-secret-syncer-qhg45" Jul 22 03:07:41.971164 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969476 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1297a647-da14-4c7b-bf94-6f4f4e679c31-dbus\") pod \"global-pull-secret-syncer-qhg45\" (UID: \"1297a647-da14-4c7b-bf94-6f4f4e679c31\") " pod="kube-system/global-pull-secret-syncer-qhg45" Jul 22 03:07:41.971164 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969497 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-host-slash\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.971164 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969515 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-host-run-netns\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.971164 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969531 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b9f02261-c5da-4a76-aa41-e09c6955ef76-cnibin\") pod \"multus-additional-cni-plugins-wth5q\" (UID: \"b9f02261-c5da-4a76-aa41-e09c6955ef76\") " pod="openshift-multus/multus-additional-cni-plugins-wth5q" Jul 22 03:07:41.971164 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969549 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-host-run-k8s-cni-cncf-io\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:41.971164 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969565 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/84476432-16cb-4559-bf1f-4fe42ea2e87a-socket-dir\") pod \"aws-ebs-csi-driver-node-w85fv\" (UID: \"84476432-16cb-4559-bf1f-4fe42ea2e87a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w85fv" Jul 22 03:07:41.971164 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969608 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b9f02261-c5da-4a76-aa41-e09c6955ef76-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wth5q\" (UID: \"b9f02261-c5da-4a76-aa41-e09c6955ef76\") " pod="openshift-multus/multus-additional-cni-plugins-wth5q" Jul 22 03:07:41.971164 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969627 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b9f02261-c5da-4a76-aa41-e09c6955ef76-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wth5q\" (UID: \"b9f02261-c5da-4a76-aa41-e09c6955ef76\") " pod="openshift-multus/multus-additional-cni-plugins-wth5q" Jul 22 03:07:41.971164 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969635 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a01dfba4-f590-412a-8438-1646c51bb1a1-env-overrides\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.971164 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969655 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-multus-conf-dir\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:41.971164 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969681 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7bfceba5-9ca9-4955-abfd-1c029a7e6062-host-slash\") pod \"iptables-alerter-n94lh\" (UID: \"7bfceba5-9ca9-4955-abfd-1c029a7e6062\") " pod="openshift-network-operator/iptables-alerter-n94lh" Jul 22 03:07:41.971164 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969699 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1297a647-da14-4c7b-bf94-6f4f4e679c31-kubelet-config\") pod \"global-pull-secret-syncer-qhg45\" (UID: \"1297a647-da14-4c7b-bf94-6f4f4e679c31\") " pod="kube-system/global-pull-secret-syncer-qhg45" Jul 22 03:07:41.971164 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969707 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-etc-openvswitch\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.971164 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969741 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-run-openvswitch\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.971879 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969763 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqjsf\" (UniqueName: \"kubernetes.io/projected/bb8b43a6-b50f-41e9-9e68-763231228640-kube-api-access-cqjsf\") pod \"network-metrics-daemon-xrbdq\" (UID: \"bb8b43a6-b50f-41e9-9e68-763231228640\") " pod="openshift-multus/network-metrics-daemon-xrbdq" Jul 22 03:07:41.971879 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969773 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/84476432-16cb-4559-bf1f-4fe42ea2e87a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w85fv\" (UID: \"84476432-16cb-4559-bf1f-4fe42ea2e87a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w85fv" Jul 22 03:07:41.971879 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969754 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7bfceba5-9ca9-4955-abfd-1c029a7e6062-iptables-alerter-script\") pod \"iptables-alerter-n94lh\" (UID: \"7bfceba5-9ca9-4955-abfd-1c029a7e6062\") " pod="openshift-network-operator/iptables-alerter-n94lh" Jul 22 03:07:41.971879 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969831 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-host-slash\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.971879 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969852 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-etc-openvswitch\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.971879 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.969889 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-run-openvswitch\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.971879 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.970002 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/84476432-16cb-4559-bf1f-4fe42ea2e87a-socket-dir\") pod \"aws-ebs-csi-driver-node-w85fv\" (UID: \"84476432-16cb-4559-bf1f-4fe42ea2e87a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w85fv" Jul 22 03:07:41.971879 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.970022 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1297a647-da14-4c7b-bf94-6f4f4e679c31-dbus\") pod \"global-pull-secret-syncer-qhg45\" (UID: \"1297a647-da14-4c7b-bf94-6f4f4e679c31\") " pod="kube-system/global-pull-secret-syncer-qhg45" Jul 22 03:07:41.971879 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.970052 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a01dfba4-f590-412a-8438-1646c51bb1a1-host-run-netns\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.971879 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.970067 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7bfceba5-9ca9-4955-abfd-1c029a7e6062-host-slash\") pod \"iptables-alerter-n94lh\" (UID: \"7bfceba5-9ca9-4955-abfd-1c029a7e6062\") " pod="openshift-network-operator/iptables-alerter-n94lh" Jul 22 03:07:41.971879 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.971675 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a01dfba4-f590-412a-8438-1646c51bb1a1-ovn-node-metrics-cert\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:41.976712 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.976680 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7ld8\" (UniqueName: \"kubernetes.io/projected/7bfceba5-9ca9-4955-abfd-1c029a7e6062-kube-api-access-m7ld8\") pod \"iptables-alerter-n94lh\" (UID: \"7bfceba5-9ca9-4955-abfd-1c029a7e6062\") " pod="openshift-network-operator/iptables-alerter-n94lh" Jul 22 03:07:41.977697 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:41.977579 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 03:07:41.977697 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:41.977602 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 03:07:41.977697 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:41.977616 2556 projected.go:196] Error preparing data for projected volume kube-api-access-x25qb for pod openshift-network-diagnostics/network-check-target-k4rxt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 03:07:41.977697 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:41.977673 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c2c0875-8ece-494b-95a1-00fe315db1e4-kube-api-access-x25qb podName:1c2c0875-8ece-494b-95a1-00fe315db1e4 nodeName:}" failed. No retries permitted until 2026-07-22 03:07:42.477655091 +0000 UTC m=+2.926638335 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-x25qb" (UniqueName: "kubernetes.io/projected/1c2c0875-8ece-494b-95a1-00fe315db1e4-kube-api-access-x25qb") pod "network-check-target-k4rxt" (UID: "1c2c0875-8ece-494b-95a1-00fe315db1e4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 03:07:41.980358 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.980292 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txdlt\" (UniqueName: \"kubernetes.io/projected/84476432-16cb-4559-bf1f-4fe42ea2e87a-kube-api-access-txdlt\") pod \"aws-ebs-csi-driver-node-w85fv\" (UID: \"84476432-16cb-4559-bf1f-4fe42ea2e87a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w85fv" Jul 22 03:07:41.981052 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.981008 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqjsf\" (UniqueName: \"kubernetes.io/projected/bb8b43a6-b50f-41e9-9e68-763231228640-kube-api-access-cqjsf\") pod \"network-metrics-daemon-xrbdq\" (UID: \"bb8b43a6-b50f-41e9-9e68-763231228640\") " pod="openshift-multus/network-metrics-daemon-xrbdq" Jul 22 03:07:41.983822 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:41.981398 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kct5z\" (UniqueName: \"kubernetes.io/projected/a01dfba4-f590-412a-8438-1646c51bb1a1-kube-api-access-kct5z\") pod \"ovnkube-node-sscf8\" (UID: \"a01dfba4-f590-412a-8438-1646c51bb1a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:42.022254 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.022206 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-13.ec2.internal" event={"ID":"1ee6828dcab126e50933f520556b028c","Type":"ContainerStarted","Data":"cb9d9e1ded369ec5e063e7054c9a234c14d86ee25627c108198092b95dc6b8b1"} Jul 22 03:07:42.023237 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.023211 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-13.ec2.internal" event={"ID":"d5cf9dba7131053469ef1265ca066507","Type":"ContainerStarted","Data":"0e0e85664bfd902ec5e0d590affd52a5da653176e77c3a40c820bf21157c7725"} Jul 22 03:07:42.071070 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071036 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-system-cni-dir\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:42.071070 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071074 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/db34ad96-aec1-457b-8c55-61fea9c7f14a-konnectivity-ca\") pod \"konnectivity-agent-68xqw\" (UID: \"db34ad96-aec1-457b-8c55-61fea9c7f14a\") " pod="kube-system/konnectivity-agent-68xqw" Jul 22 03:07:42.071276 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071091 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-etc-sysconfig\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:42.071276 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071107 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xx24f\" (UniqueName: \"kubernetes.io/projected/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-kube-api-access-xx24f\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:42.071276 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071123 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/590f0095-ecb1-43e4-9c1e-1c62a87a3b8a-host\") pod \"node-ca-rv678\" (UID: \"590f0095-ecb1-43e4-9c1e-1c62a87a3b8a\") " pod="openshift-image-registry/node-ca-rv678" Jul 22 03:07:42.071276 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071140 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-etc-kubernetes\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:42.071276 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071160 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/497ea3ce-7ad8-42f9-a52e-8094ec646cc9-hosts-file\") pod \"node-resolver-kkcg5\" (UID: \"497ea3ce-7ad8-42f9-a52e-8094ec646cc9\") " pod="openshift-dns/node-resolver-kkcg5" Jul 22 03:07:42.071276 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071160 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-system-cni-dir\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:42.071276 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071182 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-run\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:42.071276 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071216 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-etc-sysconfig\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:42.071276 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071231 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-run\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:42.071276 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071210 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/590f0095-ecb1-43e4-9c1e-1c62a87a3b8a-host\") pod \"node-ca-rv678\" (UID: \"590f0095-ecb1-43e4-9c1e-1c62a87a3b8a\") " pod="openshift-image-registry/node-ca-rv678" Jul 22 03:07:42.071276 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071263 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-host\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:42.071276 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071222 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-host\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:42.071276 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071285 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-etc-kubernetes\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:42.071857 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071306 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b9f02261-c5da-4a76-aa41-e09c6955ef76-cni-binary-copy\") pod \"multus-additional-cni-plugins-wth5q\" (UID: \"b9f02261-c5da-4a76-aa41-e09c6955ef76\") " pod="openshift-multus/multus-additional-cni-plugins-wth5q" Jul 22 03:07:42.071857 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071336 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b9f02261-c5da-4a76-aa41-e09c6955ef76-cnibin\") pod \"multus-additional-cni-plugins-wth5q\" (UID: \"b9f02261-c5da-4a76-aa41-e09c6955ef76\") " pod="openshift-multus/multus-additional-cni-plugins-wth5q" Jul 22 03:07:42.071857 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071355 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/497ea3ce-7ad8-42f9-a52e-8094ec646cc9-hosts-file\") pod \"node-resolver-kkcg5\" (UID: \"497ea3ce-7ad8-42f9-a52e-8094ec646cc9\") " pod="openshift-dns/node-resolver-kkcg5" Jul 22 03:07:42.071857 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071395 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b9f02261-c5da-4a76-aa41-e09c6955ef76-cnibin\") pod \"multus-additional-cni-plugins-wth5q\" (UID: \"b9f02261-c5da-4a76-aa41-e09c6955ef76\") " pod="openshift-multus/multus-additional-cni-plugins-wth5q" Jul 22 03:07:42.071857 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071402 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-host-run-k8s-cni-cncf-io\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:42.071857 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071362 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-host-run-k8s-cni-cncf-io\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:42.071857 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071443 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b9f02261-c5da-4a76-aa41-e09c6955ef76-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wth5q\" (UID: \"b9f02261-c5da-4a76-aa41-e09c6955ef76\") " pod="openshift-multus/multus-additional-cni-plugins-wth5q" Jul 22 03:07:42.071857 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071477 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b9f02261-c5da-4a76-aa41-e09c6955ef76-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wth5q\" (UID: \"b9f02261-c5da-4a76-aa41-e09c6955ef76\") " pod="openshift-multus/multus-additional-cni-plugins-wth5q" Jul 22 03:07:42.071857 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071499 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-multus-conf-dir\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:42.071857 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071519 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-var-lib-kubelet\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:42.071857 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071537 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-etc-tuned\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:42.071857 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071561 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/db34ad96-aec1-457b-8c55-61fea9c7f14a-agent-certs\") pod \"konnectivity-agent-68xqw\" (UID: \"db34ad96-aec1-457b-8c55-61fea9c7f14a\") " pod="kube-system/konnectivity-agent-68xqw" Jul 22 03:07:42.071857 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071593 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-var-lib-kubelet\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:42.071857 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071607 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-lib-modules\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:42.071857 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071638 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-multus-conf-dir\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:42.071857 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071647 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b9f02261-c5da-4a76-aa41-e09c6955ef76-os-release\") pod \"multus-additional-cni-plugins-wth5q\" (UID: \"b9f02261-c5da-4a76-aa41-e09c6955ef76\") " pod="openshift-multus/multus-additional-cni-plugins-wth5q" Jul 22 03:07:42.071857 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071699 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b9f02261-c5da-4a76-aa41-e09c6955ef76-os-release\") pod \"multus-additional-cni-plugins-wth5q\" (UID: \"b9f02261-c5da-4a76-aa41-e09c6955ef76\") " pod="openshift-multus/multus-additional-cni-plugins-wth5q" Jul 22 03:07:42.072647 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071699 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-hostroot\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:42.072647 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071728 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-hostroot\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:42.072647 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071733 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-lib-modules\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:42.072647 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071739 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-host-run-multus-certs\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:42.072647 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071765 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-etc-modprobe-d\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:42.072647 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071791 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-etc-sysctl-conf\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:42.072647 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071815 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-tmp\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:42.072647 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071826 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-host-run-multus-certs\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:42.072647 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071839 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcgrv\" (UniqueName: \"kubernetes.io/projected/590f0095-ecb1-43e4-9c1e-1c62a87a3b8a-kube-api-access-bcgrv\") pod \"node-ca-rv678\" (UID: \"590f0095-ecb1-43e4-9c1e-1c62a87a3b8a\") " pod="openshift-image-registry/node-ca-rv678" Jul 22 03:07:42.072647 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071853 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/db34ad96-aec1-457b-8c55-61fea9c7f14a-konnectivity-ca\") pod \"konnectivity-agent-68xqw\" (UID: \"db34ad96-aec1-457b-8c55-61fea9c7f14a\") " pod="kube-system/konnectivity-agent-68xqw" Jul 22 03:07:42.072647 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071866 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-etc-kubernetes\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:42.072647 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071898 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-etc-systemd\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:42.072647 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071906 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-etc-kubernetes\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:42.072647 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071917 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-etc-sysctl-conf\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:42.072647 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071928 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-multus-socket-dir-parent\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:42.072647 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071974 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jws4\" (UniqueName: \"kubernetes.io/projected/734b270b-0f16-4bba-9cd3-7a41b919a010-kube-api-access-5jws4\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:42.072647 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071982 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-etc-systemd\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:42.072647 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071999 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-etc-sysctl-d\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:42.073332 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.072002 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-etc-modprobe-d\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:42.073332 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.072036 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7rb9\" (UniqueName: \"kubernetes.io/projected/497ea3ce-7ad8-42f9-a52e-8094ec646cc9-kube-api-access-l7rb9\") pod \"node-resolver-kkcg5\" (UID: \"497ea3ce-7ad8-42f9-a52e-8094ec646cc9\") " pod="openshift-dns/node-resolver-kkcg5" Jul 22 03:07:42.073332 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.072068 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-host-run-netns\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:42.073332 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.072095 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-host-var-lib-kubelet\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:42.073332 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.072121 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-sys\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:42.073332 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.072146 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b9f02261-c5da-4a76-aa41-e09c6955ef76-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wth5q\" (UID: \"b9f02261-c5da-4a76-aa41-e09c6955ef76\") " pod="openshift-multus/multus-additional-cni-plugins-wth5q" Jul 22 03:07:42.073332 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.072160 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-multus-socket-dir-parent\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:42.073332 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.072171 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-host-var-lib-cni-bin\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:42.073332 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.072202 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-host-var-lib-cni-multus\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:42.073332 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.072231 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/497ea3ce-7ad8-42f9-a52e-8094ec646cc9-tmp-dir\") pod \"node-resolver-kkcg5\" (UID: \"497ea3ce-7ad8-42f9-a52e-8094ec646cc9\") " pod="openshift-dns/node-resolver-kkcg5" Jul 22 03:07:42.073332 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.072255 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/590f0095-ecb1-43e4-9c1e-1c62a87a3b8a-serviceca\") pod \"node-ca-rv678\" (UID: \"590f0095-ecb1-43e4-9c1e-1c62a87a3b8a\") " pod="openshift-image-registry/node-ca-rv678" Jul 22 03:07:42.073332 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.072279 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b9f02261-c5da-4a76-aa41-e09c6955ef76-system-cni-dir\") pod \"multus-additional-cni-plugins-wth5q\" (UID: \"b9f02261-c5da-4a76-aa41-e09c6955ef76\") " pod="openshift-multus/multus-additional-cni-plugins-wth5q" Jul 22 03:07:42.073332 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.072309 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-os-release\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:42.073332 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.072330 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/734b270b-0f16-4bba-9cd3-7a41b919a010-cni-binary-copy\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:42.073332 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.072355 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qs25k\" (UniqueName: \"kubernetes.io/projected/b9f02261-c5da-4a76-aa41-e09c6955ef76-kube-api-access-qs25k\") pod \"multus-additional-cni-plugins-wth5q\" (UID: \"b9f02261-c5da-4a76-aa41-e09c6955ef76\") " pod="openshift-multus/multus-additional-cni-plugins-wth5q" Jul 22 03:07:42.073332 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.072379 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-multus-cni-dir\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:42.073332 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.072402 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-cnibin\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:42.073332 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.072411 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b9f02261-c5da-4a76-aa41-e09c6955ef76-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wth5q\" (UID: \"b9f02261-c5da-4a76-aa41-e09c6955ef76\") " pod="openshift-multus/multus-additional-cni-plugins-wth5q" Jul 22 03:07:42.074109 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.072425 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/734b270b-0f16-4bba-9cd3-7a41b919a010-multus-daemon-config\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:42.074109 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.072427 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b9f02261-c5da-4a76-aa41-e09c6955ef76-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wth5q\" (UID: \"b9f02261-c5da-4a76-aa41-e09c6955ef76\") " pod="openshift-multus/multus-additional-cni-plugins-wth5q" Jul 22 03:07:42.074109 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.072469 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b9f02261-c5da-4a76-aa41-e09c6955ef76-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wth5q\" (UID: \"b9f02261-c5da-4a76-aa41-e09c6955ef76\") " pod="openshift-multus/multus-additional-cni-plugins-wth5q" Jul 22 03:07:42.074109 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.072532 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b9f02261-c5da-4a76-aa41-e09c6955ef76-system-cni-dir\") pod \"multus-additional-cni-plugins-wth5q\" (UID: \"b9f02261-c5da-4a76-aa41-e09c6955ef76\") " pod="openshift-multus/multus-additional-cni-plugins-wth5q" Jul 22 03:07:42.074109 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.071897 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b9f02261-c5da-4a76-aa41-e09c6955ef76-cni-binary-copy\") pod \"multus-additional-cni-plugins-wth5q\" (UID: \"b9f02261-c5da-4a76-aa41-e09c6955ef76\") " pod="openshift-multus/multus-additional-cni-plugins-wth5q" Jul 22 03:07:42.074109 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.072572 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-host-var-lib-kubelet\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:42.074109 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.072583 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-host-var-lib-cni-bin\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:42.074109 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.072122 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-etc-sysctl-d\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:42.074109 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.072613 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-sys\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:42.074109 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.072623 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-host-var-lib-cni-multus\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:42.074109 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.072616 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-host-run-netns\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:42.074109 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.072672 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-os-release\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:42.074109 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.072680 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-multus-cni-dir\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:42.074109 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.072732 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/734b270b-0f16-4bba-9cd3-7a41b919a010-cnibin\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:42.074109 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.072981 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/497ea3ce-7ad8-42f9-a52e-8094ec646cc9-tmp-dir\") pod \"node-resolver-kkcg5\" (UID: \"497ea3ce-7ad8-42f9-a52e-8094ec646cc9\") " pod="openshift-dns/node-resolver-kkcg5" Jul 22 03:07:42.074109 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.073019 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/734b270b-0f16-4bba-9cd3-7a41b919a010-multus-daemon-config\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:42.074109 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.073098 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/734b270b-0f16-4bba-9cd3-7a41b919a010-cni-binary-copy\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:42.074109 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.073194 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/590f0095-ecb1-43e4-9c1e-1c62a87a3b8a-serviceca\") pod \"node-ca-rv678\" (UID: \"590f0095-ecb1-43e4-9c1e-1c62a87a3b8a\") " pod="openshift-image-registry/node-ca-rv678" Jul 22 03:07:42.074799 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.074475 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-tmp\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:42.074799 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.074534 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-etc-tuned\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:42.075102 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.075085 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/db34ad96-aec1-457b-8c55-61fea9c7f14a-agent-certs\") pod \"konnectivity-agent-68xqw\" (UID: \"db34ad96-aec1-457b-8c55-61fea9c7f14a\") " pod="kube-system/konnectivity-agent-68xqw" Jul 22 03:07:42.086715 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.086650 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jws4\" (UniqueName: \"kubernetes.io/projected/734b270b-0f16-4bba-9cd3-7a41b919a010-kube-api-access-5jws4\") pod \"multus-fwk5q\" (UID: \"734b270b-0f16-4bba-9cd3-7a41b919a010\") " pod="openshift-multus/multus-fwk5q" Jul 22 03:07:42.086715 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.086698 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qs25k\" (UniqueName: \"kubernetes.io/projected/b9f02261-c5da-4a76-aa41-e09c6955ef76-kube-api-access-qs25k\") pod \"multus-additional-cni-plugins-wth5q\" (UID: \"b9f02261-c5da-4a76-aa41-e09c6955ef76\") " pod="openshift-multus/multus-additional-cni-plugins-wth5q" Jul 22 03:07:42.086899 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.086720 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcgrv\" (UniqueName: \"kubernetes.io/projected/590f0095-ecb1-43e4-9c1e-1c62a87a3b8a-kube-api-access-bcgrv\") pod \"node-ca-rv678\" (UID: \"590f0095-ecb1-43e4-9c1e-1c62a87a3b8a\") " pod="openshift-image-registry/node-ca-rv678" Jul 22 03:07:42.087514 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.087491 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7rb9\" (UniqueName: \"kubernetes.io/projected/497ea3ce-7ad8-42f9-a52e-8094ec646cc9-kube-api-access-l7rb9\") pod \"node-resolver-kkcg5\" (UID: \"497ea3ce-7ad8-42f9-a52e-8094ec646cc9\") " pod="openshift-dns/node-resolver-kkcg5" Jul 22 03:07:42.088175 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.088145 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xx24f\" (UniqueName: \"kubernetes.io/projected/b7fe5359-9ec6-43ae-bff7-cb11454c1a06-kube-api-access-xx24f\") pod \"tuned-vr69p\" (UID: \"b7fe5359-9ec6-43ae-bff7-cb11454c1a06\") " pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:42.157035 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.157005 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w85fv" Jul 22 03:07:42.163942 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.163922 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-n94lh" Jul 22 03:07:42.176760 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.176735 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:07:42.182652 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.182633 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kkcg5" Jul 22 03:07:42.188358 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.188342 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rv678" Jul 22 03:07:42.194800 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.194779 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wth5q" Jul 22 03:07:42.201410 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.201394 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fwk5q" Jul 22 03:07:42.206961 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.206933 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vr69p" Jul 22 03:07:42.212733 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.212718 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-68xqw" Jul 22 03:07:42.353542 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.353466 2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 03:07:42.475597 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.475560 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1297a647-da14-4c7b-bf94-6f4f4e679c31-original-pull-secret\") pod \"global-pull-secret-syncer-qhg45\" (UID: \"1297a647-da14-4c7b-bf94-6f4f4e679c31\") " pod="kube-system/global-pull-secret-syncer-qhg45" Jul 22 03:07:42.475597 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.475602 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb8b43a6-b50f-41e9-9e68-763231228640-metrics-certs\") pod \"network-metrics-daemon-xrbdq\" (UID: \"bb8b43a6-b50f-41e9-9e68-763231228640\") " pod="openshift-multus/network-metrics-daemon-xrbdq" Jul 22 03:07:42.475849 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:42.475732 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 03:07:42.475849 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:42.475803 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1297a647-da14-4c7b-bf94-6f4f4e679c31-original-pull-secret podName:1297a647-da14-4c7b-bf94-6f4f4e679c31 nodeName:}" failed. No retries permitted until 2026-07-22 03:07:43.475782761 +0000 UTC m=+3.924766000 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1297a647-da14-4c7b-bf94-6f4f4e679c31-original-pull-secret") pod "global-pull-secret-syncer-qhg45" (UID: "1297a647-da14-4c7b-bf94-6f4f4e679c31") : object "kube-system"/"original-pull-secret" not registered Jul 22 03:07:42.475849 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:42.475732 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 03:07:42.476041 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:42.475904 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb8b43a6-b50f-41e9-9e68-763231228640-metrics-certs podName:bb8b43a6-b50f-41e9-9e68-763231228640 nodeName:}" failed. No retries permitted until 2026-07-22 03:07:43.475877186 +0000 UTC m=+3.924860466 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bb8b43a6-b50f-41e9-9e68-763231228640-metrics-certs") pod "network-metrics-daemon-xrbdq" (UID: "bb8b43a6-b50f-41e9-9e68-763231228640") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 03:07:42.519318 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:42.519286 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda01dfba4_f590_412a_8438_1646c51bb1a1.slice/crio-b5f148b785a940f45ee0325a79afe963fb29eeb15302146bb84890cbc85584dc WatchSource:0}: Error finding container b5f148b785a940f45ee0325a79afe963fb29eeb15302146bb84890cbc85584dc: Status 404 returned error can't find the container with id b5f148b785a940f45ee0325a79afe963fb29eeb15302146bb84890cbc85584dc Jul 22 03:07:42.525093 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:42.525051 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7fe5359_9ec6_43ae_bff7_cb11454c1a06.slice/crio-1584d5d9278174e5d4b882cdfd1c4e1a950fae82f232185c81b0587919458f0a WatchSource:0}: Error finding container 1584d5d9278174e5d4b882cdfd1c4e1a950fae82f232185c81b0587919458f0a: Status 404 returned error can't find the container with id 1584d5d9278174e5d4b882cdfd1c4e1a950fae82f232185c81b0587919458f0a Jul 22 03:07:42.526062 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:42.526035 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84476432_16cb_4559_bf1f_4fe42ea2e87a.slice/crio-e2a39c36d8a160803ba60549796fc842205f8e298480624d2d42ec8938534c04 WatchSource:0}: Error finding container e2a39c36d8a160803ba60549796fc842205f8e298480624d2d42ec8938534c04: Status 404 returned error can't find the container with id e2a39c36d8a160803ba60549796fc842205f8e298480624d2d42ec8938534c04 Jul 22 03:07:42.526978 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:42.526915 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9f02261_c5da_4a76_aa41_e09c6955ef76.slice/crio-3c9e4164698f782f5fbf69fd246fa305b861e7789f1a3a416a3d0052e7c256a5 WatchSource:0}: Error finding container 3c9e4164698f782f5fbf69fd246fa305b861e7789f1a3a416a3d0052e7c256a5: Status 404 returned error can't find the container with id 3c9e4164698f782f5fbf69fd246fa305b861e7789f1a3a416a3d0052e7c256a5 Jul 22 03:07:42.527793 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:42.527768 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod734b270b_0f16_4bba_9cd3_7a41b919a010.slice/crio-74430ace655b3032ab9c43f1d158bf99a8870d6d3ea1e2924ef347a223d5bd8e WatchSource:0}: Error finding container 74430ace655b3032ab9c43f1d158bf99a8870d6d3ea1e2924ef347a223d5bd8e: Status 404 returned error can't find the container with id 74430ace655b3032ab9c43f1d158bf99a8870d6d3ea1e2924ef347a223d5bd8e Jul 22 03:07:42.551242 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:42.551217 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod590f0095_ecb1_43e4_9c1e_1c62a87a3b8a.slice/crio-9706819d82f09a5ab989cf48a87a7bc50abae62638fbd9d397f1650a1324043c WatchSource:0}: Error finding container 9706819d82f09a5ab989cf48a87a7bc50abae62638fbd9d397f1650a1324043c: Status 404 returned error can't find the container with id 9706819d82f09a5ab989cf48a87a7bc50abae62638fbd9d397f1650a1324043c Jul 22 03:07:42.553082 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:42.553058 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod497ea3ce_7ad8_42f9_a52e_8094ec646cc9.slice/crio-5ca2ff172222e3f309ea503dc72258e86bfcf676442e9a5a7b91ed30ab791fe0 WatchSource:0}: Error finding container 5ca2ff172222e3f309ea503dc72258e86bfcf676442e9a5a7b91ed30ab791fe0: Status 404 returned error can't find the container with id 5ca2ff172222e3f309ea503dc72258e86bfcf676442e9a5a7b91ed30ab791fe0 Jul 22 03:07:42.553606 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:42.553421 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bfceba5_9ca9_4955_abfd_1c029a7e6062.slice/crio-7d15812fccae1b4d4135d2a2350c66fd7586341a5eabc3290d39d293086126e6 WatchSource:0}: Error finding container 7d15812fccae1b4d4135d2a2350c66fd7586341a5eabc3290d39d293086126e6: Status 404 returned error can't find the container with id 7d15812fccae1b4d4135d2a2350c66fd7586341a5eabc3290d39d293086126e6 Jul 22 03:07:42.554518 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:07:42.554432 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb34ad96_aec1_457b_8c55_61fea9c7f14a.slice/crio-dd6fe67e03182812f57ac870397b226e8b63ad1dfcb26a8c1a6fc5515bba711e WatchSource:0}: Error finding container dd6fe67e03182812f57ac870397b226e8b63ad1dfcb26a8c1a6fc5515bba711e: Status 404 returned error can't find the container with id dd6fe67e03182812f57ac870397b226e8b63ad1dfcb26a8c1a6fc5515bba711e Jul 22 03:07:42.576735 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.576712 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x25qb\" (UniqueName: \"kubernetes.io/projected/1c2c0875-8ece-494b-95a1-00fe315db1e4-kube-api-access-x25qb\") pod \"network-check-target-k4rxt\" (UID: \"1c2c0875-8ece-494b-95a1-00fe315db1e4\") " pod="openshift-network-diagnostics/network-check-target-k4rxt" Jul 22 03:07:42.576878 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:42.576857 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 03:07:42.576878 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:42.576881 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 03:07:42.577027 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:42.576893 2556 projected.go:196] Error preparing data for projected volume kube-api-access-x25qb for pod openshift-network-diagnostics/network-check-target-k4rxt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 03:07:42.577027 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:42.576966 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c2c0875-8ece-494b-95a1-00fe315db1e4-kube-api-access-x25qb podName:1c2c0875-8ece-494b-95a1-00fe315db1e4 nodeName:}" failed. No retries permitted until 2026-07-22 03:07:43.576929822 +0000 UTC m=+4.025913065 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-x25qb" (UniqueName: "kubernetes.io/projected/1c2c0875-8ece-494b-95a1-00fe315db1e4-kube-api-access-x25qb") pod "network-check-target-k4rxt" (UID: "1c2c0875-8ece-494b-95a1-00fe315db1e4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 03:07:42.909001 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.908894 2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 03:02:41 +0000 UTC" deadline="2027-12-20 22:54:49.091014653 +0000 UTC" Jul 22 03:07:42.909001 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:42.908938 2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12403h47m6.182078279s" Jul 22 03:07:43.027329 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:43.027116 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-13.ec2.internal" event={"ID":"d5cf9dba7131053469ef1265ca066507","Type":"ContainerStarted","Data":"71d055db5340034a32bffe1e2b74fc8f9fa796047ea39187b464005681b8b517"} Jul 22 03:07:43.028321 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:43.028294 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-n94lh" event={"ID":"7bfceba5-9ca9-4955-abfd-1c029a7e6062","Type":"ContainerStarted","Data":"7d15812fccae1b4d4135d2a2350c66fd7586341a5eabc3290d39d293086126e6"} Jul 22 03:07:43.029494 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:43.029471 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kkcg5" event={"ID":"497ea3ce-7ad8-42f9-a52e-8094ec646cc9","Type":"ContainerStarted","Data":"5ca2ff172222e3f309ea503dc72258e86bfcf676442e9a5a7b91ed30ab791fe0"} Jul 22 03:07:43.030772 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:43.030748 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fwk5q" event={"ID":"734b270b-0f16-4bba-9cd3-7a41b919a010","Type":"ContainerStarted","Data":"74430ace655b3032ab9c43f1d158bf99a8870d6d3ea1e2924ef347a223d5bd8e"} Jul 22 03:07:43.031901 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:43.031873 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wth5q" event={"ID":"b9f02261-c5da-4a76-aa41-e09c6955ef76","Type":"ContainerStarted","Data":"3c9e4164698f782f5fbf69fd246fa305b861e7789f1a3a416a3d0052e7c256a5"} Jul 22 03:07:43.032999 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:43.032979 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vr69p" event={"ID":"b7fe5359-9ec6-43ae-bff7-cb11454c1a06","Type":"ContainerStarted","Data":"1584d5d9278174e5d4b882cdfd1c4e1a950fae82f232185c81b0587919458f0a"} Jul 22 03:07:43.034163 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:43.034144 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-68xqw" event={"ID":"db34ad96-aec1-457b-8c55-61fea9c7f14a","Type":"ContainerStarted","Data":"dd6fe67e03182812f57ac870397b226e8b63ad1dfcb26a8c1a6fc5515bba711e"} Jul 22 03:07:43.035193 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:43.035172 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rv678" event={"ID":"590f0095-ecb1-43e4-9c1e-1c62a87a3b8a","Type":"ContainerStarted","Data":"9706819d82f09a5ab989cf48a87a7bc50abae62638fbd9d397f1650a1324043c"} Jul 22 03:07:43.036129 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:43.036110 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w85fv" event={"ID":"84476432-16cb-4559-bf1f-4fe42ea2e87a","Type":"ContainerStarted","Data":"e2a39c36d8a160803ba60549796fc842205f8e298480624d2d42ec8938534c04"} Jul 22 03:07:43.037322 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:43.037297 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" event={"ID":"a01dfba4-f590-412a-8438-1646c51bb1a1","Type":"ContainerStarted","Data":"b5f148b785a940f45ee0325a79afe963fb29eeb15302146bb84890cbc85584dc"} Jul 22 03:07:43.044818 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:43.044776 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-13.ec2.internal" podStartSLOduration=2.044762663 podStartE2EDuration="2.044762663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:07:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:07:43.044725029 +0000 UTC m=+3.493708291" watchObservedRunningTime="2026-07-22 03:07:43.044762663 +0000 UTC m=+3.493745913" Jul 22 03:07:43.484361 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:43.484323 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1297a647-da14-4c7b-bf94-6f4f4e679c31-original-pull-secret\") pod \"global-pull-secret-syncer-qhg45\" (UID: \"1297a647-da14-4c7b-bf94-6f4f4e679c31\") " pod="kube-system/global-pull-secret-syncer-qhg45" Jul 22 03:07:43.484531 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:43.484374 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb8b43a6-b50f-41e9-9e68-763231228640-metrics-certs\") pod \"network-metrics-daemon-xrbdq\" (UID: \"bb8b43a6-b50f-41e9-9e68-763231228640\") " pod="openshift-multus/network-metrics-daemon-xrbdq" Jul 22 03:07:43.484531 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:43.484516 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 03:07:43.484640 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:43.484582 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb8b43a6-b50f-41e9-9e68-763231228640-metrics-certs podName:bb8b43a6-b50f-41e9-9e68-763231228640 nodeName:}" failed. No retries permitted until 2026-07-22 03:07:45.484562793 +0000 UTC m=+5.933546042 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bb8b43a6-b50f-41e9-9e68-763231228640-metrics-certs") pod "network-metrics-daemon-xrbdq" (UID: "bb8b43a6-b50f-41e9-9e68-763231228640") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 03:07:43.485070 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:43.485050 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 03:07:43.485143 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:43.485107 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1297a647-da14-4c7b-bf94-6f4f4e679c31-original-pull-secret podName:1297a647-da14-4c7b-bf94-6f4f4e679c31 nodeName:}" failed. No retries permitted until 2026-07-22 03:07:45.485091121 +0000 UTC m=+5.934074350 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1297a647-da14-4c7b-bf94-6f4f4e679c31-original-pull-secret") pod "global-pull-secret-syncer-qhg45" (UID: "1297a647-da14-4c7b-bf94-6f4f4e679c31") : object "kube-system"/"original-pull-secret" not registered Jul 22 03:07:43.585462 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:43.585414 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x25qb\" (UniqueName: \"kubernetes.io/projected/1c2c0875-8ece-494b-95a1-00fe315db1e4-kube-api-access-x25qb\") pod \"network-check-target-k4rxt\" (UID: \"1c2c0875-8ece-494b-95a1-00fe315db1e4\") " pod="openshift-network-diagnostics/network-check-target-k4rxt" Jul 22 03:07:43.585784 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:43.585761 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 03:07:43.585873 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:43.585819 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 03:07:43.585873 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:43.585834 2556 projected.go:196] Error preparing data for projected volume kube-api-access-x25qb for pod openshift-network-diagnostics/network-check-target-k4rxt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 03:07:43.586012 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:43.585897 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c2c0875-8ece-494b-95a1-00fe315db1e4-kube-api-access-x25qb podName:1c2c0875-8ece-494b-95a1-00fe315db1e4 nodeName:}" failed. No retries permitted until 2026-07-22 03:07:45.585878044 +0000 UTC m=+6.034861295 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-x25qb" (UniqueName: "kubernetes.io/projected/1c2c0875-8ece-494b-95a1-00fe315db1e4-kube-api-access-x25qb") pod "network-check-target-k4rxt" (UID: "1c2c0875-8ece-494b-95a1-00fe315db1e4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 03:07:43.909851 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:43.909641 2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 03:02:41 +0000 UTC" deadline="2028-03-27 06:03:46.876599807 +0000 UTC" Jul 22 03:07:43.909851 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:43.909677 2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14738h56m2.966926802s" Jul 22 03:07:44.028986 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:44.028908 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qhg45" Jul 22 03:07:44.029139 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:44.029082 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qhg45" podUID="1297a647-da14-4c7b-bf94-6f4f4e679c31" Jul 22 03:07:44.029214 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:44.029152 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrbdq" Jul 22 03:07:44.029269 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:44.029244 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrbdq" podUID="bb8b43a6-b50f-41e9-9e68-763231228640" Jul 22 03:07:44.029318 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:44.029293 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4rxt" Jul 22 03:07:44.029370 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:44.029352 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k4rxt" podUID="1c2c0875-8ece-494b-95a1-00fe315db1e4" Jul 22 03:07:44.056064 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:44.055214 2556 generic.go:356] "Generic (PLEG): container finished" podID="1ee6828dcab126e50933f520556b028c" containerID="106f92c030aa09e7174c4e75dee2fddfd5e29a0e3bcb5dc2d430a194ee98a7d2" exitCode=0 Jul 22 03:07:44.056064 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:44.056003 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-13.ec2.internal" event={"ID":"1ee6828dcab126e50933f520556b028c","Type":"ContainerDied","Data":"106f92c030aa09e7174c4e75dee2fddfd5e29a0e3bcb5dc2d430a194ee98a7d2"} Jul 22 03:07:45.071494 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:45.071456 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-13.ec2.internal" event={"ID":"1ee6828dcab126e50933f520556b028c","Type":"ContainerStarted","Data":"9232cd03d621ca048d6c0e4870acbe91de232601e654b10e3c58ec0d215e78c9"} Jul 22 03:07:45.093728 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:45.093360 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-13.ec2.internal" podStartSLOduration=4.093337199 podStartE2EDuration="4.093337199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:07:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:07:45.092494695 +0000 UTC m=+5.541477948" watchObservedRunningTime="2026-07-22 03:07:45.093337199 +0000 UTC m=+5.542320449" Jul 22 03:07:45.505143 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:45.505058 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1297a647-da14-4c7b-bf94-6f4f4e679c31-original-pull-secret\") pod \"global-pull-secret-syncer-qhg45\" (UID: \"1297a647-da14-4c7b-bf94-6f4f4e679c31\") " pod="kube-system/global-pull-secret-syncer-qhg45" Jul 22 03:07:45.505143 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:45.505108 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb8b43a6-b50f-41e9-9e68-763231228640-metrics-certs\") pod \"network-metrics-daemon-xrbdq\" (UID: \"bb8b43a6-b50f-41e9-9e68-763231228640\") " pod="openshift-multus/network-metrics-daemon-xrbdq" Jul 22 03:07:45.505374 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:45.505276 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 03:07:45.505374 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:45.505335 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb8b43a6-b50f-41e9-9e68-763231228640-metrics-certs podName:bb8b43a6-b50f-41e9-9e68-763231228640 nodeName:}" failed. No retries permitted until 2026-07-22 03:07:49.505316515 +0000 UTC m=+9.954299759 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bb8b43a6-b50f-41e9-9e68-763231228640-metrics-certs") pod "network-metrics-daemon-xrbdq" (UID: "bb8b43a6-b50f-41e9-9e68-763231228640") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 03:07:45.505483 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:45.505397 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 03:07:45.505483 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:45.505433 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1297a647-da14-4c7b-bf94-6f4f4e679c31-original-pull-secret podName:1297a647-da14-4c7b-bf94-6f4f4e679c31 nodeName:}" failed. No retries permitted until 2026-07-22 03:07:49.505421637 +0000 UTC m=+9.954404868 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1297a647-da14-4c7b-bf94-6f4f4e679c31-original-pull-secret") pod "global-pull-secret-syncer-qhg45" (UID: "1297a647-da14-4c7b-bf94-6f4f4e679c31") : object "kube-system"/"original-pull-secret" not registered Jul 22 03:07:45.606433 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:45.606394 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x25qb\" (UniqueName: \"kubernetes.io/projected/1c2c0875-8ece-494b-95a1-00fe315db1e4-kube-api-access-x25qb\") pod \"network-check-target-k4rxt\" (UID: \"1c2c0875-8ece-494b-95a1-00fe315db1e4\") " pod="openshift-network-diagnostics/network-check-target-k4rxt" Jul 22 03:07:45.606686 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:45.606590 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 03:07:45.606686 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:45.606612 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 03:07:45.606686 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:45.606626 2556 projected.go:196] Error preparing data for projected volume kube-api-access-x25qb for pod openshift-network-diagnostics/network-check-target-k4rxt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 03:07:45.606686 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:45.606685 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c2c0875-8ece-494b-95a1-00fe315db1e4-kube-api-access-x25qb podName:1c2c0875-8ece-494b-95a1-00fe315db1e4 nodeName:}" failed. No retries permitted until 2026-07-22 03:07:49.606666677 +0000 UTC m=+10.055649926 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-x25qb" (UniqueName: "kubernetes.io/projected/1c2c0875-8ece-494b-95a1-00fe315db1e4-kube-api-access-x25qb") pod "network-check-target-k4rxt" (UID: "1c2c0875-8ece-494b-95a1-00fe315db1e4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 03:07:46.018710 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:46.018680 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrbdq" Jul 22 03:07:46.018889 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:46.018680 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4rxt" Jul 22 03:07:46.018889 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:46.018823 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrbdq" podUID="bb8b43a6-b50f-41e9-9e68-763231228640" Jul 22 03:07:46.019042 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:46.018880 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k4rxt" podUID="1c2c0875-8ece-494b-95a1-00fe315db1e4" Jul 22 03:07:46.019042 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:46.018689 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qhg45" Jul 22 03:07:46.019042 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:46.018996 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qhg45" podUID="1297a647-da14-4c7b-bf94-6f4f4e679c31" Jul 22 03:07:48.018626 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:48.018594 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qhg45" Jul 22 03:07:48.018626 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:48.018631 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrbdq" Jul 22 03:07:48.019424 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:48.018595 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4rxt" Jul 22 03:07:48.019424 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:48.018749 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qhg45" podUID="1297a647-da14-4c7b-bf94-6f4f4e679c31" Jul 22 03:07:48.019424 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:48.018844 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k4rxt" podUID="1c2c0875-8ece-494b-95a1-00fe315db1e4" Jul 22 03:07:48.019424 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:48.018946 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrbdq" podUID="bb8b43a6-b50f-41e9-9e68-763231228640" Jul 22 03:07:49.537642 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:49.537070 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1297a647-da14-4c7b-bf94-6f4f4e679c31-original-pull-secret\") pod \"global-pull-secret-syncer-qhg45\" (UID: \"1297a647-da14-4c7b-bf94-6f4f4e679c31\") " pod="kube-system/global-pull-secret-syncer-qhg45" Jul 22 03:07:49.537642 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:49.537119 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb8b43a6-b50f-41e9-9e68-763231228640-metrics-certs\") pod \"network-metrics-daemon-xrbdq\" (UID: \"bb8b43a6-b50f-41e9-9e68-763231228640\") " pod="openshift-multus/network-metrics-daemon-xrbdq" Jul 22 03:07:49.537642 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:49.537243 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 03:07:49.537642 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:49.537248 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 03:07:49.537642 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:49.537299 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb8b43a6-b50f-41e9-9e68-763231228640-metrics-certs podName:bb8b43a6-b50f-41e9-9e68-763231228640 nodeName:}" failed. No retries permitted until 2026-07-22 03:07:57.53728027 +0000 UTC m=+17.986263512 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bb8b43a6-b50f-41e9-9e68-763231228640-metrics-certs") pod "network-metrics-daemon-xrbdq" (UID: "bb8b43a6-b50f-41e9-9e68-763231228640") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 03:07:49.537642 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:49.537317 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1297a647-da14-4c7b-bf94-6f4f4e679c31-original-pull-secret podName:1297a647-da14-4c7b-bf94-6f4f4e679c31 nodeName:}" failed. No retries permitted until 2026-07-22 03:07:57.537307667 +0000 UTC m=+17.986290901 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1297a647-da14-4c7b-bf94-6f4f4e679c31-original-pull-secret") pod "global-pull-secret-syncer-qhg45" (UID: "1297a647-da14-4c7b-bf94-6f4f4e679c31") : object "kube-system"/"original-pull-secret" not registered Jul 22 03:07:49.637787 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:49.637751 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x25qb\" (UniqueName: \"kubernetes.io/projected/1c2c0875-8ece-494b-95a1-00fe315db1e4-kube-api-access-x25qb\") pod \"network-check-target-k4rxt\" (UID: \"1c2c0875-8ece-494b-95a1-00fe315db1e4\") " pod="openshift-network-diagnostics/network-check-target-k4rxt" Jul 22 03:07:49.637984 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:49.637942 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 03:07:49.638060 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:49.637989 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 03:07:49.638060 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:49.638003 2556 projected.go:196] Error preparing data for projected volume kube-api-access-x25qb for pod openshift-network-diagnostics/network-check-target-k4rxt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 03:07:49.638169 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:49.638065 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c2c0875-8ece-494b-95a1-00fe315db1e4-kube-api-access-x25qb podName:1c2c0875-8ece-494b-95a1-00fe315db1e4 nodeName:}" failed. No retries permitted until 2026-07-22 03:07:57.63804671 +0000 UTC m=+18.087029950 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-x25qb" (UniqueName: "kubernetes.io/projected/1c2c0875-8ece-494b-95a1-00fe315db1e4-kube-api-access-x25qb") pod "network-check-target-k4rxt" (UID: "1c2c0875-8ece-494b-95a1-00fe315db1e4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 03:07:50.020071 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:50.019560 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrbdq" Jul 22 03:07:50.020071 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:50.019678 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrbdq" podUID="bb8b43a6-b50f-41e9-9e68-763231228640" Jul 22 03:07:50.020071 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:50.019764 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4rxt" Jul 22 03:07:50.020071 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:50.019871 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k4rxt" podUID="1c2c0875-8ece-494b-95a1-00fe315db1e4" Jul 22 03:07:50.020071 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:50.019933 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qhg45" Jul 22 03:07:50.020071 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:50.020049 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qhg45" podUID="1297a647-da14-4c7b-bf94-6f4f4e679c31" Jul 22 03:07:52.017905 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:52.017873 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4rxt" Jul 22 03:07:52.018285 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:52.017932 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qhg45" Jul 22 03:07:52.018285 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:52.017943 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrbdq" Jul 22 03:07:52.018285 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:52.018056 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k4rxt" podUID="1c2c0875-8ece-494b-95a1-00fe315db1e4" Jul 22 03:07:52.018417 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:52.018313 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrbdq" podUID="bb8b43a6-b50f-41e9-9e68-763231228640" Jul 22 03:07:52.018449 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:52.018423 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qhg45" podUID="1297a647-da14-4c7b-bf94-6f4f4e679c31" Jul 22 03:07:54.021348 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:54.021315 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4rxt" Jul 22 03:07:54.021814 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:54.021422 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qhg45" Jul 22 03:07:54.021814 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:54.021439 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrbdq" Jul 22 03:07:54.021814 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:54.021432 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k4rxt" podUID="1c2c0875-8ece-494b-95a1-00fe315db1e4" Jul 22 03:07:54.021814 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:54.021546 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qhg45" podUID="1297a647-da14-4c7b-bf94-6f4f4e679c31" Jul 22 03:07:54.021814 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:54.021629 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrbdq" podUID="bb8b43a6-b50f-41e9-9e68-763231228640" Jul 22 03:07:56.018489 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:56.018451 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4rxt" Jul 22 03:07:56.018489 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:56.018487 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qhg45" Jul 22 03:07:56.019059 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:56.018452 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrbdq" Jul 22 03:07:56.019059 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:56.018584 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k4rxt" podUID="1c2c0875-8ece-494b-95a1-00fe315db1e4" Jul 22 03:07:56.019059 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:56.018649 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrbdq" podUID="bb8b43a6-b50f-41e9-9e68-763231228640" Jul 22 03:07:56.019059 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:56.018736 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qhg45" podUID="1297a647-da14-4c7b-bf94-6f4f4e679c31" Jul 22 03:07:57.597823 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:57.597789 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1297a647-da14-4c7b-bf94-6f4f4e679c31-original-pull-secret\") pod \"global-pull-secret-syncer-qhg45\" (UID: \"1297a647-da14-4c7b-bf94-6f4f4e679c31\") " pod="kube-system/global-pull-secret-syncer-qhg45" Jul 22 03:07:57.597823 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:57.597826 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb8b43a6-b50f-41e9-9e68-763231228640-metrics-certs\") pod \"network-metrics-daemon-xrbdq\" (UID: \"bb8b43a6-b50f-41e9-9e68-763231228640\") " pod="openshift-multus/network-metrics-daemon-xrbdq" Jul 22 03:07:57.598262 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:57.597941 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 03:07:57.598262 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:57.597966 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 03:07:57.598262 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:57.598020 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb8b43a6-b50f-41e9-9e68-763231228640-metrics-certs podName:bb8b43a6-b50f-41e9-9e68-763231228640 nodeName:}" failed. No retries permitted until 2026-07-22 03:08:13.598000882 +0000 UTC m=+34.046984110 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bb8b43a6-b50f-41e9-9e68-763231228640-metrics-certs") pod "network-metrics-daemon-xrbdq" (UID: "bb8b43a6-b50f-41e9-9e68-763231228640") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 03:07:57.598262 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:57.598039 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1297a647-da14-4c7b-bf94-6f4f4e679c31-original-pull-secret podName:1297a647-da14-4c7b-bf94-6f4f4e679c31 nodeName:}" failed. No retries permitted until 2026-07-22 03:08:13.598030979 +0000 UTC m=+34.047014207 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1297a647-da14-4c7b-bf94-6f4f4e679c31-original-pull-secret") pod "global-pull-secret-syncer-qhg45" (UID: "1297a647-da14-4c7b-bf94-6f4f4e679c31") : object "kube-system"/"original-pull-secret" not registered Jul 22 03:07:57.698288 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:57.698254 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x25qb\" (UniqueName: \"kubernetes.io/projected/1c2c0875-8ece-494b-95a1-00fe315db1e4-kube-api-access-x25qb\") pod \"network-check-target-k4rxt\" (UID: \"1c2c0875-8ece-494b-95a1-00fe315db1e4\") " pod="openshift-network-diagnostics/network-check-target-k4rxt" Jul 22 03:07:57.698438 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:57.698392 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 03:07:57.698438 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:57.698419 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 03:07:57.698438 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:57.698433 2556 projected.go:196] Error preparing data for projected volume kube-api-access-x25qb for pod openshift-network-diagnostics/network-check-target-k4rxt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 03:07:57.698576 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:57.698496 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c2c0875-8ece-494b-95a1-00fe315db1e4-kube-api-access-x25qb podName:1c2c0875-8ece-494b-95a1-00fe315db1e4 nodeName:}" failed. No retries permitted until 2026-07-22 03:08:13.698482248 +0000 UTC m=+34.147465475 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-x25qb" (UniqueName: "kubernetes.io/projected/1c2c0875-8ece-494b-95a1-00fe315db1e4-kube-api-access-x25qb") pod "network-check-target-k4rxt" (UID: "1c2c0875-8ece-494b-95a1-00fe315db1e4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 03:07:58.018515 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:58.018436 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrbdq" Jul 22 03:07:58.018515 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:58.018479 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qhg45" Jul 22 03:07:58.018739 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:07:58.018449 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4rxt" Jul 22 03:07:58.018739 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:58.018586 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrbdq" podUID="bb8b43a6-b50f-41e9-9e68-763231228640" Jul 22 03:07:58.018739 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:58.018648 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qhg45" podUID="1297a647-da14-4c7b-bf94-6f4f4e679c31" Jul 22 03:07:58.018876 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:07:58.018751 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k4rxt" podUID="1c2c0875-8ece-494b-95a1-00fe315db1e4" Jul 22 03:08:00.019638 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:00.019602 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrbdq" Jul 22 03:08:00.020148 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:00.019701 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4rxt" Jul 22 03:08:00.020148 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:00.019732 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qhg45" Jul 22 03:08:00.020148 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:00.019754 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrbdq" podUID="bb8b43a6-b50f-41e9-9e68-763231228640" Jul 22 03:08:00.020148 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:00.019807 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qhg45" podUID="1297a647-da14-4c7b-bf94-6f4f4e679c31" Jul 22 03:08:00.020148 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:00.019863 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k4rxt" podUID="1c2c0875-8ece-494b-95a1-00fe315db1e4" Jul 22 03:08:02.018624 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:02.018309 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4rxt" Jul 22 03:08:02.019459 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:02.018309 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qhg45" Jul 22 03:08:02.019459 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:02.018704 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k4rxt" podUID="1c2c0875-8ece-494b-95a1-00fe315db1e4" Jul 22 03:08:02.019459 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:02.018309 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrbdq" Jul 22 03:08:02.019459 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:02.018775 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qhg45" podUID="1297a647-da14-4c7b-bf94-6f4f4e679c31" Jul 22 03:08:02.019459 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:02.018871 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrbdq" podUID="bb8b43a6-b50f-41e9-9e68-763231228640" Jul 22 03:08:02.108108 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:02.108062 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kkcg5" event={"ID":"497ea3ce-7ad8-42f9-a52e-8094ec646cc9","Type":"ContainerStarted","Data":"78bd3037057da9c4f2f097224a889766a9f05f5ccdddb706e96b8b8b85a6fe5a"} Jul 22 03:08:02.109815 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:02.109783 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fwk5q" event={"ID":"734b270b-0f16-4bba-9cd3-7a41b919a010","Type":"ContainerStarted","Data":"ebaa9ccb9a80dbf53b00e620fa8f3089a33d5e5ce531f475a58adc962f1e6e87"} Jul 22 03:08:02.111406 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:02.111379 2556 generic.go:356] "Generic (PLEG): container finished" podID="b9f02261-c5da-4a76-aa41-e09c6955ef76" containerID="feb9ea236bda2f093b48b7d1c352b2360e2d0b1170f43a978cfc7830c3f8a5b0" exitCode=0 Jul 22 03:08:02.111531 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:02.111469 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wth5q" event={"ID":"b9f02261-c5da-4a76-aa41-e09c6955ef76","Type":"ContainerDied","Data":"feb9ea236bda2f093b48b7d1c352b2360e2d0b1170f43a978cfc7830c3f8a5b0"} Jul 22 03:08:02.113087 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:02.113060 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vr69p" event={"ID":"b7fe5359-9ec6-43ae-bff7-cb11454c1a06","Type":"ContainerStarted","Data":"f24098ebb64ddc9d22650a7781c0cccb1fd475fb1159408f0b8dd381e3c80ce2"} Jul 22 03:08:02.114584 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:02.114561 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-68xqw" event={"ID":"db34ad96-aec1-457b-8c55-61fea9c7f14a","Type":"ContainerStarted","Data":"50b0d6b5e26cd1ef748c52793bc36fe17bc324dad65e4cbc740fc7310afe88b2"} Jul 22 03:08:02.116150 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:02.116120 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rv678" event={"ID":"590f0095-ecb1-43e4-9c1e-1c62a87a3b8a","Type":"ContainerStarted","Data":"b79de49a7883d1132fcde5780c68e42d60495ae4f1850915951851b76cbc650b"} Jul 22 03:08:02.117801 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:02.117776 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w85fv" event={"ID":"84476432-16cb-4559-bf1f-4fe42ea2e87a","Type":"ContainerStarted","Data":"9b4883571b324ff04dabacd43f00534b037b2f693df52023beeb1c1e1ae515a1"} Jul 22 03:08:02.120886 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:02.120862 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscf8_a01dfba4-f590-412a-8438-1646c51bb1a1/ovn-acl-logging/0.log" Jul 22 03:08:02.121264 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:02.121242 2556 generic.go:356] "Generic (PLEG): container finished" podID="a01dfba4-f590-412a-8438-1646c51bb1a1" containerID="9dcea0970ea00380599b5fb2e7313c052c237f486ed43ab6cd7fcd56ea161096" exitCode=1 Jul 22 03:08:02.121370 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:02.121273 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" event={"ID":"a01dfba4-f590-412a-8438-1646c51bb1a1","Type":"ContainerStarted","Data":"48723d3a4242c67f9d48fe32b161224b5371fdc7b74e231c4cdd68d2fd66c3a5"} Jul 22 03:08:02.121370 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:02.121338 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" event={"ID":"a01dfba4-f590-412a-8438-1646c51bb1a1","Type":"ContainerStarted","Data":"af9ecd1adf7ed60720751bd0e8073f81d63cfe37c2120db2c634ea4be4e264ac"} Jul 22 03:08:02.121480 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:02.121390 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" event={"ID":"a01dfba4-f590-412a-8438-1646c51bb1a1","Type":"ContainerStarted","Data":"0a1c421fdbc8a80741ea6660c1fb0bab2878360e4654a7165f1613b1cde198d5"} Jul 22 03:08:02.121480 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:02.121441 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" event={"ID":"a01dfba4-f590-412a-8438-1646c51bb1a1","Type":"ContainerStarted","Data":"ae10e7d6f609124aa4ac2b2b2718b8cb3bb73c9152d9383ed4eb2ab1f39213d6"} Jul 22 03:08:02.121584 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:02.121488 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" event={"ID":"a01dfba4-f590-412a-8438-1646c51bb1a1","Type":"ContainerDied","Data":"9dcea0970ea00380599b5fb2e7313c052c237f486ed43ab6cd7fcd56ea161096"} Jul 22 03:08:02.121584 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:02.121539 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" event={"ID":"a01dfba4-f590-412a-8438-1646c51bb1a1","Type":"ContainerStarted","Data":"1d22c485db0a142f37c260850093fbf0d3f66cb3db5682e125000c221ce5a135"} Jul 22 03:08:02.126136 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:02.126098 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-kkcg5" podStartSLOduration=3.674944631 podStartE2EDuration="22.126086803s" totalImagesPullingTime="18.451142172s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:07:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:08:02.124628319 +0000 UTC m=+22.573611593" watchObservedRunningTime="2026-07-22 03:08:02.126086803 +0000 UTC m=+22.575070052" Jul 22 03:08:02.144785 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:02.144740 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rv678" podStartSLOduration=3.693342444 podStartE2EDuration="22.144728337s" totalImagesPullingTime="18.451385893s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:07:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:08:02.14451819 +0000 UTC m=+22.593501440" watchObservedRunningTime="2026-07-22 03:08:02.144728337 +0000 UTC m=+22.593711585" Jul 22 03:08:02.211831 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:02.211779 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-vr69p" podStartSLOduration=3.730214952 podStartE2EDuration="22.211762449s" totalImagesPullingTime="18.481547497s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:07:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:08:02.211445359 +0000 UTC m=+22.660428609" watchObservedRunningTime="2026-07-22 03:08:02.211762449 +0000 UTC m=+22.660745698" Jul 22 03:08:02.232763 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:02.232724 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-68xqw" podStartSLOduration=8.275802801 podStartE2EDuration="22.23271066s" totalImagesPullingTime="13.956907859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:07:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:08:02.232382881 +0000 UTC m=+22.681366129" watchObservedRunningTime="2026-07-22 03:08:02.23271066 +0000 UTC m=+22.681693921" Jul 22 03:08:02.275184 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:02.275129 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fwk5q" podStartSLOduration=3.769909704 podStartE2EDuration="22.275111497s" totalImagesPullingTime="18.505201793s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:07:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:08:02.274918017 +0000 UTC m=+22.723901267" watchObservedRunningTime="2026-07-22 03:08:02.275111497 +0000 UTC m=+22.724094747" Jul 22 03:08:02.469356 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:02.469324 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:02.543352 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:02.543318 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:02.691640 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:02.691617 2556 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 22 03:08:02.924302 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:02.924210 2556 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-22T03:08:02.691636434Z","UUID":"d29a5c8a-7f63-48fe-87b3-80936c632340","Handler":null,"Name":"","Endpoint":""} Jul 22 03:08:02.925673 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:02.925647 2556 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 22 03:08:02.925759 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:02.925682 2556 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 22 03:08:03.125158 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:03.125125 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w85fv" event={"ID":"84476432-16cb-4559-bf1f-4fe42ea2e87a","Type":"ContainerStarted","Data":"3fb2c9d50689ac3a20f0213720dc12aa625250d2c2800a6fc11b354fb8d0b0a6"} Jul 22 03:08:03.126488 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:03.126453 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-n94lh" event={"ID":"7bfceba5-9ca9-4955-abfd-1c029a7e6062","Type":"ContainerStarted","Data":"3af083a2212e5e52463e2a206eaeb09fb3553378b27f13a9111d24d862463ce8"} Jul 22 03:08:03.143165 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:03.143127 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-n94lh" podStartSLOduration=4.6923973839999995 podStartE2EDuration="23.143113687s" totalImagesPullingTime="18.450716303s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:07:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:08:03.1430724 +0000 UTC m=+23.592055649" watchObservedRunningTime="2026-07-22 03:08:03.143113687 +0000 UTC m=+23.592096936" Jul 22 03:08:03.265920 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:03.265836 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:03.358302 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:03.358266 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:04.022601 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:04.022574 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrbdq" Jul 22 03:08:04.022601 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:04.022605 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qhg45" Jul 22 03:08:04.022762 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:04.022605 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4rxt" Jul 22 03:08:04.022762 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:04.022704 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrbdq" podUID="bb8b43a6-b50f-41e9-9e68-763231228640" Jul 22 03:08:04.022865 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:04.022804 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qhg45" podUID="1297a647-da14-4c7b-bf94-6f4f4e679c31" Jul 22 03:08:04.022915 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:04.022902 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k4rxt" podUID="1c2c0875-8ece-494b-95a1-00fe315db1e4" Jul 22 03:08:04.130881 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:04.130803 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w85fv" event={"ID":"84476432-16cb-4559-bf1f-4fe42ea2e87a","Type":"ContainerStarted","Data":"d5377a841241502c41a6bac1e61f3d524c9dfa716b15042fecfde6f31b8ed187"} Jul 22 03:08:04.133526 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:04.133510 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscf8_a01dfba4-f590-412a-8438-1646c51bb1a1/ovn-acl-logging/0.log" Jul 22 03:08:04.133877 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:04.133858 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" event={"ID":"a01dfba4-f590-412a-8438-1646c51bb1a1","Type":"ContainerStarted","Data":"0353550764bca648b6a156500584294ec98b7922e0db0ba0795c93f34c360070"} Jul 22 03:08:04.149962 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:04.149916 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w85fv" podStartSLOduration=3.180245385 podStartE2EDuration="24.149904972s" totalImagesPullingTime="20.969659587s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:07:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:08:04.149616126 +0000 UTC m=+24.598599375" watchObservedRunningTime="2026-07-22 03:08:04.149904972 +0000 UTC m=+24.598888220" Jul 22 03:08:04.453410 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:04.453331 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-68xqw" Jul 22 03:08:04.453936 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:04.453917 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-68xqw" Jul 22 03:08:05.136405 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:05.136156 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-68xqw" Jul 22 03:08:05.136838 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:05.136607 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-68xqw" Jul 22 03:08:05.210857 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:05.210820 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:05.472627 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:05.472550 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:05.571392 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:05.571360 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:05.675177 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:05.675142 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:05.738662 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:05.738580 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:06.021080 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:06.021008 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrbdq" Jul 22 03:08:06.021080 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:06.021021 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qhg45" Jul 22 03:08:06.021324 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:06.021013 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4rxt" Jul 22 03:08:06.021324 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:06.021132 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrbdq" podUID="bb8b43a6-b50f-41e9-9e68-763231228640" Jul 22 03:08:06.021324 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:06.021208 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k4rxt" podUID="1c2c0875-8ece-494b-95a1-00fe315db1e4" Jul 22 03:08:06.021324 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:06.021291 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qhg45" podUID="1297a647-da14-4c7b-bf94-6f4f4e679c31" Jul 22 03:08:06.478515 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:06.478478 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:06.798877 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:06.798776 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:06.914159 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:06.914118 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:07.412570 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:07.412408 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:07.511517 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:07.511489 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:08.020468 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:08.020442 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qhg45" Jul 22 03:08:08.020641 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:08.020533 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrbdq" Jul 22 03:08:08.020641 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:08.020537 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qhg45" podUID="1297a647-da14-4c7b-bf94-6f4f4e679c31" Jul 22 03:08:08.020641 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:08.020577 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4rxt" Jul 22 03:08:08.020806 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:08.020666 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrbdq" podUID="bb8b43a6-b50f-41e9-9e68-763231228640" Jul 22 03:08:08.020806 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:08.020748 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k4rxt" podUID="1c2c0875-8ece-494b-95a1-00fe315db1e4" Jul 22 03:08:08.144960 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:08.144922 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscf8_a01dfba4-f590-412a-8438-1646c51bb1a1/ovn-acl-logging/0.log" Jul 22 03:08:08.145349 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:08.145322 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" event={"ID":"a01dfba4-f590-412a-8438-1646c51bb1a1","Type":"ContainerStarted","Data":"812b29c26a1104cda143758cce63cd5da872933cab09c5d09d2ef4ff8fb84ae1"} Jul 22 03:08:08.145576 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:08.145557 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:08:08.145770 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:08.145754 2556 scope.go:117] "RemoveContainer" containerID="9dcea0970ea00380599b5fb2e7313c052c237f486ed43ab6cd7fcd56ea161096" Jul 22 03:08:08.147201 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:08.147177 2556 generic.go:356] "Generic (PLEG): container finished" podID="b9f02261-c5da-4a76-aa41-e09c6955ef76" containerID="e68f88f6fa04737a3814931c04573898a64b3fa0f230ea0217ef4f4823490da2" exitCode=0 Jul 22 03:08:08.147295 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:08.147213 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wth5q" event={"ID":"b9f02261-c5da-4a76-aa41-e09c6955ef76","Type":"ContainerDied","Data":"e68f88f6fa04737a3814931c04573898a64b3fa0f230ea0217ef4f4823490da2"} Jul 22 03:08:08.161632 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:08.161594 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:08:09.151647 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:09.151560 2556 generic.go:356] "Generic (PLEG): container finished" podID="b9f02261-c5da-4a76-aa41-e09c6955ef76" containerID="3d2e3133e862a9800403c87d02be822d7ab170c138e69ec6d225406a2b799ba7" exitCode=0 Jul 22 03:08:09.152031 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:09.151641 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wth5q" event={"ID":"b9f02261-c5da-4a76-aa41-e09c6955ef76","Type":"ContainerDied","Data":"3d2e3133e862a9800403c87d02be822d7ab170c138e69ec6d225406a2b799ba7"} Jul 22 03:08:09.155324 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:09.155308 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscf8_a01dfba4-f590-412a-8438-1646c51bb1a1/ovn-acl-logging/0.log" Jul 22 03:08:09.155644 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:09.155623 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" event={"ID":"a01dfba4-f590-412a-8438-1646c51bb1a1","Type":"ContainerStarted","Data":"e30acbcc3edcced98db8b89cee21154e0a40defe94d9947bd9b9097003346eca"} Jul 22 03:08:09.156027 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:09.156007 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:08:09.156107 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:09.156067 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:08:09.170273 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:09.170253 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:08:09.215777 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:09.215731 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" podStartSLOduration=10.660370333 podStartE2EDuration="29.215716274s" totalImagesPullingTime="18.555345941s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:07:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:08:09.213785219 +0000 UTC m=+29.662768469" watchObservedRunningTime="2026-07-22 03:08:09.215716274 +0000 UTC m=+29.664699522" Jul 22 03:08:09.470770 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:09.470694 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qhg45"] Jul 22 03:08:09.470899 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:09.470805 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qhg45" Jul 22 03:08:09.470899 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:09.470884 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qhg45" podUID="1297a647-da14-4c7b-bf94-6f4f4e679c31" Jul 22 03:08:09.473439 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:09.473414 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-k4rxt"] Jul 22 03:08:09.473562 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:09.473512 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4rxt" Jul 22 03:08:09.473619 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:09.473603 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k4rxt" podUID="1c2c0875-8ece-494b-95a1-00fe315db1e4" Jul 22 03:08:09.474048 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:09.474026 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xrbdq"] Jul 22 03:08:09.474144 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:09.474125 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrbdq" Jul 22 03:08:09.474242 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:09.474226 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrbdq" podUID="bb8b43a6-b50f-41e9-9e68-763231228640" Jul 22 03:08:09.578521 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:09.578490 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:09.638314 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:09.638278 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:10.164049 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:10.163852 2556 generic.go:356] "Generic (PLEG): container finished" podID="b9f02261-c5da-4a76-aa41-e09c6955ef76" containerID="907b99c232601bc9ed18d83ea221a91190cab5badfe2899e80b9963c0a97af76" exitCode=0 Jul 22 03:08:10.164049 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:10.163931 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wth5q" event={"ID":"b9f02261-c5da-4a76-aa41-e09c6955ef76","Type":"ContainerDied","Data":"907b99c232601bc9ed18d83ea221a91190cab5badfe2899e80b9963c0a97af76"} Jul 22 03:08:10.494601 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:10.494519 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:11.017679 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:11.017644 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qhg45" Jul 22 03:08:11.017849 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:11.017754 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4rxt" Jul 22 03:08:11.017849 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:11.017764 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qhg45" podUID="1297a647-da14-4c7b-bf94-6f4f4e679c31" Jul 22 03:08:11.017849 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:11.017783 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrbdq" Jul 22 03:08:11.018026 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:11.017865 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k4rxt" podUID="1c2c0875-8ece-494b-95a1-00fe315db1e4" Jul 22 03:08:11.018026 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:11.017965 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrbdq" podUID="bb8b43a6-b50f-41e9-9e68-763231228640" Jul 22 03:08:11.513537 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:11.513503 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:11.589577 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:11.589543 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:12.450873 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:12.450848 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kkcg5_497ea3ce-7ad8-42f9-a52e-8094ec646cc9/dns-node-resolver/0.log" Jul 22 03:08:13.017842 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:13.017808 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qhg45" Jul 22 03:08:13.017842 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:13.017826 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrbdq" Jul 22 03:08:13.017842 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:13.017840 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4rxt" Jul 22 03:08:13.018508 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:13.017940 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qhg45" podUID="1297a647-da14-4c7b-bf94-6f4f4e679c31" Jul 22 03:08:13.018508 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:13.018037 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrbdq" podUID="bb8b43a6-b50f-41e9-9e68-763231228640" Jul 22 03:08:13.018508 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:13.018121 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k4rxt" podUID="1c2c0875-8ece-494b-95a1-00fe315db1e4" Jul 22 03:08:13.614461 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:13.614421 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1297a647-da14-4c7b-bf94-6f4f4e679c31-original-pull-secret\") pod \"global-pull-secret-syncer-qhg45\" (UID: \"1297a647-da14-4c7b-bf94-6f4f4e679c31\") " pod="kube-system/global-pull-secret-syncer-qhg45" Jul 22 03:08:13.614461 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:13.614468 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb8b43a6-b50f-41e9-9e68-763231228640-metrics-certs\") pod \"network-metrics-daemon-xrbdq\" (UID: \"bb8b43a6-b50f-41e9-9e68-763231228640\") " pod="openshift-multus/network-metrics-daemon-xrbdq" Jul 22 03:08:13.614673 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:13.614586 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 03:08:13.614673 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:13.614611 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 03:08:13.614673 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:13.614665 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1297a647-da14-4c7b-bf94-6f4f4e679c31-original-pull-secret podName:1297a647-da14-4c7b-bf94-6f4f4e679c31 nodeName:}" failed. No retries permitted until 2026-07-22 03:08:45.614644289 +0000 UTC m=+66.063627516 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1297a647-da14-4c7b-bf94-6f4f4e679c31-original-pull-secret") pod "global-pull-secret-syncer-qhg45" (UID: "1297a647-da14-4c7b-bf94-6f4f4e679c31") : object "kube-system"/"original-pull-secret" not registered Jul 22 03:08:13.614795 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:13.614688 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb8b43a6-b50f-41e9-9e68-763231228640-metrics-certs podName:bb8b43a6-b50f-41e9-9e68-763231228640 nodeName:}" failed. No retries permitted until 2026-07-22 03:08:45.614677998 +0000 UTC m=+66.063661229 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bb8b43a6-b50f-41e9-9e68-763231228640-metrics-certs") pod "network-metrics-daemon-xrbdq" (UID: "bb8b43a6-b50f-41e9-9e68-763231228640") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 03:08:13.715876 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:13.715842 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x25qb\" (UniqueName: \"kubernetes.io/projected/1c2c0875-8ece-494b-95a1-00fe315db1e4-kube-api-access-x25qb\") pod \"network-check-target-k4rxt\" (UID: \"1c2c0875-8ece-494b-95a1-00fe315db1e4\") " pod="openshift-network-diagnostics/network-check-target-k4rxt" Jul 22 03:08:13.716097 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:13.716074 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 03:08:13.716159 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:13.716106 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 03:08:13.716159 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:13.716119 2556 projected.go:196] Error preparing data for projected volume kube-api-access-x25qb for pod openshift-network-diagnostics/network-check-target-k4rxt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 03:08:13.716235 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:13.716193 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c2c0875-8ece-494b-95a1-00fe315db1e4-kube-api-access-x25qb podName:1c2c0875-8ece-494b-95a1-00fe315db1e4 nodeName:}" failed. No retries permitted until 2026-07-22 03:08:45.716173994 +0000 UTC m=+66.165157243 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-x25qb" (UniqueName: "kubernetes.io/projected/1c2c0875-8ece-494b-95a1-00fe315db1e4-kube-api-access-x25qb") pod "network-check-target-k4rxt" (UID: "1c2c0875-8ece-494b-95a1-00fe315db1e4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 03:08:13.849431 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:13.849399 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rv678_590f0095-ecb1-43e4-9c1e-1c62a87a3b8a/node-ca/0.log" Jul 22 03:08:14.917875 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:14.917848 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-13.ec2.internal" event="NodeReady" Jul 22 03:08:14.918349 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:14.917998 2556 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 22 03:08:14.957590 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:14.957530 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7fcbd97f78-7hd7h"] Jul 22 03:08:14.978843 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:14.978816 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" Jul 22 03:08:14.979535 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:14.979497 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7fcbd97f78-7hd7h"] Jul 22 03:08:14.979996 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:14.979977 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zs5nq"] Jul 22 03:08:14.982513 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:14.982056 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 22 03:08:14.982513 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:14.982120 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 22 03:08:14.982513 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:14.982330 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-ndl8t\"" Jul 22 03:08:14.982513 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:14.982387 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 22 03:08:14.988207 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:14.988187 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 22 03:08:14.996740 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:14.996709 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qb5ws"] Jul 22 03:08:14.997084 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:14.997066 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zs5nq" Jul 22 03:08:14.999556 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:14.999536 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 22 03:08:14.999668 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:14.999574 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ms47k\"" Jul 22 03:08:14.999668 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:14.999596 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 22 03:08:14.999943 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:14.999921 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 22 03:08:14.999943 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:14.999933 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 22 03:08:15.016337 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.016312 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qb5ws" Jul 22 03:08:15.016337 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.016325 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zs5nq"] Jul 22 03:08:15.016476 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.016370 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qb5ws"] Jul 22 03:08:15.018243 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.018224 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4rxt" Jul 22 03:08:15.018375 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.018362 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qhg45" Jul 22 03:08:15.018553 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.018535 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrbdq" Jul 22 03:08:15.018839 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.018822 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 22 03:08:15.018839 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.018835 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 22 03:08:15.018947 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.018903 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-pc67s\"" Jul 22 03:08:15.019012 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.018978 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 22 03:08:15.020802 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.020784 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 22 03:08:15.020889 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.020836 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 22 03:08:15.020889 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.020862 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 03:08:15.021295 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.021241 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-glt2k\"" Jul 22 03:08:15.021295 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.021279 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 03:08:15.021427 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.021297 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4gddc\"" Jul 22 03:08:15.071711 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.071683 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:15.074653 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.074630 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-d8rr4"] Jul 22 03:08:15.087197 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.087178 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-d8rr4" Jul 22 03:08:15.090191 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.090134 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bmpz5\"" Jul 22 03:08:15.090342 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.090253 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 22 03:08:15.090441 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.090425 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 22 03:08:15.090626 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.090607 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-d8rr4"] Jul 22 03:08:15.124775 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.124755 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0dbb2e21-4aa1-43df-8917-adc7034719b0-registry-tls\") pod \"image-registry-7fcbd97f78-7hd7h\" (UID: \"0dbb2e21-4aa1-43df-8917-adc7034719b0\") " pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" Jul 22 03:08:15.124871 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.124783 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0dbb2e21-4aa1-43df-8917-adc7034719b0-ca-trust-extracted\") pod \"image-registry-7fcbd97f78-7hd7h\" (UID: \"0dbb2e21-4aa1-43df-8917-adc7034719b0\") " pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" Jul 22 03:08:15.124871 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.124815 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0dbb2e21-4aa1-43df-8917-adc7034719b0-bound-sa-token\") pod \"image-registry-7fcbd97f78-7hd7h\" (UID: \"0dbb2e21-4aa1-43df-8917-adc7034719b0\") " pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" Jul 22 03:08:15.124871 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.124833 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncgfn\" (UniqueName: \"kubernetes.io/projected/57b3d756-0f91-4adb-bfcf-92561b58d767-kube-api-access-ncgfn\") pod \"ingress-canary-qb5ws\" (UID: \"57b3d756-0f91-4adb-bfcf-92561b58d767\") " pod="openshift-ingress-canary/ingress-canary-qb5ws" Jul 22 03:08:15.125152 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.124875 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c5dc9285-733c-4cf4-abad-90ec9da9674d-crio-socket\") pod \"insights-runtime-extractor-zs5nq\" (UID: \"c5dc9285-733c-4cf4-abad-90ec9da9674d\") " pod="openshift-insights/insights-runtime-extractor-zs5nq" Jul 22 03:08:15.125152 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.124908 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c5dc9285-733c-4cf4-abad-90ec9da9674d-data-volume\") pod \"insights-runtime-extractor-zs5nq\" (UID: \"c5dc9285-733c-4cf4-abad-90ec9da9674d\") " pod="openshift-insights/insights-runtime-extractor-zs5nq" Jul 22 03:08:15.125152 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.124940 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0dbb2e21-4aa1-43df-8917-adc7034719b0-registry-certificates\") pod \"image-registry-7fcbd97f78-7hd7h\" (UID: \"0dbb2e21-4aa1-43df-8917-adc7034719b0\") " pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" Jul 22 03:08:15.125152 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.124978 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0dbb2e21-4aa1-43df-8917-adc7034719b0-image-registry-private-configuration\") pod \"image-registry-7fcbd97f78-7hd7h\" (UID: \"0dbb2e21-4aa1-43df-8917-adc7034719b0\") " pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" Jul 22 03:08:15.125152 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.124996 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5dc9285-733c-4cf4-abad-90ec9da9674d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zs5nq\" (UID: \"c5dc9285-733c-4cf4-abad-90ec9da9674d\") " pod="openshift-insights/insights-runtime-extractor-zs5nq" Jul 22 03:08:15.125152 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.125058 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0dbb2e21-4aa1-43df-8917-adc7034719b0-trusted-ca\") pod \"image-registry-7fcbd97f78-7hd7h\" (UID: \"0dbb2e21-4aa1-43df-8917-adc7034719b0\") " pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" Jul 22 03:08:15.125152 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.125099 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5dc9285-733c-4cf4-abad-90ec9da9674d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zs5nq\" (UID: \"c5dc9285-733c-4cf4-abad-90ec9da9674d\") " pod="openshift-insights/insights-runtime-extractor-zs5nq" Jul 22 03:08:15.125152 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.125115 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/57b3d756-0f91-4adb-bfcf-92561b58d767-cert\") pod \"ingress-canary-qb5ws\" (UID: \"57b3d756-0f91-4adb-bfcf-92561b58d767\") " pod="openshift-ingress-canary/ingress-canary-qb5ws" Jul 22 03:08:15.125152 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.125132 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0dbb2e21-4aa1-43df-8917-adc7034719b0-installation-pull-secrets\") pod \"image-registry-7fcbd97f78-7hd7h\" (UID: \"0dbb2e21-4aa1-43df-8917-adc7034719b0\") " pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" Jul 22 03:08:15.125152 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.125154 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ff4g6\" (UniqueName: \"kubernetes.io/projected/0dbb2e21-4aa1-43df-8917-adc7034719b0-kube-api-access-ff4g6\") pod \"image-registry-7fcbd97f78-7hd7h\" (UID: \"0dbb2e21-4aa1-43df-8917-adc7034719b0\") " pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" Jul 22 03:08:15.125489 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.125198 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5jlw\" (UniqueName: \"kubernetes.io/projected/c5dc9285-733c-4cf4-abad-90ec9da9674d-kube-api-access-l5jlw\") pod \"insights-runtime-extractor-zs5nq\" (UID: \"c5dc9285-733c-4cf4-abad-90ec9da9674d\") " pod="openshift-insights/insights-runtime-extractor-zs5nq" Jul 22 03:08:15.146607 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.146582 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:15.226320 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.226297 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c5dc9285-733c-4cf4-abad-90ec9da9674d-crio-socket\") pod \"insights-runtime-extractor-zs5nq\" (UID: \"c5dc9285-733c-4cf4-abad-90ec9da9674d\") " pod="openshift-insights/insights-runtime-extractor-zs5nq" Jul 22 03:08:15.226419 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.226328 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c5dc9285-733c-4cf4-abad-90ec9da9674d-data-volume\") pod \"insights-runtime-extractor-zs5nq\" (UID: \"c5dc9285-733c-4cf4-abad-90ec9da9674d\") " pod="openshift-insights/insights-runtime-extractor-zs5nq" Jul 22 03:08:15.226419 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.226349 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01af83ea-a187-4abb-ba83-7d45027fb098-tmp-dir\") pod \"dns-default-d8rr4\" (UID: \"01af83ea-a187-4abb-ba83-7d45027fb098\") " pod="openshift-dns/dns-default-d8rr4" Jul 22 03:08:15.226419 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.226403 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0dbb2e21-4aa1-43df-8917-adc7034719b0-registry-certificates\") pod \"image-registry-7fcbd97f78-7hd7h\" (UID: \"0dbb2e21-4aa1-43df-8917-adc7034719b0\") " pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" Jul 22 03:08:15.226556 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.226423 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0dbb2e21-4aa1-43df-8917-adc7034719b0-image-registry-private-configuration\") pod \"image-registry-7fcbd97f78-7hd7h\" (UID: \"0dbb2e21-4aa1-43df-8917-adc7034719b0\") " pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" Jul 22 03:08:15.226556 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.226446 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5dc9285-733c-4cf4-abad-90ec9da9674d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zs5nq\" (UID: \"c5dc9285-733c-4cf4-abad-90ec9da9674d\") " pod="openshift-insights/insights-runtime-extractor-zs5nq" Jul 22 03:08:15.226556 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.226470 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0dbb2e21-4aa1-43df-8917-adc7034719b0-trusted-ca\") pod \"image-registry-7fcbd97f78-7hd7h\" (UID: \"0dbb2e21-4aa1-43df-8917-adc7034719b0\") " pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" Jul 22 03:08:15.226556 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.226496 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5dc9285-733c-4cf4-abad-90ec9da9674d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zs5nq\" (UID: \"c5dc9285-733c-4cf4-abad-90ec9da9674d\") " pod="openshift-insights/insights-runtime-extractor-zs5nq" Jul 22 03:08:15.226556 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.226510 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/57b3d756-0f91-4adb-bfcf-92561b58d767-cert\") pod \"ingress-canary-qb5ws\" (UID: \"57b3d756-0f91-4adb-bfcf-92561b58d767\") " pod="openshift-ingress-canary/ingress-canary-qb5ws" Jul 22 03:08:15.226556 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.226526 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c5dc9285-733c-4cf4-abad-90ec9da9674d-crio-socket\") pod \"insights-runtime-extractor-zs5nq\" (UID: \"c5dc9285-733c-4cf4-abad-90ec9da9674d\") " pod="openshift-insights/insights-runtime-extractor-zs5nq" Jul 22 03:08:15.226837 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.226534 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0dbb2e21-4aa1-43df-8917-adc7034719b0-installation-pull-secrets\") pod \"image-registry-7fcbd97f78-7hd7h\" (UID: \"0dbb2e21-4aa1-43df-8917-adc7034719b0\") " pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" Jul 22 03:08:15.226837 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.226593 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ff4g6\" (UniqueName: \"kubernetes.io/projected/0dbb2e21-4aa1-43df-8917-adc7034719b0-kube-api-access-ff4g6\") pod \"image-registry-7fcbd97f78-7hd7h\" (UID: \"0dbb2e21-4aa1-43df-8917-adc7034719b0\") " pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" Jul 22 03:08:15.226837 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.226622 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5jlw\" (UniqueName: \"kubernetes.io/projected/c5dc9285-733c-4cf4-abad-90ec9da9674d-kube-api-access-l5jlw\") pod \"insights-runtime-extractor-zs5nq\" (UID: \"c5dc9285-733c-4cf4-abad-90ec9da9674d\") " pod="openshift-insights/insights-runtime-extractor-zs5nq" Jul 22 03:08:15.226837 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.226653 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/01af83ea-a187-4abb-ba83-7d45027fb098-config-volume\") pod \"dns-default-d8rr4\" (UID: \"01af83ea-a187-4abb-ba83-7d45027fb098\") " pod="openshift-dns/dns-default-d8rr4" Jul 22 03:08:15.226837 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.226677 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/01af83ea-a187-4abb-ba83-7d45027fb098-metrics-tls\") pod \"dns-default-d8rr4\" (UID: \"01af83ea-a187-4abb-ba83-7d45027fb098\") " pod="openshift-dns/dns-default-d8rr4" Jul 22 03:08:15.226837 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.226706 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c5dc9285-733c-4cf4-abad-90ec9da9674d-data-volume\") pod \"insights-runtime-extractor-zs5nq\" (UID: \"c5dc9285-733c-4cf4-abad-90ec9da9674d\") " pod="openshift-insights/insights-runtime-extractor-zs5nq" Jul 22 03:08:15.226837 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.226716 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0dbb2e21-4aa1-43df-8917-adc7034719b0-registry-tls\") pod \"image-registry-7fcbd97f78-7hd7h\" (UID: \"0dbb2e21-4aa1-43df-8917-adc7034719b0\") " pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" Jul 22 03:08:15.226837 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.226743 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0dbb2e21-4aa1-43df-8917-adc7034719b0-ca-trust-extracted\") pod \"image-registry-7fcbd97f78-7hd7h\" (UID: \"0dbb2e21-4aa1-43df-8917-adc7034719b0\") " pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" Jul 22 03:08:15.226837 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.226767 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2j6l\" (UniqueName: \"kubernetes.io/projected/01af83ea-a187-4abb-ba83-7d45027fb098-kube-api-access-m2j6l\") pod \"dns-default-d8rr4\" (UID: \"01af83ea-a187-4abb-ba83-7d45027fb098\") " pod="openshift-dns/dns-default-d8rr4" Jul 22 03:08:15.226837 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.226812 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0dbb2e21-4aa1-43df-8917-adc7034719b0-bound-sa-token\") pod \"image-registry-7fcbd97f78-7hd7h\" (UID: \"0dbb2e21-4aa1-43df-8917-adc7034719b0\") " pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" Jul 22 03:08:15.227323 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.226843 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncgfn\" (UniqueName: \"kubernetes.io/projected/57b3d756-0f91-4adb-bfcf-92561b58d767-kube-api-access-ncgfn\") pod \"ingress-canary-qb5ws\" (UID: \"57b3d756-0f91-4adb-bfcf-92561b58d767\") " pod="openshift-ingress-canary/ingress-canary-qb5ws" Jul 22 03:08:15.227323 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:15.226928 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 03:08:15.227323 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:15.226999 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5dc9285-733c-4cf4-abad-90ec9da9674d-insights-runtime-extractor-tls podName:c5dc9285-733c-4cf4-abad-90ec9da9674d nodeName:}" failed. No retries permitted until 2026-07-22 03:08:15.726982723 +0000 UTC m=+36.175965969 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c5dc9285-733c-4cf4-abad-90ec9da9674d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zs5nq" (UID: "c5dc9285-733c-4cf4-abad-90ec9da9674d") : secret "insights-runtime-extractor-tls" not found Jul 22 03:08:15.227323 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.227193 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0dbb2e21-4aa1-43df-8917-adc7034719b0-registry-certificates\") pod \"image-registry-7fcbd97f78-7hd7h\" (UID: \"0dbb2e21-4aa1-43df-8917-adc7034719b0\") " pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" Jul 22 03:08:15.227323 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:15.227223 2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 03:08:15.227323 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:15.227239 2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fcbd97f78-7hd7h: secret "image-registry-tls" not found Jul 22 03:08:15.227672 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:15.227335 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0dbb2e21-4aa1-43df-8917-adc7034719b0-registry-tls podName:0dbb2e21-4aa1-43df-8917-adc7034719b0 nodeName:}" failed. No retries permitted until 2026-07-22 03:08:15.727317414 +0000 UTC m=+36.176300644 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0dbb2e21-4aa1-43df-8917-adc7034719b0-registry-tls") pod "image-registry-7fcbd97f78-7hd7h" (UID: "0dbb2e21-4aa1-43df-8917-adc7034719b0") : secret "image-registry-tls" not found Jul 22 03:08:15.227672 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:15.227391 2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 03:08:15.227672 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:15.227439 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c5dc9285-733c-4cf4-abad-90ec9da9674d-kube-rbac-proxy-cm podName:c5dc9285-733c-4cf4-abad-90ec9da9674d nodeName:}" failed. No retries permitted until 2026-07-22 03:08:15.72742732 +0000 UTC m=+36.176410584 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c5dc9285-733c-4cf4-abad-90ec9da9674d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zs5nq" (UID: "c5dc9285-733c-4cf4-abad-90ec9da9674d") : configmap "kube-rbac-proxy" not found Jul 22 03:08:15.227825 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.227704 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0dbb2e21-4aa1-43df-8917-adc7034719b0-trusted-ca\") pod \"image-registry-7fcbd97f78-7hd7h\" (UID: \"0dbb2e21-4aa1-43df-8917-adc7034719b0\") " pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" Jul 22 03:08:15.227825 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.227717 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0dbb2e21-4aa1-43df-8917-adc7034719b0-ca-trust-extracted\") pod \"image-registry-7fcbd97f78-7hd7h\" (UID: \"0dbb2e21-4aa1-43df-8917-adc7034719b0\") " pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" Jul 22 03:08:15.232064 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.232038 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0dbb2e21-4aa1-43df-8917-adc7034719b0-image-registry-private-configuration\") pod \"image-registry-7fcbd97f78-7hd7h\" (UID: \"0dbb2e21-4aa1-43df-8917-adc7034719b0\") " pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" Jul 22 03:08:15.232172 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.232048 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/57b3d756-0f91-4adb-bfcf-92561b58d767-cert\") pod \"ingress-canary-qb5ws\" (UID: \"57b3d756-0f91-4adb-bfcf-92561b58d767\") " pod="openshift-ingress-canary/ingress-canary-qb5ws" Jul 22 03:08:15.232172 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.232096 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0dbb2e21-4aa1-43df-8917-adc7034719b0-installation-pull-secrets\") pod \"image-registry-7fcbd97f78-7hd7h\" (UID: \"0dbb2e21-4aa1-43df-8917-adc7034719b0\") " pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" Jul 22 03:08:15.236446 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.236337 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0dbb2e21-4aa1-43df-8917-adc7034719b0-bound-sa-token\") pod \"image-registry-7fcbd97f78-7hd7h\" (UID: \"0dbb2e21-4aa1-43df-8917-adc7034719b0\") " pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" Jul 22 03:08:15.236540 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.236521 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ff4g6\" (UniqueName: \"kubernetes.io/projected/0dbb2e21-4aa1-43df-8917-adc7034719b0-kube-api-access-ff4g6\") pod \"image-registry-7fcbd97f78-7hd7h\" (UID: \"0dbb2e21-4aa1-43df-8917-adc7034719b0\") " pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" Jul 22 03:08:15.236609 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.236588 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5jlw\" (UniqueName: \"kubernetes.io/projected/c5dc9285-733c-4cf4-abad-90ec9da9674d-kube-api-access-l5jlw\") pod \"insights-runtime-extractor-zs5nq\" (UID: \"c5dc9285-733c-4cf4-abad-90ec9da9674d\") " pod="openshift-insights/insights-runtime-extractor-zs5nq" Jul 22 03:08:15.237296 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.237277 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncgfn\" (UniqueName: \"kubernetes.io/projected/57b3d756-0f91-4adb-bfcf-92561b58d767-kube-api-access-ncgfn\") pod \"ingress-canary-qb5ws\" (UID: \"57b3d756-0f91-4adb-bfcf-92561b58d767\") " pod="openshift-ingress-canary/ingress-canary-qb5ws" Jul 22 03:08:15.246415 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.246394 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:15.327518 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.327492 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/01af83ea-a187-4abb-ba83-7d45027fb098-config-volume\") pod \"dns-default-d8rr4\" (UID: \"01af83ea-a187-4abb-ba83-7d45027fb098\") " pod="openshift-dns/dns-default-d8rr4" Jul 22 03:08:15.327624 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.327528 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/01af83ea-a187-4abb-ba83-7d45027fb098-metrics-tls\") pod \"dns-default-d8rr4\" (UID: \"01af83ea-a187-4abb-ba83-7d45027fb098\") " pod="openshift-dns/dns-default-d8rr4" Jul 22 03:08:15.327624 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.327545 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qb5ws" Jul 22 03:08:15.327624 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.327570 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2j6l\" (UniqueName: \"kubernetes.io/projected/01af83ea-a187-4abb-ba83-7d45027fb098-kube-api-access-m2j6l\") pod \"dns-default-d8rr4\" (UID: \"01af83ea-a187-4abb-ba83-7d45027fb098\") " pod="openshift-dns/dns-default-d8rr4" Jul 22 03:08:15.327771 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.327624 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01af83ea-a187-4abb-ba83-7d45027fb098-tmp-dir\") pod \"dns-default-d8rr4\" (UID: \"01af83ea-a187-4abb-ba83-7d45027fb098\") " pod="openshift-dns/dns-default-d8rr4" Jul 22 03:08:15.327771 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:15.327711 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 03:08:15.327858 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:15.327787 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01af83ea-a187-4abb-ba83-7d45027fb098-metrics-tls podName:01af83ea-a187-4abb-ba83-7d45027fb098 nodeName:}" failed. No retries permitted until 2026-07-22 03:08:15.827768566 +0000 UTC m=+36.276751798 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/01af83ea-a187-4abb-ba83-7d45027fb098-metrics-tls") pod "dns-default-d8rr4" (UID: "01af83ea-a187-4abb-ba83-7d45027fb098") : secret "dns-default-metrics-tls" not found Jul 22 03:08:15.327928 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.327910 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01af83ea-a187-4abb-ba83-7d45027fb098-tmp-dir\") pod \"dns-default-d8rr4\" (UID: \"01af83ea-a187-4abb-ba83-7d45027fb098\") " pod="openshift-dns/dns-default-d8rr4" Jul 22 03:08:15.328210 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.328191 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/01af83ea-a187-4abb-ba83-7d45027fb098-config-volume\") pod \"dns-default-d8rr4\" (UID: \"01af83ea-a187-4abb-ba83-7d45027fb098\") " pod="openshift-dns/dns-default-d8rr4" Jul 22 03:08:15.335908 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.335888 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2j6l\" (UniqueName: \"kubernetes.io/projected/01af83ea-a187-4abb-ba83-7d45027fb098-kube-api-access-m2j6l\") pod \"dns-default-d8rr4\" (UID: \"01af83ea-a187-4abb-ba83-7d45027fb098\") " pod="openshift-dns/dns-default-d8rr4" Jul 22 03:08:15.512345 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.512126 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qb5ws"] Jul 22 03:08:15.515153 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:08:15.515131 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57b3d756_0f91_4adb_bfcf_92561b58d767.slice/crio-570034e0519a7870c701e9a0e8da3369147bae6a4d2800e02bdbc9837e7bac7a WatchSource:0}: Error finding container 570034e0519a7870c701e9a0e8da3369147bae6a4d2800e02bdbc9837e7bac7a: Status 404 returned error can't find the container with id 570034e0519a7870c701e9a0e8da3369147bae6a4d2800e02bdbc9837e7bac7a Jul 22 03:08:15.731303 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.731269 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5dc9285-733c-4cf4-abad-90ec9da9674d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zs5nq\" (UID: \"c5dc9285-733c-4cf4-abad-90ec9da9674d\") " pod="openshift-insights/insights-runtime-extractor-zs5nq" Jul 22 03:08:15.731303 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.731309 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5dc9285-733c-4cf4-abad-90ec9da9674d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zs5nq\" (UID: \"c5dc9285-733c-4cf4-abad-90ec9da9674d\") " pod="openshift-insights/insights-runtime-extractor-zs5nq" Jul 22 03:08:15.731498 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.731339 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0dbb2e21-4aa1-43df-8917-adc7034719b0-registry-tls\") pod \"image-registry-7fcbd97f78-7hd7h\" (UID: \"0dbb2e21-4aa1-43df-8917-adc7034719b0\") " pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" Jul 22 03:08:15.731498 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:15.731420 2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 03:08:15.731498 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:15.731422 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 03:08:15.731498 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:15.731479 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5dc9285-733c-4cf4-abad-90ec9da9674d-insights-runtime-extractor-tls podName:c5dc9285-733c-4cf4-abad-90ec9da9674d nodeName:}" failed. No retries permitted until 2026-07-22 03:08:16.7314635 +0000 UTC m=+37.180446727 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c5dc9285-733c-4cf4-abad-90ec9da9674d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zs5nq" (UID: "c5dc9285-733c-4cf4-abad-90ec9da9674d") : secret "insights-runtime-extractor-tls" not found Jul 22 03:08:15.731498 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:15.731431 2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fcbd97f78-7hd7h: secret "image-registry-tls" not found Jul 22 03:08:15.731677 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:15.731499 2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 03:08:15.731677 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:15.731552 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c5dc9285-733c-4cf4-abad-90ec9da9674d-kube-rbac-proxy-cm podName:c5dc9285-733c-4cf4-abad-90ec9da9674d nodeName:}" failed. No retries permitted until 2026-07-22 03:08:16.731537903 +0000 UTC m=+37.180521149 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c5dc9285-733c-4cf4-abad-90ec9da9674d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zs5nq" (UID: "c5dc9285-733c-4cf4-abad-90ec9da9674d") : configmap "kube-rbac-proxy" not found Jul 22 03:08:15.731677 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:15.731566 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0dbb2e21-4aa1-43df-8917-adc7034719b0-registry-tls podName:0dbb2e21-4aa1-43df-8917-adc7034719b0 nodeName:}" failed. No retries permitted until 2026-07-22 03:08:16.731560332 +0000 UTC m=+37.180543593 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0dbb2e21-4aa1-43df-8917-adc7034719b0-registry-tls") pod "image-registry-7fcbd97f78-7hd7h" (UID: "0dbb2e21-4aa1-43df-8917-adc7034719b0") : secret "image-registry-tls" not found Jul 22 03:08:15.832420 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:15.832389 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/01af83ea-a187-4abb-ba83-7d45027fb098-metrics-tls\") pod \"dns-default-d8rr4\" (UID: \"01af83ea-a187-4abb-ba83-7d45027fb098\") " pod="openshift-dns/dns-default-d8rr4" Jul 22 03:08:15.832548 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:15.832489 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 03:08:15.832548 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:15.832535 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01af83ea-a187-4abb-ba83-7d45027fb098-metrics-tls podName:01af83ea-a187-4abb-ba83-7d45027fb098 nodeName:}" failed. No retries permitted until 2026-07-22 03:08:16.832523823 +0000 UTC m=+37.281507051 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/01af83ea-a187-4abb-ba83-7d45027fb098-metrics-tls") pod "dns-default-d8rr4" (UID: "01af83ea-a187-4abb-ba83-7d45027fb098") : secret "dns-default-metrics-tls" not found Jul 22 03:08:16.181164 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:16.181079 2556 generic.go:356] "Generic (PLEG): container finished" podID="b9f02261-c5da-4a76-aa41-e09c6955ef76" containerID="4afeaa4a573e6ff747ed4c77433f1aa37646eb1fd16264172fed688c700255ba" exitCode=0 Jul 22 03:08:16.181658 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:16.181170 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wth5q" event={"ID":"b9f02261-c5da-4a76-aa41-e09c6955ef76","Type":"ContainerDied","Data":"4afeaa4a573e6ff747ed4c77433f1aa37646eb1fd16264172fed688c700255ba"} Jul 22 03:08:16.182481 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:16.182440 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qb5ws" event={"ID":"57b3d756-0f91-4adb-bfcf-92561b58d767","Type":"ContainerStarted","Data":"570034e0519a7870c701e9a0e8da3369147bae6a4d2800e02bdbc9837e7bac7a"} Jul 22 03:08:16.740858 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:16.740815 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5dc9285-733c-4cf4-abad-90ec9da9674d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zs5nq\" (UID: \"c5dc9285-733c-4cf4-abad-90ec9da9674d\") " pod="openshift-insights/insights-runtime-extractor-zs5nq" Jul 22 03:08:16.741037 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:16.740881 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5dc9285-733c-4cf4-abad-90ec9da9674d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zs5nq\" (UID: \"c5dc9285-733c-4cf4-abad-90ec9da9674d\") " pod="openshift-insights/insights-runtime-extractor-zs5nq" Jul 22 03:08:16.741037 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:16.740933 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0dbb2e21-4aa1-43df-8917-adc7034719b0-registry-tls\") pod \"image-registry-7fcbd97f78-7hd7h\" (UID: \"0dbb2e21-4aa1-43df-8917-adc7034719b0\") " pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" Jul 22 03:08:16.741037 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:16.740932 2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 03:08:16.741037 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:16.741034 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c5dc9285-733c-4cf4-abad-90ec9da9674d-kube-rbac-proxy-cm podName:c5dc9285-733c-4cf4-abad-90ec9da9674d nodeName:}" failed. No retries permitted until 2026-07-22 03:08:18.741013358 +0000 UTC m=+39.189996596 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c5dc9285-733c-4cf4-abad-90ec9da9674d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zs5nq" (UID: "c5dc9285-733c-4cf4-abad-90ec9da9674d") : configmap "kube-rbac-proxy" not found Jul 22 03:08:16.741258 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:16.741042 2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 03:08:16.741258 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:16.741050 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 03:08:16.741258 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:16.741060 2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fcbd97f78-7hd7h: secret "image-registry-tls" not found Jul 22 03:08:16.741258 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:16.741117 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0dbb2e21-4aa1-43df-8917-adc7034719b0-registry-tls podName:0dbb2e21-4aa1-43df-8917-adc7034719b0 nodeName:}" failed. No retries permitted until 2026-07-22 03:08:18.741100804 +0000 UTC m=+39.190084045 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0dbb2e21-4aa1-43df-8917-adc7034719b0-registry-tls") pod "image-registry-7fcbd97f78-7hd7h" (UID: "0dbb2e21-4aa1-43df-8917-adc7034719b0") : secret "image-registry-tls" not found Jul 22 03:08:16.741258 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:16.741139 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5dc9285-733c-4cf4-abad-90ec9da9674d-insights-runtime-extractor-tls podName:c5dc9285-733c-4cf4-abad-90ec9da9674d nodeName:}" failed. No retries permitted until 2026-07-22 03:08:18.741129973 +0000 UTC m=+39.190113200 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c5dc9285-733c-4cf4-abad-90ec9da9674d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zs5nq" (UID: "c5dc9285-733c-4cf4-abad-90ec9da9674d") : secret "insights-runtime-extractor-tls" not found Jul 22 03:08:16.842369 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:16.842338 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/01af83ea-a187-4abb-ba83-7d45027fb098-metrics-tls\") pod \"dns-default-d8rr4\" (UID: \"01af83ea-a187-4abb-ba83-7d45027fb098\") " pod="openshift-dns/dns-default-d8rr4" Jul 22 03:08:16.842548 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:16.842468 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 03:08:16.842548 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:16.842545 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01af83ea-a187-4abb-ba83-7d45027fb098-metrics-tls podName:01af83ea-a187-4abb-ba83-7d45027fb098 nodeName:}" failed. No retries permitted until 2026-07-22 03:08:18.842522264 +0000 UTC m=+39.291505507 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/01af83ea-a187-4abb-ba83-7d45027fb098-metrics-tls") pod "dns-default-d8rr4" (UID: "01af83ea-a187-4abb-ba83-7d45027fb098") : secret "dns-default-metrics-tls" not found Jul 22 03:08:17.190597 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:17.190552 2556 generic.go:356] "Generic (PLEG): container finished" podID="b9f02261-c5da-4a76-aa41-e09c6955ef76" containerID="34c3e68b9f1361837d41433dca9314e7fbce81f4e21109be14bb9e6ed3521d18" exitCode=0 Jul 22 03:08:17.191127 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:17.190619 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wth5q" event={"ID":"b9f02261-c5da-4a76-aa41-e09c6955ef76","Type":"ContainerDied","Data":"34c3e68b9f1361837d41433dca9314e7fbce81f4e21109be14bb9e6ed3521d18"} Jul 22 03:08:17.615406 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:17.615364 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:17.713856 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:17.713813 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:17.986578 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:17.986363 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:18.087881 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:18.087854 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:18.195632 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:18.195558 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wth5q" event={"ID":"b9f02261-c5da-4a76-aa41-e09c6955ef76","Type":"ContainerStarted","Data":"f6d31968520877de365337813590413c55012adf934921f7ed7884c4264f1918"} Jul 22 03:08:18.222113 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:18.222069 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wth5q" podStartSLOduration=8.23952682 podStartE2EDuration="38.222052819s" totalImagesPullingTime="29.982525999s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:07:40 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:08:18.220456579 +0000 UTC m=+38.669439822" watchObservedRunningTime="2026-07-22 03:08:18.222052819 +0000 UTC m=+38.671036068" Jul 22 03:08:18.347205 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:18.347170 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:18.432436 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:18.432402 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:18.661617 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:18.661582 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:18.731183 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:18.731149 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:18.758227 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:18.758202 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5dc9285-733c-4cf4-abad-90ec9da9674d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zs5nq\" (UID: \"c5dc9285-733c-4cf4-abad-90ec9da9674d\") " pod="openshift-insights/insights-runtime-extractor-zs5nq" Jul 22 03:08:18.758342 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:18.758239 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5dc9285-733c-4cf4-abad-90ec9da9674d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zs5nq\" (UID: \"c5dc9285-733c-4cf4-abad-90ec9da9674d\") " pod="openshift-insights/insights-runtime-extractor-zs5nq" Jul 22 03:08:18.758342 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:18.758333 2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 03:08:18.758438 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:18.758357 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0dbb2e21-4aa1-43df-8917-adc7034719b0-registry-tls\") pod \"image-registry-7fcbd97f78-7hd7h\" (UID: \"0dbb2e21-4aa1-43df-8917-adc7034719b0\") " pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" Jul 22 03:08:18.758438 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:18.758400 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c5dc9285-733c-4cf4-abad-90ec9da9674d-kube-rbac-proxy-cm podName:c5dc9285-733c-4cf4-abad-90ec9da9674d nodeName:}" failed. No retries permitted until 2026-07-22 03:08:22.758380209 +0000 UTC m=+43.207363438 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c5dc9285-733c-4cf4-abad-90ec9da9674d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zs5nq" (UID: "c5dc9285-733c-4cf4-abad-90ec9da9674d") : configmap "kube-rbac-proxy" not found Jul 22 03:08:18.758529 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:18.758452 2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 03:08:18.758529 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:18.758461 2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fcbd97f78-7hd7h: secret "image-registry-tls" not found Jul 22 03:08:18.758529 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:18.758495 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0dbb2e21-4aa1-43df-8917-adc7034719b0-registry-tls podName:0dbb2e21-4aa1-43df-8917-adc7034719b0 nodeName:}" failed. No retries permitted until 2026-07-22 03:08:22.75848523 +0000 UTC m=+43.207468457 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0dbb2e21-4aa1-43df-8917-adc7034719b0-registry-tls") pod "image-registry-7fcbd97f78-7hd7h" (UID: "0dbb2e21-4aa1-43df-8917-adc7034719b0") : secret "image-registry-tls" not found Jul 22 03:08:18.758529 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:18.758450 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 03:08:18.758529 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:18.758525 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5dc9285-733c-4cf4-abad-90ec9da9674d-insights-runtime-extractor-tls podName:c5dc9285-733c-4cf4-abad-90ec9da9674d nodeName:}" failed. No retries permitted until 2026-07-22 03:08:22.758518585 +0000 UTC m=+43.207501812 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c5dc9285-733c-4cf4-abad-90ec9da9674d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zs5nq" (UID: "c5dc9285-733c-4cf4-abad-90ec9da9674d") : secret "insights-runtime-extractor-tls" not found Jul 22 03:08:18.859320 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:18.859289 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/01af83ea-a187-4abb-ba83-7d45027fb098-metrics-tls\") pod \"dns-default-d8rr4\" (UID: \"01af83ea-a187-4abb-ba83-7d45027fb098\") " pod="openshift-dns/dns-default-d8rr4" Jul 22 03:08:18.859473 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:18.859453 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 03:08:18.859535 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:18.859527 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01af83ea-a187-4abb-ba83-7d45027fb098-metrics-tls podName:01af83ea-a187-4abb-ba83-7d45027fb098 nodeName:}" failed. No retries permitted until 2026-07-22 03:08:22.859507545 +0000 UTC m=+43.308490791 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/01af83ea-a187-4abb-ba83-7d45027fb098-metrics-tls") pod "dns-default-d8rr4" (UID: "01af83ea-a187-4abb-ba83-7d45027fb098") : secret "dns-default-metrics-tls" not found Jul 22 03:08:19.159449 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:19.159412 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:19.199077 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:19.199046 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qb5ws" event={"ID":"57b3d756-0f91-4adb-bfcf-92561b58d767","Type":"ContainerStarted","Data":"b8ad69e0883d2e22647e16a35bf31079befbabac768155a98ddad530ab029412"} Jul 22 03:08:19.218622 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:19.218573 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qb5ws" podStartSLOduration=2.64970658 podStartE2EDuration="5.21856013s" totalImagesPullingTime="2.56885355s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:08:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:08:19.218526552 +0000 UTC m=+39.667509800" watchObservedRunningTime="2026-07-22 03:08:19.21856013 +0000 UTC m=+39.667543378" Jul 22 03:08:19.236018 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:19.235993 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:19.559973 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:19.559870 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:19.657087 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:19.657053 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:19.822698 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:19.822672 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:20.269232 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:20.269149 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:20.391910 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:20.391875 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:21.615389 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:21.615358 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:21.713843 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:21.713813 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:21.843997 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:21.843963 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-s8j5w"] Jul 22 03:08:21.878929 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:21.878859 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-s8j5w"] Jul 22 03:08:21.879085 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:21.878987 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-s8j5w" Jul 22 03:08:21.882267 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:21.882243 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 22 03:08:21.882374 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:21.882319 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 22 03:08:21.882374 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:21.882335 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 22 03:08:21.882460 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:21.882377 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 22 03:08:21.883596 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:21.883579 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-rxqv7\"" Jul 22 03:08:21.973656 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:21.973621 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:21.985635 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:21.985611 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/4ec55c86-a659-42c3-bbe4-72e4df40f7f7-signing-key\") pod \"service-ca-58598695f7-s8j5w\" (UID: \"4ec55c86-a659-42c3-bbe4-72e4df40f7f7\") " pod="openshift-service-ca/service-ca-58598695f7-s8j5w" Jul 22 03:08:21.985750 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:21.985643 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lg5nb\" (UniqueName: \"kubernetes.io/projected/4ec55c86-a659-42c3-bbe4-72e4df40f7f7-kube-api-access-lg5nb\") pod \"service-ca-58598695f7-s8j5w\" (UID: \"4ec55c86-a659-42c3-bbe4-72e4df40f7f7\") " pod="openshift-service-ca/service-ca-58598695f7-s8j5w" Jul 22 03:08:21.985750 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:21.985669 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/4ec55c86-a659-42c3-bbe4-72e4df40f7f7-signing-cabundle\") pod \"service-ca-58598695f7-s8j5w\" (UID: \"4ec55c86-a659-42c3-bbe4-72e4df40f7f7\") " pod="openshift-service-ca/service-ca-58598695f7-s8j5w" Jul 22 03:08:22.071298 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:22.071261 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:22.086278 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:22.086257 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/4ec55c86-a659-42c3-bbe4-72e4df40f7f7-signing-key\") pod \"service-ca-58598695f7-s8j5w\" (UID: \"4ec55c86-a659-42c3-bbe4-72e4df40f7f7\") " pod="openshift-service-ca/service-ca-58598695f7-s8j5w" Jul 22 03:08:22.086360 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:22.086285 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lg5nb\" (UniqueName: \"kubernetes.io/projected/4ec55c86-a659-42c3-bbe4-72e4df40f7f7-kube-api-access-lg5nb\") pod \"service-ca-58598695f7-s8j5w\" (UID: \"4ec55c86-a659-42c3-bbe4-72e4df40f7f7\") " pod="openshift-service-ca/service-ca-58598695f7-s8j5w" Jul 22 03:08:22.086360 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:22.086305 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/4ec55c86-a659-42c3-bbe4-72e4df40f7f7-signing-cabundle\") pod \"service-ca-58598695f7-s8j5w\" (UID: \"4ec55c86-a659-42c3-bbe4-72e4df40f7f7\") " pod="openshift-service-ca/service-ca-58598695f7-s8j5w" Jul 22 03:08:22.087006 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:22.086985 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/4ec55c86-a659-42c3-bbe4-72e4df40f7f7-signing-cabundle\") pod \"service-ca-58598695f7-s8j5w\" (UID: \"4ec55c86-a659-42c3-bbe4-72e4df40f7f7\") " pod="openshift-service-ca/service-ca-58598695f7-s8j5w" Jul 22 03:08:22.090312 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:22.090294 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/4ec55c86-a659-42c3-bbe4-72e4df40f7f7-signing-key\") pod \"service-ca-58598695f7-s8j5w\" (UID: \"4ec55c86-a659-42c3-bbe4-72e4df40f7f7\") " pod="openshift-service-ca/service-ca-58598695f7-s8j5w" Jul 22 03:08:22.095212 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:22.095192 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lg5nb\" (UniqueName: \"kubernetes.io/projected/4ec55c86-a659-42c3-bbe4-72e4df40f7f7-kube-api-access-lg5nb\") pod \"service-ca-58598695f7-s8j5w\" (UID: \"4ec55c86-a659-42c3-bbe4-72e4df40f7f7\") " pod="openshift-service-ca/service-ca-58598695f7-s8j5w" Jul 22 03:08:22.188195 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:22.188115 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-s8j5w" Jul 22 03:08:22.336441 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:22.336405 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-s8j5w"] Jul 22 03:08:22.339030 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:08:22.338999 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ec55c86_a659_42c3_bbe4_72e4df40f7f7.slice/crio-931d606e810753950bc63d4d6766ba40695109bf3a7b2110f07c6b0967344b2b WatchSource:0}: Error finding container 931d606e810753950bc63d4d6766ba40695109bf3a7b2110f07c6b0967344b2b: Status 404 returned error can't find the container with id 931d606e810753950bc63d4d6766ba40695109bf3a7b2110f07c6b0967344b2b Jul 22 03:08:22.502629 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:22.502546 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:22.564451 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:22.564419 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:22.790146 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:22.790071 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5dc9285-733c-4cf4-abad-90ec9da9674d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zs5nq\" (UID: \"c5dc9285-733c-4cf4-abad-90ec9da9674d\") " pod="openshift-insights/insights-runtime-extractor-zs5nq" Jul 22 03:08:22.790146 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:22.790111 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5dc9285-733c-4cf4-abad-90ec9da9674d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zs5nq\" (UID: \"c5dc9285-733c-4cf4-abad-90ec9da9674d\") " pod="openshift-insights/insights-runtime-extractor-zs5nq" Jul 22 03:08:22.790545 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:22.790191 2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 03:08:22.790545 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:22.790260 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c5dc9285-733c-4cf4-abad-90ec9da9674d-kube-rbac-proxy-cm podName:c5dc9285-733c-4cf4-abad-90ec9da9674d nodeName:}" failed. No retries permitted until 2026-07-22 03:08:30.790245077 +0000 UTC m=+51.239228303 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c5dc9285-733c-4cf4-abad-90ec9da9674d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zs5nq" (UID: "c5dc9285-733c-4cf4-abad-90ec9da9674d") : configmap "kube-rbac-proxy" not found Jul 22 03:08:22.790545 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:22.790278 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 03:08:22.790545 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:22.790330 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5dc9285-733c-4cf4-abad-90ec9da9674d-insights-runtime-extractor-tls podName:c5dc9285-733c-4cf4-abad-90ec9da9674d nodeName:}" failed. No retries permitted until 2026-07-22 03:08:30.790319368 +0000 UTC m=+51.239302595 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c5dc9285-733c-4cf4-abad-90ec9da9674d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zs5nq" (UID: "c5dc9285-733c-4cf4-abad-90ec9da9674d") : secret "insights-runtime-extractor-tls" not found Jul 22 03:08:22.790545 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:22.790346 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0dbb2e21-4aa1-43df-8917-adc7034719b0-registry-tls\") pod \"image-registry-7fcbd97f78-7hd7h\" (UID: \"0dbb2e21-4aa1-43df-8917-adc7034719b0\") " pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" Jul 22 03:08:22.790545 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:22.790450 2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 03:08:22.790545 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:22.790459 2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fcbd97f78-7hd7h: secret "image-registry-tls" not found Jul 22 03:08:22.790545 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:22.790491 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0dbb2e21-4aa1-43df-8917-adc7034719b0-registry-tls podName:0dbb2e21-4aa1-43df-8917-adc7034719b0 nodeName:}" failed. No retries permitted until 2026-07-22 03:08:30.790482471 +0000 UTC m=+51.239465698 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0dbb2e21-4aa1-43df-8917-adc7034719b0-registry-tls") pod "image-registry-7fcbd97f78-7hd7h" (UID: "0dbb2e21-4aa1-43df-8917-adc7034719b0") : secret "image-registry-tls" not found Jul 22 03:08:22.891229 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:22.891188 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/01af83ea-a187-4abb-ba83-7d45027fb098-metrics-tls\") pod \"dns-default-d8rr4\" (UID: \"01af83ea-a187-4abb-ba83-7d45027fb098\") " pod="openshift-dns/dns-default-d8rr4" Jul 22 03:08:22.891377 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:22.891325 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 03:08:22.891430 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:22.891389 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01af83ea-a187-4abb-ba83-7d45027fb098-metrics-tls podName:01af83ea-a187-4abb-ba83-7d45027fb098 nodeName:}" failed. No retries permitted until 2026-07-22 03:08:30.891370663 +0000 UTC m=+51.340353894 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/01af83ea-a187-4abb-ba83-7d45027fb098-metrics-tls") pod "dns-default-d8rr4" (UID: "01af83ea-a187-4abb-ba83-7d45027fb098") : secret "dns-default-metrics-tls" not found Jul 22 03:08:23.209994 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:23.209941 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-s8j5w" event={"ID":"4ec55c86-a659-42c3-bbe4-72e4df40f7f7","Type":"ContainerStarted","Data":"931d606e810753950bc63d4d6766ba40695109bf3a7b2110f07c6b0967344b2b"} Jul 22 03:08:23.312914 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:23.312873 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:24.654825 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:24.654796 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:25.216064 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:25.216028 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-s8j5w" event={"ID":"4ec55c86-a659-42c3-bbe4-72e4df40f7f7","Type":"ContainerStarted","Data":"9454a76a6e9a2610af66205d247c9a9264bc8b769bee0150e207ab2cc4e0f6ea"} Jul 22 03:08:25.235390 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:25.235342 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-s8j5w" podStartSLOduration=1.9694159199999999 podStartE2EDuration="4.235328753s" totalImagesPullingTime="2.265912833s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:08:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:08:25.234199524 +0000 UTC m=+45.683182773" watchObservedRunningTime="2026-07-22 03:08:25.235328753 +0000 UTC m=+45.684312002" Jul 22 03:08:25.747551 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:25.747490 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:25.916229 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:25.916201 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:26.007776 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:26.007699 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:26.120525 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:26.120487 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:26.195532 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:26.195500 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:27.040801 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:27.040765 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:27.242089 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:27.242052 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:27.641055 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:27.641019 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:27.838907 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:27.838871 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:28.258934 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:28.258901 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:28.645933 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:28.645902 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:29.287984 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:29.287938 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:29.435372 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:29.435339 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:29.588033 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:29.588005 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:29.834679 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:29.834647 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:30.041612 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:30.041531 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:30.239946 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:30.239908 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:30.307402 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:30.307328 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:30.646470 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:30.646434 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:30.854943 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:30.854913 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0dbb2e21-4aa1-43df-8917-adc7034719b0-registry-tls\") pod \"image-registry-7fcbd97f78-7hd7h\" (UID: \"0dbb2e21-4aa1-43df-8917-adc7034719b0\") " pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" Jul 22 03:08:30.855133 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:30.855007 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5dc9285-733c-4cf4-abad-90ec9da9674d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zs5nq\" (UID: \"c5dc9285-733c-4cf4-abad-90ec9da9674d\") " pod="openshift-insights/insights-runtime-extractor-zs5nq" Jul 22 03:08:30.855133 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:30.855063 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5dc9285-733c-4cf4-abad-90ec9da9674d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zs5nq\" (UID: \"c5dc9285-733c-4cf4-abad-90ec9da9674d\") " pod="openshift-insights/insights-runtime-extractor-zs5nq" Jul 22 03:08:30.855487 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:30.855464 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5dc9285-733c-4cf4-abad-90ec9da9674d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zs5nq\" (UID: \"c5dc9285-733c-4cf4-abad-90ec9da9674d\") " pod="openshift-insights/insights-runtime-extractor-zs5nq" Jul 22 03:08:30.857259 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:30.857229 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5dc9285-733c-4cf4-abad-90ec9da9674d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zs5nq\" (UID: \"c5dc9285-733c-4cf4-abad-90ec9da9674d\") " pod="openshift-insights/insights-runtime-extractor-zs5nq" Jul 22 03:08:30.857259 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:30.857257 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0dbb2e21-4aa1-43df-8917-adc7034719b0-registry-tls\") pod \"image-registry-7fcbd97f78-7hd7h\" (UID: \"0dbb2e21-4aa1-43df-8917-adc7034719b0\") " pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" Jul 22 03:08:30.889240 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:30.889217 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" Jul 22 03:08:30.906427 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:30.906375 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zs5nq" Jul 22 03:08:30.955947 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:30.955915 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/01af83ea-a187-4abb-ba83-7d45027fb098-metrics-tls\") pod \"dns-default-d8rr4\" (UID: \"01af83ea-a187-4abb-ba83-7d45027fb098\") " pod="openshift-dns/dns-default-d8rr4" Jul 22 03:08:30.958581 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:30.958462 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/01af83ea-a187-4abb-ba83-7d45027fb098-metrics-tls\") pod \"dns-default-d8rr4\" (UID: \"01af83ea-a187-4abb-ba83-7d45027fb098\") " pod="openshift-dns/dns-default-d8rr4" Jul 22 03:08:30.995567 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:30.995534 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-d8rr4" Jul 22 03:08:31.060036 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:31.059993 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7fcbd97f78-7hd7h"] Jul 22 03:08:31.063622 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:08:31.063598 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dbb2e21_4aa1_43df_8917_adc7034719b0.slice/crio-3d3fdbdf0ae75d9e5a06af3b1fab56edf85fd41c7112d269cb566e7605c1b7d2 WatchSource:0}: Error finding container 3d3fdbdf0ae75d9e5a06af3b1fab56edf85fd41c7112d269cb566e7605c1b7d2: Status 404 returned error can't find the container with id 3d3fdbdf0ae75d9e5a06af3b1fab56edf85fd41c7112d269cb566e7605c1b7d2 Jul 22 03:08:31.067347 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:31.067326 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zs5nq"] Jul 22 03:08:31.071454 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:08:31.071427 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5dc9285_733c_4cf4_abad_90ec9da9674d.slice/crio-19d148113ade361dc4b86311383ed4ccdc2f30775389dd5d38a5af851cce38d1 WatchSource:0}: Error finding container 19d148113ade361dc4b86311383ed4ccdc2f30775389dd5d38a5af851cce38d1: Status 404 returned error can't find the container with id 19d148113ade361dc4b86311383ed4ccdc2f30775389dd5d38a5af851cce38d1 Jul 22 03:08:31.155564 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:31.155539 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-d8rr4"] Jul 22 03:08:31.158472 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:08:31.158450 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01af83ea_a187_4abb_ba83_7d45027fb098.slice/crio-0c1df04a01b3939e71df049ee207b6ae62ae1424527875430990e59ae9ff54a3 WatchSource:0}: Error finding container 0c1df04a01b3939e71df049ee207b6ae62ae1424527875430990e59ae9ff54a3: Status 404 returned error can't find the container with id 0c1df04a01b3939e71df049ee207b6ae62ae1424527875430990e59ae9ff54a3 Jul 22 03:08:31.226196 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:31.226164 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:31.231879 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:31.231851 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zs5nq" event={"ID":"c5dc9285-733c-4cf4-abad-90ec9da9674d","Type":"ContainerStarted","Data":"19d148113ade361dc4b86311383ed4ccdc2f30775389dd5d38a5af851cce38d1"} Jul 22 03:08:31.233364 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:31.233343 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" event={"ID":"0dbb2e21-4aa1-43df-8917-adc7034719b0","Type":"ContainerStarted","Data":"5d7ade5f0188a34d1f762c644d3541a13e88fbf27fc803040bd47449401ec18a"} Jul 22 03:08:31.233450 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:31.233378 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" event={"ID":"0dbb2e21-4aa1-43df-8917-adc7034719b0","Type":"ContainerStarted","Data":"3d3fdbdf0ae75d9e5a06af3b1fab56edf85fd41c7112d269cb566e7605c1b7d2"} Jul 22 03:08:31.233450 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:31.233432 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" Jul 22 03:08:31.234452 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:31.234435 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-d8rr4" event={"ID":"01af83ea-a187-4abb-ba83-7d45027fb098","Type":"ContainerStarted","Data":"0c1df04a01b3939e71df049ee207b6ae62ae1424527875430990e59ae9ff54a3"} Jul 22 03:08:31.258766 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:31.258725 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" podStartSLOduration=24.258712329 podStartE2EDuration="24.258712329s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:08:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:08:31.258495247 +0000 UTC m=+51.707478495" watchObservedRunningTime="2026-07-22 03:08:31.258712329 +0000 UTC m=+51.707695577" Jul 22 03:08:31.301872 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:31.301843 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:31.636132 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:31.636099 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:32.052728 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:32.052646 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:32.121360 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:32.121326 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:32.482840 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:32.482800 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:32.657778 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:32.657611 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:33.047969 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:33.047326 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:33.231683 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:33.231154 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:34.258922 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:34.258890 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:34.439540 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:34.439503 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:34.890782 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:34.890747 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:34.956227 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:34.956165 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:35.040153 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:35.040097 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:35.441042 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:35.441000 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:35.846578 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:35.846544 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:36.248280 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:36.248198 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:36.506317 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:36.506220 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:37.255393 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:37.255360 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-d8rr4" event={"ID":"01af83ea-a187-4abb-ba83-7d45027fb098","Type":"ContainerStarted","Data":"0641d33b4e59a708f5a4c8a0c9770c6a730eda58a48cc7d4c5a3e47fa01877b8"} Jul 22 03:08:37.255607 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:37.255421 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-d8rr4" event={"ID":"01af83ea-a187-4abb-ba83-7d45027fb098","Type":"ContainerStarted","Data":"ecedd7a84f2159c885aeaefe847f8e575345e0a0510ef0ff85310dc7ee76d1af"} Jul 22 03:08:37.255607 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:37.255469 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-d8rr4" Jul 22 03:08:37.257112 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:37.257085 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zs5nq" event={"ID":"c5dc9285-733c-4cf4-abad-90ec9da9674d","Type":"ContainerStarted","Data":"8ac28c0b2d8afd1be4d8c5977c67184fa28a1ce578393e02c66922da4293019e"} Jul 22 03:08:37.275434 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:37.275387 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-d8rr4" podStartSLOduration=16.838904881 podStartE2EDuration="22.275370198s" totalImagesPullingTime="5.436465317s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:08:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:08:37.274473594 +0000 UTC m=+57.723456848" watchObservedRunningTime="2026-07-22 03:08:37.275370198 +0000 UTC m=+57.724353444" Jul 22 03:08:40.270099 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:40.270019 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zs5nq" event={"ID":"c5dc9285-733c-4cf4-abad-90ec9da9674d","Type":"ContainerStarted","Data":"ee3d589883b06f74fe24772ee0d706077ea7d8659edc0787711e886b29b9bb36"} Jul 22 03:08:41.177368 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:41.177336 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sscf8" Jul 22 03:08:42.557133 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:42.557094 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:42.654688 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:42.654653 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:42.759429 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:42.759389 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:42.964272 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:42.964223 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-77c596ccbb-jh2b7"] Jul 22 03:08:42.974273 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:42.974238 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77c596ccbb-jh2b7" Jul 22 03:08:42.976858 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:42.976827 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 22 03:08:42.977293 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:42.977269 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 22 03:08:42.978212 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:42.978179 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 22 03:08:42.978452 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:42.978418 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 22 03:08:42.986736 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:42.986710 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-77c596ccbb-jh2b7"] Jul 22 03:08:43.000033 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.000006 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7fcbd97f78-7hd7h"] Jul 22 03:08:43.036017 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.035981 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ktf9k"] Jul 22 03:08:43.039288 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.039262 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ktf9k" Jul 22 03:08:43.042756 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.042731 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 22 03:08:43.044344 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.044317 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-mm92k"] Jul 22 03:08:43.046569 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.046549 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-mm92k" Jul 22 03:08:43.046926 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.046820 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-l2n9t\"" Jul 22 03:08:43.049472 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.049442 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 22 03:08:43.049566 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.049472 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-2c5wp\"" Jul 22 03:08:43.049639 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.049573 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 22 03:08:43.056399 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.056345 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ktf9k"] Jul 22 03:08:43.057362 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.057340 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-mm92k"] Jul 22 03:08:43.058020 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.057982 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/3c2bd6da-59b8-4e46-b4ca-e6b75c9a3ceb-klusterlet-config\") pod \"klusterlet-addon-workmgr-77c596ccbb-jh2b7\" (UID: \"3c2bd6da-59b8-4e46-b4ca-e6b75c9a3ceb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77c596ccbb-jh2b7" Jul 22 03:08:43.058125 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.058030 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c97kz\" (UniqueName: \"kubernetes.io/projected/3c2bd6da-59b8-4e46-b4ca-e6b75c9a3ceb-kube-api-access-c97kz\") pod \"klusterlet-addon-workmgr-77c596ccbb-jh2b7\" (UID: \"3c2bd6da-59b8-4e46-b4ca-e6b75c9a3ceb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77c596ccbb-jh2b7" Jul 22 03:08:43.058125 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.058081 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3c2bd6da-59b8-4e46-b4ca-e6b75c9a3ceb-tmp\") pod \"klusterlet-addon-workmgr-77c596ccbb-jh2b7\" (UID: \"3c2bd6da-59b8-4e46-b4ca-e6b75c9a3ceb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77c596ccbb-jh2b7" Jul 22 03:08:43.060912 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.060878 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:43.148060 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.148028 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-8458747fff-b6gnb"] Jul 22 03:08:43.152108 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.152081 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8458747fff-b6gnb" Jul 22 03:08:43.158414 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.158387 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3c2bd6da-59b8-4e46-b4ca-e6b75c9a3ceb-tmp\") pod \"klusterlet-addon-workmgr-77c596ccbb-jh2b7\" (UID: \"3c2bd6da-59b8-4e46-b4ca-e6b75c9a3ceb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77c596ccbb-jh2b7" Jul 22 03:08:43.158557 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.158429 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/faa204cb-ed7e-4afe-9115-af504e1c48b6-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-mm92k\" (UID: \"faa204cb-ed7e-4afe-9115-af504e1c48b6\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-mm92k" Jul 22 03:08:43.158557 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.158478 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/faa204cb-ed7e-4afe-9115-af504e1c48b6-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-mm92k\" (UID: \"faa204cb-ed7e-4afe-9115-af504e1c48b6\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-mm92k" Jul 22 03:08:43.158557 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.158507 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/3c2bd6da-59b8-4e46-b4ca-e6b75c9a3ceb-klusterlet-config\") pod \"klusterlet-addon-workmgr-77c596ccbb-jh2b7\" (UID: \"3c2bd6da-59b8-4e46-b4ca-e6b75c9a3ceb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77c596ccbb-jh2b7" Jul 22 03:08:43.158557 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.158553 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7b4c464a-27f8-4ed7-bf38-1ef06032ac5a-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-ktf9k\" (UID: \"7b4c464a-27f8-4ed7-bf38-1ef06032ac5a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ktf9k" Jul 22 03:08:43.158940 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.158628 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c97kz\" (UniqueName: \"kubernetes.io/projected/3c2bd6da-59b8-4e46-b4ca-e6b75c9a3ceb-kube-api-access-c97kz\") pod \"klusterlet-addon-workmgr-77c596ccbb-jh2b7\" (UID: \"3c2bd6da-59b8-4e46-b4ca-e6b75c9a3ceb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77c596ccbb-jh2b7" Jul 22 03:08:43.158940 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.158876 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3c2bd6da-59b8-4e46-b4ca-e6b75c9a3ceb-tmp\") pod \"klusterlet-addon-workmgr-77c596ccbb-jh2b7\" (UID: \"3c2bd6da-59b8-4e46-b4ca-e6b75c9a3ceb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77c596ccbb-jh2b7" Jul 22 03:08:43.161945 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.161922 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/3c2bd6da-59b8-4e46-b4ca-e6b75c9a3ceb-klusterlet-config\") pod \"klusterlet-addon-workmgr-77c596ccbb-jh2b7\" (UID: \"3c2bd6da-59b8-4e46-b4ca-e6b75c9a3ceb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77c596ccbb-jh2b7" Jul 22 03:08:43.180734 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.180665 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8458747fff-b6gnb"] Jul 22 03:08:43.182176 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.182150 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c97kz\" (UniqueName: \"kubernetes.io/projected/3c2bd6da-59b8-4e46-b4ca-e6b75c9a3ceb-kube-api-access-c97kz\") pod \"klusterlet-addon-workmgr-77c596ccbb-jh2b7\" (UID: \"3c2bd6da-59b8-4e46-b4ca-e6b75c9a3ceb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77c596ccbb-jh2b7" Jul 22 03:08:43.216727 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.216646 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:43.259133 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.259081 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/faa204cb-ed7e-4afe-9115-af504e1c48b6-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-mm92k\" (UID: \"faa204cb-ed7e-4afe-9115-af504e1c48b6\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-mm92k" Jul 22 03:08:43.259313 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.259145 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b402f8e9-6401-4160-a5c2-baa9dfbc7cd8-installation-pull-secrets\") pod \"image-registry-8458747fff-b6gnb\" (UID: \"b402f8e9-6401-4160-a5c2-baa9dfbc7cd8\") " pod="openshift-image-registry/image-registry-8458747fff-b6gnb" Jul 22 03:08:43.259313 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.259205 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7b4c464a-27f8-4ed7-bf38-1ef06032ac5a-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-ktf9k\" (UID: \"7b4c464a-27f8-4ed7-bf38-1ef06032ac5a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ktf9k" Jul 22 03:08:43.259313 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.259229 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b402f8e9-6401-4160-a5c2-baa9dfbc7cd8-bound-sa-token\") pod \"image-registry-8458747fff-b6gnb\" (UID: \"b402f8e9-6401-4160-a5c2-baa9dfbc7cd8\") " pod="openshift-image-registry/image-registry-8458747fff-b6gnb" Jul 22 03:08:43.259313 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.259246 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b402f8e9-6401-4160-a5c2-baa9dfbc7cd8-image-registry-private-configuration\") pod \"image-registry-8458747fff-b6gnb\" (UID: \"b402f8e9-6401-4160-a5c2-baa9dfbc7cd8\") " pod="openshift-image-registry/image-registry-8458747fff-b6gnb" Jul 22 03:08:43.259313 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.259265 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b402f8e9-6401-4160-a5c2-baa9dfbc7cd8-trusted-ca\") pod \"image-registry-8458747fff-b6gnb\" (UID: \"b402f8e9-6401-4160-a5c2-baa9dfbc7cd8\") " pod="openshift-image-registry/image-registry-8458747fff-b6gnb" Jul 22 03:08:43.259535 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.259336 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b402f8e9-6401-4160-a5c2-baa9dfbc7cd8-registry-tls\") pod \"image-registry-8458747fff-b6gnb\" (UID: \"b402f8e9-6401-4160-a5c2-baa9dfbc7cd8\") " pod="openshift-image-registry/image-registry-8458747fff-b6gnb" Jul 22 03:08:43.259535 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.259374 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7x9l\" (UniqueName: \"kubernetes.io/projected/b402f8e9-6401-4160-a5c2-baa9dfbc7cd8-kube-api-access-f7x9l\") pod \"image-registry-8458747fff-b6gnb\" (UID: \"b402f8e9-6401-4160-a5c2-baa9dfbc7cd8\") " pod="openshift-image-registry/image-registry-8458747fff-b6gnb" Jul 22 03:08:43.259535 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.259424 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b402f8e9-6401-4160-a5c2-baa9dfbc7cd8-ca-trust-extracted\") pod \"image-registry-8458747fff-b6gnb\" (UID: \"b402f8e9-6401-4160-a5c2-baa9dfbc7cd8\") " pod="openshift-image-registry/image-registry-8458747fff-b6gnb" Jul 22 03:08:43.259535 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.259455 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/faa204cb-ed7e-4afe-9115-af504e1c48b6-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-mm92k\" (UID: \"faa204cb-ed7e-4afe-9115-af504e1c48b6\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-mm92k" Jul 22 03:08:43.259535 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.259485 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b402f8e9-6401-4160-a5c2-baa9dfbc7cd8-registry-certificates\") pod \"image-registry-8458747fff-b6gnb\" (UID: \"b402f8e9-6401-4160-a5c2-baa9dfbc7cd8\") " pod="openshift-image-registry/image-registry-8458747fff-b6gnb" Jul 22 03:08:43.259923 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.259897 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/faa204cb-ed7e-4afe-9115-af504e1c48b6-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-mm92k\" (UID: \"faa204cb-ed7e-4afe-9115-af504e1c48b6\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-mm92k" Jul 22 03:08:43.261890 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.261868 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7b4c464a-27f8-4ed7-bf38-1ef06032ac5a-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-ktf9k\" (UID: \"7b4c464a-27f8-4ed7-bf38-1ef06032ac5a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ktf9k" Jul 22 03:08:43.262028 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.261978 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/faa204cb-ed7e-4afe-9115-af504e1c48b6-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-mm92k\" (UID: \"faa204cb-ed7e-4afe-9115-af504e1c48b6\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-mm92k" Jul 22 03:08:43.287134 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.287107 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77c596ccbb-jh2b7" Jul 22 03:08:43.312583 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.312547 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:43.351304 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.351272 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ktf9k" Jul 22 03:08:43.360145 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.360114 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b402f8e9-6401-4160-a5c2-baa9dfbc7cd8-installation-pull-secrets\") pod \"image-registry-8458747fff-b6gnb\" (UID: \"b402f8e9-6401-4160-a5c2-baa9dfbc7cd8\") " pod="openshift-image-registry/image-registry-8458747fff-b6gnb" Jul 22 03:08:43.360254 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.360199 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-mm92k" Jul 22 03:08:43.360309 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.360279 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b402f8e9-6401-4160-a5c2-baa9dfbc7cd8-bound-sa-token\") pod \"image-registry-8458747fff-b6gnb\" (UID: \"b402f8e9-6401-4160-a5c2-baa9dfbc7cd8\") " pod="openshift-image-registry/image-registry-8458747fff-b6gnb" Jul 22 03:08:43.360372 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.360321 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b402f8e9-6401-4160-a5c2-baa9dfbc7cd8-image-registry-private-configuration\") pod \"image-registry-8458747fff-b6gnb\" (UID: \"b402f8e9-6401-4160-a5c2-baa9dfbc7cd8\") " pod="openshift-image-registry/image-registry-8458747fff-b6gnb" Jul 22 03:08:43.360372 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.360351 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b402f8e9-6401-4160-a5c2-baa9dfbc7cd8-trusted-ca\") pod \"image-registry-8458747fff-b6gnb\" (UID: \"b402f8e9-6401-4160-a5c2-baa9dfbc7cd8\") " pod="openshift-image-registry/image-registry-8458747fff-b6gnb" Jul 22 03:08:43.360477 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.360400 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b402f8e9-6401-4160-a5c2-baa9dfbc7cd8-registry-tls\") pod \"image-registry-8458747fff-b6gnb\" (UID: \"b402f8e9-6401-4160-a5c2-baa9dfbc7cd8\") " pod="openshift-image-registry/image-registry-8458747fff-b6gnb" Jul 22 03:08:43.360477 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.360439 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7x9l\" (UniqueName: \"kubernetes.io/projected/b402f8e9-6401-4160-a5c2-baa9dfbc7cd8-kube-api-access-f7x9l\") pod \"image-registry-8458747fff-b6gnb\" (UID: \"b402f8e9-6401-4160-a5c2-baa9dfbc7cd8\") " pod="openshift-image-registry/image-registry-8458747fff-b6gnb" Jul 22 03:08:43.360566 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.360472 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b402f8e9-6401-4160-a5c2-baa9dfbc7cd8-ca-trust-extracted\") pod \"image-registry-8458747fff-b6gnb\" (UID: \"b402f8e9-6401-4160-a5c2-baa9dfbc7cd8\") " pod="openshift-image-registry/image-registry-8458747fff-b6gnb" Jul 22 03:08:43.360566 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.360507 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b402f8e9-6401-4160-a5c2-baa9dfbc7cd8-registry-certificates\") pod \"image-registry-8458747fff-b6gnb\" (UID: \"b402f8e9-6401-4160-a5c2-baa9dfbc7cd8\") " pod="openshift-image-registry/image-registry-8458747fff-b6gnb" Jul 22 03:08:43.361281 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.361224 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b402f8e9-6401-4160-a5c2-baa9dfbc7cd8-ca-trust-extracted\") pod \"image-registry-8458747fff-b6gnb\" (UID: \"b402f8e9-6401-4160-a5c2-baa9dfbc7cd8\") " pod="openshift-image-registry/image-registry-8458747fff-b6gnb" Jul 22 03:08:43.361404 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.361371 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b402f8e9-6401-4160-a5c2-baa9dfbc7cd8-registry-certificates\") pod \"image-registry-8458747fff-b6gnb\" (UID: \"b402f8e9-6401-4160-a5c2-baa9dfbc7cd8\") " pod="openshift-image-registry/image-registry-8458747fff-b6gnb" Jul 22 03:08:43.361729 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.361707 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b402f8e9-6401-4160-a5c2-baa9dfbc7cd8-trusted-ca\") pod \"image-registry-8458747fff-b6gnb\" (UID: \"b402f8e9-6401-4160-a5c2-baa9dfbc7cd8\") " pod="openshift-image-registry/image-registry-8458747fff-b6gnb" Jul 22 03:08:43.363191 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.363172 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b402f8e9-6401-4160-a5c2-baa9dfbc7cd8-installation-pull-secrets\") pod \"image-registry-8458747fff-b6gnb\" (UID: \"b402f8e9-6401-4160-a5c2-baa9dfbc7cd8\") " pod="openshift-image-registry/image-registry-8458747fff-b6gnb" Jul 22 03:08:43.363514 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.363490 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b402f8e9-6401-4160-a5c2-baa9dfbc7cd8-registry-tls\") pod \"image-registry-8458747fff-b6gnb\" (UID: \"b402f8e9-6401-4160-a5c2-baa9dfbc7cd8\") " pod="openshift-image-registry/image-registry-8458747fff-b6gnb" Jul 22 03:08:43.363659 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.363640 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b402f8e9-6401-4160-a5c2-baa9dfbc7cd8-image-registry-private-configuration\") pod \"image-registry-8458747fff-b6gnb\" (UID: \"b402f8e9-6401-4160-a5c2-baa9dfbc7cd8\") " pod="openshift-image-registry/image-registry-8458747fff-b6gnb" Jul 22 03:08:43.371942 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.371923 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b402f8e9-6401-4160-a5c2-baa9dfbc7cd8-bound-sa-token\") pod \"image-registry-8458747fff-b6gnb\" (UID: \"b402f8e9-6401-4160-a5c2-baa9dfbc7cd8\") " pod="openshift-image-registry/image-registry-8458747fff-b6gnb" Jul 22 03:08:43.372621 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.372603 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7x9l\" (UniqueName: \"kubernetes.io/projected/b402f8e9-6401-4160-a5c2-baa9dfbc7cd8-kube-api-access-f7x9l\") pod \"image-registry-8458747fff-b6gnb\" (UID: \"b402f8e9-6401-4160-a5c2-baa9dfbc7cd8\") " pod="openshift-image-registry/image-registry-8458747fff-b6gnb" Jul 22 03:08:43.462014 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.461979 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8458747fff-b6gnb" Jul 22 03:08:43.871799 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:43.871766 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8458747fff-b6gnb"] Jul 22 03:08:43.877307 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:08:43.877285 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb402f8e9_6401_4160_a5c2_baa9dfbc7cd8.slice/crio-540dabdbc4815de2fd65999a935394ce8e9e3d19dd2b79ca15da3c42fa253318 WatchSource:0}: Error finding container 540dabdbc4815de2fd65999a935394ce8e9e3d19dd2b79ca15da3c42fa253318: Status 404 returned error can't find the container with id 540dabdbc4815de2fd65999a935394ce8e9e3d19dd2b79ca15da3c42fa253318 Jul 22 03:08:44.083825 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:44.083795 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-77c596ccbb-jh2b7"] Jul 22 03:08:44.086633 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:08:44.086597 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c2bd6da_59b8_4e46_b4ca_e6b75c9a3ceb.slice/crio-b8c39dcc2f7316a42a43314efb36a1ee11315aa3e5b2d14f31df00e354dfcc88 WatchSource:0}: Error finding container b8c39dcc2f7316a42a43314efb36a1ee11315aa3e5b2d14f31df00e354dfcc88: Status 404 returned error can't find the container with id b8c39dcc2f7316a42a43314efb36a1ee11315aa3e5b2d14f31df00e354dfcc88 Jul 22 03:08:44.095736 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:44.095646 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ktf9k"] Jul 22 03:08:44.096845 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:44.096822 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-mm92k"] Jul 22 03:08:44.097490 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:08:44.097466 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b4c464a_27f8_4ed7_bf38_1ef06032ac5a.slice/crio-71bb0d557ef8090669afed6199d17e12ecb3e17c6c2c40a08549e250a51f5ee0 WatchSource:0}: Error finding container 71bb0d557ef8090669afed6199d17e12ecb3e17c6c2c40a08549e250a51f5ee0: Status 404 returned error can't find the container with id 71bb0d557ef8090669afed6199d17e12ecb3e17c6c2c40a08549e250a51f5ee0 Jul 22 03:08:44.098129 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:08:44.098107 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaa204cb_ed7e_4afe_9115_af504e1c48b6.slice/crio-d1e134ea445372b2d0e37bd57db38825e3f4d781bd2a7da899b7a73c818665e5 WatchSource:0}: Error finding container d1e134ea445372b2d0e37bd57db38825e3f4d781bd2a7da899b7a73c818665e5: Status 404 returned error can't find the container with id d1e134ea445372b2d0e37bd57db38825e3f4d781bd2a7da899b7a73c818665e5 Jul 22 03:08:44.285168 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:44.285132 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8458747fff-b6gnb" event={"ID":"b402f8e9-6401-4160-a5c2-baa9dfbc7cd8","Type":"ContainerStarted","Data":"edea72e476412cde0f2e61577ce1579ca9efaa183f050281f94a6d344400dea6"} Jul 22 03:08:44.285337 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:44.285184 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8458747fff-b6gnb" event={"ID":"b402f8e9-6401-4160-a5c2-baa9dfbc7cd8","Type":"ContainerStarted","Data":"540dabdbc4815de2fd65999a935394ce8e9e3d19dd2b79ca15da3c42fa253318"} Jul 22 03:08:44.285337 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:44.285232 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-8458747fff-b6gnb" Jul 22 03:08:44.287167 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:44.287143 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zs5nq" event={"ID":"c5dc9285-733c-4cf4-abad-90ec9da9674d","Type":"ContainerStarted","Data":"10dce2d7e46d51e86bb21317d762ab30512fd4c4351937d2475b028ce5e5afe6"} Jul 22 03:08:44.288343 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:44.288315 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ktf9k" event={"ID":"7b4c464a-27f8-4ed7-bf38-1ef06032ac5a","Type":"ContainerStarted","Data":"71bb0d557ef8090669afed6199d17e12ecb3e17c6c2c40a08549e250a51f5ee0"} Jul 22 03:08:44.289325 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:44.289309 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77c596ccbb-jh2b7" event={"ID":"3c2bd6da-59b8-4e46-b4ca-e6b75c9a3ceb","Type":"ContainerStarted","Data":"b8c39dcc2f7316a42a43314efb36a1ee11315aa3e5b2d14f31df00e354dfcc88"} Jul 22 03:08:44.290239 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:44.290222 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-mm92k" event={"ID":"faa204cb-ed7e-4afe-9115-af504e1c48b6","Type":"ContainerStarted","Data":"d1e134ea445372b2d0e37bd57db38825e3f4d781bd2a7da899b7a73c818665e5"} Jul 22 03:08:44.308768 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:44.308729 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-8458747fff-b6gnb" podStartSLOduration=1.308715865 podStartE2EDuration="1.308715865s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:08:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:08:44.307537701 +0000 UTC m=+64.756520950" watchObservedRunningTime="2026-07-22 03:08:44.308715865 +0000 UTC m=+64.757699140" Jul 22 03:08:44.340577 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:44.340492 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zs5nq" podStartSLOduration=17.759650421 podStartE2EDuration="30.340478512s" totalImagesPullingTime="12.580828091s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:08:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:08:44.33717028 +0000 UTC m=+64.786153532" watchObservedRunningTime="2026-07-22 03:08:44.340478512 +0000 UTC m=+64.789461760" Jul 22 03:08:45.686167 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:45.685918 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1297a647-da14-4c7b-bf94-6f4f4e679c31-original-pull-secret\") pod \"global-pull-secret-syncer-qhg45\" (UID: \"1297a647-da14-4c7b-bf94-6f4f4e679c31\") " pod="kube-system/global-pull-secret-syncer-qhg45" Jul 22 03:08:45.686167 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:45.685985 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb8b43a6-b50f-41e9-9e68-763231228640-metrics-certs\") pod \"network-metrics-daemon-xrbdq\" (UID: \"bb8b43a6-b50f-41e9-9e68-763231228640\") " pod="openshift-multus/network-metrics-daemon-xrbdq" Jul 22 03:08:45.689052 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:45.688846 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 03:08:45.689052 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:45.688981 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 03:08:45.700165 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:45.700109 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb8b43a6-b50f-41e9-9e68-763231228640-metrics-certs\") pod \"network-metrics-daemon-xrbdq\" (UID: \"bb8b43a6-b50f-41e9-9e68-763231228640\") " pod="openshift-multus/network-metrics-daemon-xrbdq" Jul 22 03:08:45.701496 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:45.701469 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1297a647-da14-4c7b-bf94-6f4f4e679c31-original-pull-secret\") pod \"global-pull-secret-syncer-qhg45\" (UID: \"1297a647-da14-4c7b-bf94-6f4f4e679c31\") " pod="kube-system/global-pull-secret-syncer-qhg45" Jul 22 03:08:45.786840 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:45.786808 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x25qb\" (UniqueName: \"kubernetes.io/projected/1c2c0875-8ece-494b-95a1-00fe315db1e4-kube-api-access-x25qb\") pod \"network-check-target-k4rxt\" (UID: \"1c2c0875-8ece-494b-95a1-00fe315db1e4\") " pod="openshift-network-diagnostics/network-check-target-k4rxt" Jul 22 03:08:45.789809 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:45.789740 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 22 03:08:45.801496 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:45.801470 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 22 03:08:45.810099 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:45.810078 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x25qb\" (UniqueName: \"kubernetes.io/projected/1c2c0875-8ece-494b-95a1-00fe315db1e4-kube-api-access-x25qb\") pod \"network-check-target-k4rxt\" (UID: \"1c2c0875-8ece-494b-95a1-00fe315db1e4\") " pod="openshift-network-diagnostics/network-check-target-k4rxt" Jul 22 03:08:45.939384 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:45.939301 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qhg45" Jul 22 03:08:45.945279 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:45.945255 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-glt2k\"" Jul 22 03:08:45.945885 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:45.945870 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4gddc\"" Jul 22 03:08:45.952702 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:45.952685 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4rxt" Jul 22 03:08:45.953849 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:45.953833 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrbdq" Jul 22 03:08:46.247567 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:46.247457 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:46.341770 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:46.341734 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:47.264328 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:47.264300 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-d8rr4" Jul 22 03:08:48.412101 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:48.412078 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xrbdq"] Jul 22 03:08:48.414323 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:08:48.414285 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb8b43a6_b50f_41e9_9e68_763231228640.slice/crio-2a32ddf9f3bcaa087973a562f04241645f14bed9aeaff775b0ea58d47eea000a WatchSource:0}: Error finding container 2a32ddf9f3bcaa087973a562f04241645f14bed9aeaff775b0ea58d47eea000a: Status 404 returned error can't find the container with id 2a32ddf9f3bcaa087973a562f04241645f14bed9aeaff775b0ea58d47eea000a Jul 22 03:08:48.637259 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:48.637228 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-k4rxt"] Jul 22 03:08:48.640213 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:08:48.640178 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c2c0875_8ece_494b_95a1_00fe315db1e4.slice/crio-ed3bea82c4b4c9e639fdab4bde5a3a7f941f68711813b800173ff10ba3beb599 WatchSource:0}: Error finding container ed3bea82c4b4c9e639fdab4bde5a3a7f941f68711813b800173ff10ba3beb599: Status 404 returned error can't find the container with id ed3bea82c4b4c9e639fdab4bde5a3a7f941f68711813b800173ff10ba3beb599 Jul 22 03:08:48.640725 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:48.640706 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qhg45"] Jul 22 03:08:48.642820 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:08:48.642785 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1297a647_da14_4c7b_bf94_6f4f4e679c31.slice/crio-f7c64a84011217df50420be4fd68c538c394de78f42d6291c55e50971f02507d WatchSource:0}: Error finding container f7c64a84011217df50420be4fd68c538c394de78f42d6291c55e50971f02507d: Status 404 returned error can't find the container with id f7c64a84011217df50420be4fd68c538c394de78f42d6291c55e50971f02507d Jul 22 03:08:49.317722 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:49.317274 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77c596ccbb-jh2b7" event={"ID":"3c2bd6da-59b8-4e46-b4ca-e6b75c9a3ceb","Type":"ContainerStarted","Data":"2fe29fcc8b437921e826772aaf43549361d20f3ebed13aec481cd689f6311ad3"} Jul 22 03:08:49.317722 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:49.317540 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77c596ccbb-jh2b7" Jul 22 03:08:49.319546 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:49.319516 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77c596ccbb-jh2b7" Jul 22 03:08:49.319732 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:49.319706 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xrbdq" event={"ID":"bb8b43a6-b50f-41e9-9e68-763231228640","Type":"ContainerStarted","Data":"2a32ddf9f3bcaa087973a562f04241645f14bed9aeaff775b0ea58d47eea000a"} Jul 22 03:08:49.322695 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:49.322660 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-mm92k" event={"ID":"faa204cb-ed7e-4afe-9115-af504e1c48b6","Type":"ContainerStarted","Data":"28d37dfc842997f692e00d03f715f0b6ab7533335e176be69bc11e1d2c635cb5"} Jul 22 03:08:49.324107 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:49.324048 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qhg45" event={"ID":"1297a647-da14-4c7b-bf94-6f4f4e679c31","Type":"ContainerStarted","Data":"f7c64a84011217df50420be4fd68c538c394de78f42d6291c55e50971f02507d"} Jul 22 03:08:49.325415 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:49.325394 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-k4rxt" event={"ID":"1c2c0875-8ece-494b-95a1-00fe315db1e4","Type":"ContainerStarted","Data":"ed3bea82c4b4c9e639fdab4bde5a3a7f941f68711813b800173ff10ba3beb599"} Jul 22 03:08:49.327621 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:49.327595 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ktf9k" event={"ID":"7b4c464a-27f8-4ed7-bf38-1ef06032ac5a","Type":"ContainerStarted","Data":"5bc660f7619d8cdef0f08afedad2261724b1cc4e076aaca4eb0fb37c736b9533"} Jul 22 03:08:49.328504 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:49.327969 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ktf9k" Jul 22 03:08:49.335642 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:49.335614 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ktf9k" Jul 22 03:08:49.336505 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:49.336429 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77c596ccbb-jh2b7" podStartSLOduration=2.772238731 podStartE2EDuration="7.336412147s" totalImagesPullingTime="4.564173416s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:08:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:08:49.334315099 +0000 UTC m=+69.783298350" watchObservedRunningTime="2026-07-22 03:08:49.336412147 +0000 UTC m=+69.785395395" Jul 22 03:08:49.398655 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:49.397961 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ktf9k" podStartSLOduration=2.24520806 podStartE2EDuration="6.397924007s" totalImagesPullingTime="4.152715947s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:08:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:08:49.375824439 +0000 UTC m=+69.824807689" watchObservedRunningTime="2026-07-22 03:08:49.397924007 +0000 UTC m=+69.846907258" Jul 22 03:08:49.398655 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:49.398073 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-mm92k" podStartSLOduration=2.246817963 podStartE2EDuration="6.398060405s" totalImagesPullingTime="4.151242442s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:08:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:08:49.395870077 +0000 UTC m=+69.844853327" watchObservedRunningTime="2026-07-22 03:08:49.398060405 +0000 UTC m=+69.847043655" Jul 22 03:08:49.668241 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:49.668076 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:49.803613 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:49.803447 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:49.858410 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:49.858381 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-jf5kv"] Jul 22 03:08:49.861748 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:49.861727 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-jf5kv" Jul 22 03:08:49.865844 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:49.865314 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 22 03:08:49.869286 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:49.869261 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 22 03:08:49.869416 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:49.869398 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-bg82h\"" Jul 22 03:08:49.869631 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:49.869596 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 22 03:08:49.869631 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:49.869616 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 22 03:08:49.869998 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:49.869842 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 22 03:08:49.877886 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:49.877679 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-jf5kv"] Jul 22 03:08:49.919022 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:49.918939 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/465a5cf5-d8c4-46bb-905e-7992138abf82-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-jf5kv\" (UID: \"465a5cf5-d8c4-46bb-905e-7992138abf82\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jf5kv" Jul 22 03:08:49.919022 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:49.919012 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfx4p\" (UniqueName: \"kubernetes.io/projected/465a5cf5-d8c4-46bb-905e-7992138abf82-kube-api-access-dfx4p\") pod \"prometheus-operator-5945b5cfd-jf5kv\" (UID: \"465a5cf5-d8c4-46bb-905e-7992138abf82\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jf5kv" Jul 22 03:08:49.919241 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:49.919059 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/465a5cf5-d8c4-46bb-905e-7992138abf82-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-jf5kv\" (UID: \"465a5cf5-d8c4-46bb-905e-7992138abf82\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jf5kv" Jul 22 03:08:49.919241 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:49.919088 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/465a5cf5-d8c4-46bb-905e-7992138abf82-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-jf5kv\" (UID: \"465a5cf5-d8c4-46bb-905e-7992138abf82\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jf5kv" Jul 22 03:08:49.926730 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:49.926650 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:50.020555 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:50.019478 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/465a5cf5-d8c4-46bb-905e-7992138abf82-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-jf5kv\" (UID: \"465a5cf5-d8c4-46bb-905e-7992138abf82\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jf5kv" Jul 22 03:08:50.020555 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:50.019543 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfx4p\" (UniqueName: \"kubernetes.io/projected/465a5cf5-d8c4-46bb-905e-7992138abf82-kube-api-access-dfx4p\") pod \"prometheus-operator-5945b5cfd-jf5kv\" (UID: \"465a5cf5-d8c4-46bb-905e-7992138abf82\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jf5kv" Jul 22 03:08:50.020555 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:50.019596 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/465a5cf5-d8c4-46bb-905e-7992138abf82-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-jf5kv\" (UID: \"465a5cf5-d8c4-46bb-905e-7992138abf82\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jf5kv" Jul 22 03:08:50.020555 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:50.019630 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/465a5cf5-d8c4-46bb-905e-7992138abf82-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-jf5kv\" (UID: \"465a5cf5-d8c4-46bb-905e-7992138abf82\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jf5kv" Jul 22 03:08:50.020929 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:50.020884 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/465a5cf5-d8c4-46bb-905e-7992138abf82-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-jf5kv\" (UID: \"465a5cf5-d8c4-46bb-905e-7992138abf82\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jf5kv" Jul 22 03:08:50.022645 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:50.022619 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/465a5cf5-d8c4-46bb-905e-7992138abf82-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-jf5kv\" (UID: \"465a5cf5-d8c4-46bb-905e-7992138abf82\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jf5kv" Jul 22 03:08:50.022924 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:50.022901 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/465a5cf5-d8c4-46bb-905e-7992138abf82-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-jf5kv\" (UID: \"465a5cf5-d8c4-46bb-905e-7992138abf82\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jf5kv" Jul 22 03:08:50.031674 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:50.031621 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfx4p\" (UniqueName: \"kubernetes.io/projected/465a5cf5-d8c4-46bb-905e-7992138abf82-kube-api-access-dfx4p\") pod \"prometheus-operator-5945b5cfd-jf5kv\" (UID: \"465a5cf5-d8c4-46bb-905e-7992138abf82\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jf5kv" Jul 22 03:08:50.052108 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:50.052026 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:50.175606 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:50.175524 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-jf5kv" Jul 22 03:08:50.335033 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:50.334997 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xrbdq" event={"ID":"bb8b43a6-b50f-41e9-9e68-763231228640","Type":"ContainerStarted","Data":"8b85f0eea2596a859c38247b56eca0b8924cfffeba565f59547494b375de54a3"} Jul 22 03:08:50.335194 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:50.335085 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xrbdq" event={"ID":"bb8b43a6-b50f-41e9-9e68-763231228640","Type":"ContainerStarted","Data":"71d1feeaef5a418f9653f9c963e2112123ef679b1f3bf8ec137501e8bd41aff5"} Jul 22 03:08:50.337901 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:50.337841 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-jf5kv"] Jul 22 03:08:50.342509 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:08:50.342473 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod465a5cf5_d8c4_46bb_905e_7992138abf82.slice/crio-20ed4c9d25374f6762b63bc7ea8b7ea568ae34c2fdaaa57d8f252c9d1b0437dc WatchSource:0}: Error finding container 20ed4c9d25374f6762b63bc7ea8b7ea568ae34c2fdaaa57d8f252c9d1b0437dc: Status 404 returned error can't find the container with id 20ed4c9d25374f6762b63bc7ea8b7ea568ae34c2fdaaa57d8f252c9d1b0437dc Jul 22 03:08:50.359905 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:50.359080 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xrbdq" podStartSLOduration=69.075605006 podStartE2EDuration="1m10.359059178s" totalImagesPullingTime="1.283454172s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:07:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:08:50.356751147 +0000 UTC m=+70.805734397" watchObservedRunningTime="2026-07-22 03:08:50.359059178 +0000 UTC m=+70.808042431" Jul 22 03:08:50.508242 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:50.508145 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:50.603976 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:50.603930 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:51.340785 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:51.340743 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-jf5kv" event={"ID":"465a5cf5-d8c4-46bb-905e-7992138abf82","Type":"ContainerStarted","Data":"20ed4c9d25374f6762b63bc7ea8b7ea568ae34c2fdaaa57d8f252c9d1b0437dc"} Jul 22 03:08:51.750019 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:51.749928 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:51.863552 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:51.863516 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:53.005898 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:53.005866 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" Jul 22 03:08:53.969858 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:53.969769 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:54.355167 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:54.355132 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-jf5kv" event={"ID":"465a5cf5-d8c4-46bb-905e-7992138abf82","Type":"ContainerStarted","Data":"3a3c174848b649b8217b65d103a94465e4d1593d6ac79ba7e311c667c09f4148"} Jul 22 03:08:54.355596 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:54.355192 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-jf5kv" event={"ID":"465a5cf5-d8c4-46bb-905e-7992138abf82","Type":"ContainerStarted","Data":"34a829afd5a640b1664eff379724258f1188fe8dcd894759d9a3fc835dfffcc0"} Jul 22 03:08:54.356557 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:54.356533 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qhg45" event={"ID":"1297a647-da14-4c7b-bf94-6f4f4e679c31","Type":"ContainerStarted","Data":"6df20b457b9d76292291406b51469058f1e6691cca5e1925cb1daf32454c2f2f"} Jul 22 03:08:54.357887 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:54.357865 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-k4rxt" event={"ID":"1c2c0875-8ece-494b-95a1-00fe315db1e4","Type":"ContainerStarted","Data":"874ab008be1dba7a9347b2566a1f7f4264e0cbdf7a2e829ec3118e0c9f641e9b"} Jul 22 03:08:54.358012 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:54.357987 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-k4rxt" Jul 22 03:08:54.376342 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:54.376291 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-jf5kv" podStartSLOduration=2.053902714 podStartE2EDuration="5.376276175s" totalImagesPullingTime="3.322373461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:08:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:08:54.374086131 +0000 UTC m=+74.823069379" watchObservedRunningTime="2026-07-22 03:08:54.376276175 +0000 UTC m=+74.825259424" Jul 22 03:08:54.396873 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:54.396834 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-k4rxt" podStartSLOduration=69.417033442 podStartE2EDuration="1m14.396820796s" totalImagesPullingTime="4.979787354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:07:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:08:54.395549376 +0000 UTC m=+74.844532636" watchObservedRunningTime="2026-07-22 03:08:54.396820796 +0000 UTC m=+74.845804044" Jul 22 03:08:54.411485 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:54.411445 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-qhg45" podStartSLOduration=69.420770884 podStartE2EDuration="1m14.411431528s" totalImagesPullingTime="4.990660644s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:07:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:08:54.411273453 +0000 UTC m=+74.860256727" watchObservedRunningTime="2026-07-22 03:08:54.411431528 +0000 UTC m=+74.860414767" Jul 22 03:08:55.692674 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:55.692637 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:55.758748 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:55.758718 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:55.982435 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:55.982352 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:56.068874 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.068836 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:56.202867 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.202830 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:56.260496 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.260402 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-blt8n"] Jul 22 03:08:56.266054 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.266028 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-blt8n" Jul 22 03:08:56.269008 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.268992 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 22 03:08:56.269255 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.269238 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-4pp55\"" Jul 22 03:08:56.269312 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.269253 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 22 03:08:56.274185 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.274169 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 22 03:08:56.283915 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.283896 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-gh6jj"] Jul 22 03:08:56.287589 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.287573 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-gh6jj" Jul 22 03:08:56.292930 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.292913 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 22 03:08:56.293572 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.293548 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 22 03:08:56.293888 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.293871 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 22 03:08:56.293990 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.293875 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-xzbb6\"" Jul 22 03:08:56.312464 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.312439 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-gh6jj"] Jul 22 03:08:56.329996 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.329967 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:56.373196 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.373161 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/168e0b81-ebe6-402a-99c6-e7881d11718a-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-gh6jj\" (UID: \"168e0b81-ebe6-402a-99c6-e7881d11718a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gh6jj" Jul 22 03:08:56.373359 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.373208 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlg5x\" (UniqueName: \"kubernetes.io/projected/168e0b81-ebe6-402a-99c6-e7881d11718a-kube-api-access-qlg5x\") pod \"kube-state-metrics-56c5594674-gh6jj\" (UID: \"168e0b81-ebe6-402a-99c6-e7881d11718a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gh6jj" Jul 22 03:08:56.373359 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.373239 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a8f83db4-079b-44ff-a400-0762ba4a1dfd-node-exporter-accelerators-collector-config\") pod \"node-exporter-blt8n\" (UID: \"a8f83db4-079b-44ff-a400-0762ba4a1dfd\") " pod="openshift-monitoring/node-exporter-blt8n" Jul 22 03:08:56.373359 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.373270 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/168e0b81-ebe6-402a-99c6-e7881d11718a-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-gh6jj\" (UID: \"168e0b81-ebe6-402a-99c6-e7881d11718a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gh6jj" Jul 22 03:08:56.373359 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.373321 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a8f83db4-079b-44ff-a400-0762ba4a1dfd-node-exporter-tls\") pod \"node-exporter-blt8n\" (UID: \"a8f83db4-079b-44ff-a400-0762ba4a1dfd\") " pod="openshift-monitoring/node-exporter-blt8n" Jul 22 03:08:56.373587 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.373364 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8f83db4-079b-44ff-a400-0762ba4a1dfd-metrics-client-ca\") pod \"node-exporter-blt8n\" (UID: \"a8f83db4-079b-44ff-a400-0762ba4a1dfd\") " pod="openshift-monitoring/node-exporter-blt8n" Jul 22 03:08:56.373587 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.373394 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/168e0b81-ebe6-402a-99c6-e7881d11718a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-gh6jj\" (UID: \"168e0b81-ebe6-402a-99c6-e7881d11718a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gh6jj" Jul 22 03:08:56.373587 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.373429 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a8f83db4-079b-44ff-a400-0762ba4a1dfd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-blt8n\" (UID: \"a8f83db4-079b-44ff-a400-0762ba4a1dfd\") " pod="openshift-monitoring/node-exporter-blt8n" Jul 22 03:08:56.373587 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.373497 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a8f83db4-079b-44ff-a400-0762ba4a1dfd-node-exporter-textfile\") pod \"node-exporter-blt8n\" (UID: \"a8f83db4-079b-44ff-a400-0762ba4a1dfd\") " pod="openshift-monitoring/node-exporter-blt8n" Jul 22 03:08:56.373587 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.373526 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a8f83db4-079b-44ff-a400-0762ba4a1dfd-node-exporter-wtmp\") pod \"node-exporter-blt8n\" (UID: \"a8f83db4-079b-44ff-a400-0762ba4a1dfd\") " pod="openshift-monitoring/node-exporter-blt8n" Jul 22 03:08:56.373587 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.373550 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/168e0b81-ebe6-402a-99c6-e7881d11718a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-gh6jj\" (UID: \"168e0b81-ebe6-402a-99c6-e7881d11718a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gh6jj" Jul 22 03:08:56.373776 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.373614 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvzqs\" (UniqueName: \"kubernetes.io/projected/a8f83db4-079b-44ff-a400-0762ba4a1dfd-kube-api-access-bvzqs\") pod \"node-exporter-blt8n\" (UID: \"a8f83db4-079b-44ff-a400-0762ba4a1dfd\") " pod="openshift-monitoring/node-exporter-blt8n" Jul 22 03:08:56.373776 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.373640 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/168e0b81-ebe6-402a-99c6-e7881d11718a-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-gh6jj\" (UID: \"168e0b81-ebe6-402a-99c6-e7881d11718a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gh6jj" Jul 22 03:08:56.373776 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.373666 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8f83db4-079b-44ff-a400-0762ba4a1dfd-sys\") pod \"node-exporter-blt8n\" (UID: \"a8f83db4-079b-44ff-a400-0762ba4a1dfd\") " pod="openshift-monitoring/node-exporter-blt8n" Jul 22 03:08:56.373776 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.373681 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a8f83db4-079b-44ff-a400-0762ba4a1dfd-root\") pod \"node-exporter-blt8n\" (UID: \"a8f83db4-079b-44ff-a400-0762ba4a1dfd\") " pod="openshift-monitoring/node-exporter-blt8n" Jul 22 03:08:56.417033 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.416998 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:56.474091 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.474055 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvzqs\" (UniqueName: \"kubernetes.io/projected/a8f83db4-079b-44ff-a400-0762ba4a1dfd-kube-api-access-bvzqs\") pod \"node-exporter-blt8n\" (UID: \"a8f83db4-079b-44ff-a400-0762ba4a1dfd\") " pod="openshift-monitoring/node-exporter-blt8n" Jul 22 03:08:56.474091 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.474094 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/168e0b81-ebe6-402a-99c6-e7881d11718a-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-gh6jj\" (UID: \"168e0b81-ebe6-402a-99c6-e7881d11718a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gh6jj" Jul 22 03:08:56.474327 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.474122 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8f83db4-079b-44ff-a400-0762ba4a1dfd-sys\") pod \"node-exporter-blt8n\" (UID: \"a8f83db4-079b-44ff-a400-0762ba4a1dfd\") " pod="openshift-monitoring/node-exporter-blt8n" Jul 22 03:08:56.474327 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.474147 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a8f83db4-079b-44ff-a400-0762ba4a1dfd-root\") pod \"node-exporter-blt8n\" (UID: \"a8f83db4-079b-44ff-a400-0762ba4a1dfd\") " pod="openshift-monitoring/node-exporter-blt8n" Jul 22 03:08:56.474327 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.474173 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/168e0b81-ebe6-402a-99c6-e7881d11718a-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-gh6jj\" (UID: \"168e0b81-ebe6-402a-99c6-e7881d11718a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gh6jj" Jul 22 03:08:56.474327 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.474206 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlg5x\" (UniqueName: \"kubernetes.io/projected/168e0b81-ebe6-402a-99c6-e7881d11718a-kube-api-access-qlg5x\") pod \"kube-state-metrics-56c5594674-gh6jj\" (UID: \"168e0b81-ebe6-402a-99c6-e7881d11718a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gh6jj" Jul 22 03:08:56.474327 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.474210 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8f83db4-079b-44ff-a400-0762ba4a1dfd-sys\") pod \"node-exporter-blt8n\" (UID: \"a8f83db4-079b-44ff-a400-0762ba4a1dfd\") " pod="openshift-monitoring/node-exporter-blt8n" Jul 22 03:08:56.474327 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.474233 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a8f83db4-079b-44ff-a400-0762ba4a1dfd-node-exporter-accelerators-collector-config\") pod \"node-exporter-blt8n\" (UID: \"a8f83db4-079b-44ff-a400-0762ba4a1dfd\") " pod="openshift-monitoring/node-exporter-blt8n" Jul 22 03:08:56.474327 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.474233 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a8f83db4-079b-44ff-a400-0762ba4a1dfd-root\") pod \"node-exporter-blt8n\" (UID: \"a8f83db4-079b-44ff-a400-0762ba4a1dfd\") " pod="openshift-monitoring/node-exporter-blt8n" Jul 22 03:08:56.474327 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.474277 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/168e0b81-ebe6-402a-99c6-e7881d11718a-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-gh6jj\" (UID: \"168e0b81-ebe6-402a-99c6-e7881d11718a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gh6jj" Jul 22 03:08:56.474327 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.474315 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a8f83db4-079b-44ff-a400-0762ba4a1dfd-node-exporter-tls\") pod \"node-exporter-blt8n\" (UID: \"a8f83db4-079b-44ff-a400-0762ba4a1dfd\") " pod="openshift-monitoring/node-exporter-blt8n" Jul 22 03:08:56.474754 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:56.474334 2556 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 22 03:08:56.474754 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.474349 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8f83db4-079b-44ff-a400-0762ba4a1dfd-metrics-client-ca\") pod \"node-exporter-blt8n\" (UID: \"a8f83db4-079b-44ff-a400-0762ba4a1dfd\") " pod="openshift-monitoring/node-exporter-blt8n" Jul 22 03:08:56.474754 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.474375 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/168e0b81-ebe6-402a-99c6-e7881d11718a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-gh6jj\" (UID: \"168e0b81-ebe6-402a-99c6-e7881d11718a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gh6jj" Jul 22 03:08:56.474754 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:08:56.474399 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/168e0b81-ebe6-402a-99c6-e7881d11718a-kube-state-metrics-tls podName:168e0b81-ebe6-402a-99c6-e7881d11718a nodeName:}" failed. No retries permitted until 2026-07-22 03:08:56.974379536 +0000 UTC m=+77.423362769 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/168e0b81-ebe6-402a-99c6-e7881d11718a-kube-state-metrics-tls") pod "kube-state-metrics-56c5594674-gh6jj" (UID: "168e0b81-ebe6-402a-99c6-e7881d11718a") : secret "kube-state-metrics-tls" not found Jul 22 03:08:56.474754 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.474454 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a8f83db4-079b-44ff-a400-0762ba4a1dfd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-blt8n\" (UID: \"a8f83db4-079b-44ff-a400-0762ba4a1dfd\") " pod="openshift-monitoring/node-exporter-blt8n" Jul 22 03:08:56.474754 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.474507 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a8f83db4-079b-44ff-a400-0762ba4a1dfd-node-exporter-textfile\") pod \"node-exporter-blt8n\" (UID: \"a8f83db4-079b-44ff-a400-0762ba4a1dfd\") " pod="openshift-monitoring/node-exporter-blt8n" Jul 22 03:08:56.474754 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.474541 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a8f83db4-079b-44ff-a400-0762ba4a1dfd-node-exporter-wtmp\") pod \"node-exporter-blt8n\" (UID: \"a8f83db4-079b-44ff-a400-0762ba4a1dfd\") " pod="openshift-monitoring/node-exporter-blt8n" Jul 22 03:08:56.474754 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.474550 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/168e0b81-ebe6-402a-99c6-e7881d11718a-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-gh6jj\" (UID: \"168e0b81-ebe6-402a-99c6-e7881d11718a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gh6jj" Jul 22 03:08:56.474754 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.474567 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/168e0b81-ebe6-402a-99c6-e7881d11718a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-gh6jj\" (UID: \"168e0b81-ebe6-402a-99c6-e7881d11718a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gh6jj" Jul 22 03:08:56.475212 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.474826 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a8f83db4-079b-44ff-a400-0762ba4a1dfd-node-exporter-accelerators-collector-config\") pod \"node-exporter-blt8n\" (UID: \"a8f83db4-079b-44ff-a400-0762ba4a1dfd\") " pod="openshift-monitoring/node-exporter-blt8n" Jul 22 03:08:56.475212 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.474931 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8f83db4-079b-44ff-a400-0762ba4a1dfd-metrics-client-ca\") pod \"node-exporter-blt8n\" (UID: \"a8f83db4-079b-44ff-a400-0762ba4a1dfd\") " pod="openshift-monitoring/node-exporter-blt8n" Jul 22 03:08:56.475212 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.475027 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a8f83db4-079b-44ff-a400-0762ba4a1dfd-node-exporter-wtmp\") pod \"node-exporter-blt8n\" (UID: \"a8f83db4-079b-44ff-a400-0762ba4a1dfd\") " pod="openshift-monitoring/node-exporter-blt8n" Jul 22 03:08:56.475212 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.475092 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a8f83db4-079b-44ff-a400-0762ba4a1dfd-node-exporter-textfile\") pod \"node-exporter-blt8n\" (UID: \"a8f83db4-079b-44ff-a400-0762ba4a1dfd\") " pod="openshift-monitoring/node-exporter-blt8n" Jul 22 03:08:56.475415 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.475371 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/168e0b81-ebe6-402a-99c6-e7881d11718a-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-gh6jj\" (UID: \"168e0b81-ebe6-402a-99c6-e7881d11718a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gh6jj" Jul 22 03:08:56.475534 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.475511 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/168e0b81-ebe6-402a-99c6-e7881d11718a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-gh6jj\" (UID: \"168e0b81-ebe6-402a-99c6-e7881d11718a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gh6jj" Jul 22 03:08:56.476916 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.476880 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a8f83db4-079b-44ff-a400-0762ba4a1dfd-node-exporter-tls\") pod \"node-exporter-blt8n\" (UID: \"a8f83db4-079b-44ff-a400-0762ba4a1dfd\") " pod="openshift-monitoring/node-exporter-blt8n" Jul 22 03:08:56.477024 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.476970 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a8f83db4-079b-44ff-a400-0762ba4a1dfd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-blt8n\" (UID: \"a8f83db4-079b-44ff-a400-0762ba4a1dfd\") " pod="openshift-monitoring/node-exporter-blt8n" Jul 22 03:08:56.477336 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.477317 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/168e0b81-ebe6-402a-99c6-e7881d11718a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-gh6jj\" (UID: \"168e0b81-ebe6-402a-99c6-e7881d11718a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gh6jj" Jul 22 03:08:56.483152 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.483104 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlg5x\" (UniqueName: \"kubernetes.io/projected/168e0b81-ebe6-402a-99c6-e7881d11718a-kube-api-access-qlg5x\") pod \"kube-state-metrics-56c5594674-gh6jj\" (UID: \"168e0b81-ebe6-402a-99c6-e7881d11718a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gh6jj" Jul 22 03:08:56.483552 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.483529 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvzqs\" (UniqueName: \"kubernetes.io/projected/a8f83db4-079b-44ff-a400-0762ba4a1dfd-kube-api-access-bvzqs\") pod \"node-exporter-blt8n\" (UID: \"a8f83db4-079b-44ff-a400-0762ba4a1dfd\") " pod="openshift-monitoring/node-exporter-blt8n" Jul 22 03:08:56.521058 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.520978 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:56.574762 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.574734 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-blt8n" Jul 22 03:08:56.589658 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:08:56.589630 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8f83db4_079b_44ff_a400_0762ba4a1dfd.slice/crio-7688867820babffb5bade8e8dee54fd7b198c1f90367a593827901ed205cfa57 WatchSource:0}: Error finding container 7688867820babffb5bade8e8dee54fd7b198c1f90367a593827901ed205cfa57: Status 404 returned error can't find the container with id 7688867820babffb5bade8e8dee54fd7b198c1f90367a593827901ed205cfa57 Jul 22 03:08:56.623261 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.623225 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:56.719739 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.719701 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:56.819612 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.819575 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:56.977036 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.976996 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/168e0b81-ebe6-402a-99c6-e7881d11718a-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-gh6jj\" (UID: \"168e0b81-ebe6-402a-99c6-e7881d11718a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gh6jj" Jul 22 03:08:56.979550 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:56.979523 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/168e0b81-ebe6-402a-99c6-e7881d11718a-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-gh6jj\" (UID: \"168e0b81-ebe6-402a-99c6-e7881d11718a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gh6jj" Jul 22 03:08:57.195691 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:57.195616 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-gh6jj" Jul 22 03:08:57.250779 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:57.250744 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:57.356399 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:57.356362 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:57.373016 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:57.372982 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-blt8n" event={"ID":"a8f83db4-079b-44ff-a400-0762ba4a1dfd","Type":"ContainerStarted","Data":"7688867820babffb5bade8e8dee54fd7b198c1f90367a593827901ed205cfa57"} Jul 22 03:08:57.543901 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:57.543878 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-gh6jj"] Jul 22 03:08:57.545745 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:08:57.545719 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod168e0b81_ebe6_402a_99c6_e7881d11718a.slice/crio-1d7d5fb64ff8a3d366d1e0b4a248394257d762fcfc71995a95c574562593ac33 WatchSource:0}: Error finding container 1d7d5fb64ff8a3d366d1e0b4a248394257d762fcfc71995a95c574562593ac33: Status 404 returned error can't find the container with id 1d7d5fb64ff8a3d366d1e0b4a248394257d762fcfc71995a95c574562593ac33 Jul 22 03:08:57.547589 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:57.547557 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:57.749105 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:57.749015 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:57.819904 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:57.819855 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:58.377775 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:58.377741 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-gh6jj" event={"ID":"168e0b81-ebe6-402a-99c6-e7881d11718a","Type":"ContainerStarted","Data":"1d7d5fb64ff8a3d366d1e0b4a248394257d762fcfc71995a95c574562593ac33"} Jul 22 03:08:58.379159 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:58.379132 2556 generic.go:356] "Generic (PLEG): container finished" podID="a8f83db4-079b-44ff-a400-0762ba4a1dfd" containerID="a0f014cf61ad3afff20b8c6c142bad63520e190826731b06cff366871292ca8b" exitCode=0 Jul 22 03:08:58.379288 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:58.379178 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-blt8n" event={"ID":"a8f83db4-079b-44ff-a400-0762ba4a1dfd","Type":"ContainerDied","Data":"a0f014cf61ad3afff20b8c6c142bad63520e190826731b06cff366871292ca8b"} Jul 22 03:08:59.178173 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:59.178097 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:59.278908 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:59.278877 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:59.389064 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:59.388980 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-blt8n" event={"ID":"a8f83db4-079b-44ff-a400-0762ba4a1dfd","Type":"ContainerStarted","Data":"0cb9c94f4b1714cfb8792a1e7eae84d6952216eb9aa19c37debe77a1c6a3715d"} Jul 22 03:08:59.389231 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:59.389072 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-blt8n" event={"ID":"a8f83db4-079b-44ff-a400-0762ba4a1dfd","Type":"ContainerStarted","Data":"b833ca4d24799a7d913bf110b865c2fdc7bc345936761a131c579053e29c2062"} Jul 22 03:08:59.391450 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:59.391302 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-gh6jj" event={"ID":"168e0b81-ebe6-402a-99c6-e7881d11718a","Type":"ContainerStarted","Data":"13a24a5fb5d2e1321c539878fa651048b129984c6c1226135eb55a1eba036cb5"} Jul 22 03:08:59.391450 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:59.391357 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-gh6jj" event={"ID":"168e0b81-ebe6-402a-99c6-e7881d11718a","Type":"ContainerStarted","Data":"515af6d00ed3da5802bd492b673b1363506635cc5a6960e5c10a25f1fc13510e"} Jul 22 03:08:59.391450 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:59.391388 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-gh6jj" event={"ID":"168e0b81-ebe6-402a-99c6-e7881d11718a","Type":"ContainerStarted","Data":"b3935dbf997b3e5d40ccf0b5ad208623de87aaa1a625afca94d3a1ec22ad4530"} Jul 22 03:08:59.411339 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:59.411284 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-blt8n" podStartSLOduration=2.474002358 podStartE2EDuration="3.411267769s" totalImagesPullingTime="867.8362ms" totalInitContainerRuntime="69.429211ms" isStatefulPod=true podCreationTimestamp="2026-07-22 03:08:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:08:59.409395038 +0000 UTC m=+79.858378287" watchObservedRunningTime="2026-07-22 03:08:59.411267769 +0000 UTC m=+79.860251017" Jul 22 03:08:59.430524 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:59.430266 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-gh6jj" podStartSLOduration=1.868099371 podStartE2EDuration="3.430241359s" totalImagesPullingTime="1.562141988s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:08:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:08:59.429995698 +0000 UTC m=+79.878978972" watchObservedRunningTime="2026-07-22 03:08:59.430241359 +0000 UTC m=+79.879224608" Jul 22 03:08:59.452737 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:59.452693 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:59.574416 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:08:59.574382 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:00.986996 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:00.986867 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-mpzw6"] Jul 22 03:09:00.990107 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:00.990088 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-mpzw6" Jul 22 03:09:00.992645 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:00.992623 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 22 03:09:00.992726 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:00.992715 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-dr62c\"" Jul 22 03:09:00.999058 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:00.999035 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-mpzw6"] Jul 22 03:09:00.999669 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:00.999644 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:01.114841 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:01.114804 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/48041ebc-ec47-4b82-86b1-50b154e9b453-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-mpzw6\" (UID: \"48041ebc-ec47-4b82-86b1-50b154e9b453\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-mpzw6" Jul 22 03:09:01.115053 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:01.114878 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwzq9\" (UniqueName: \"kubernetes.io/projected/48041ebc-ec47-4b82-86b1-50b154e9b453-kube-api-access-gwzq9\") pod \"monitoring-plugin-7b7b7bc8bf-mpzw6\" (UID: \"48041ebc-ec47-4b82-86b1-50b154e9b453\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-mpzw6" Jul 22 03:09:01.141898 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:01.141853 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:01.216139 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:01.216096 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/48041ebc-ec47-4b82-86b1-50b154e9b453-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-mpzw6\" (UID: \"48041ebc-ec47-4b82-86b1-50b154e9b453\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-mpzw6" Jul 22 03:09:01.216307 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:01.216159 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwzq9\" (UniqueName: \"kubernetes.io/projected/48041ebc-ec47-4b82-86b1-50b154e9b453-kube-api-access-gwzq9\") pod \"monitoring-plugin-7b7b7bc8bf-mpzw6\" (UID: \"48041ebc-ec47-4b82-86b1-50b154e9b453\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-mpzw6" Jul 22 03:09:01.218476 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:01.218457 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/48041ebc-ec47-4b82-86b1-50b154e9b453-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-mpzw6\" (UID: \"48041ebc-ec47-4b82-86b1-50b154e9b453\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-mpzw6" Jul 22 03:09:01.224560 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:01.224536 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwzq9\" (UniqueName: \"kubernetes.io/projected/48041ebc-ec47-4b82-86b1-50b154e9b453-kube-api-access-gwzq9\") pod \"monitoring-plugin-7b7b7bc8bf-mpzw6\" (UID: \"48041ebc-ec47-4b82-86b1-50b154e9b453\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-mpzw6" Jul 22 03:09:01.302642 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:01.302567 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-mpzw6" Jul 22 03:09:01.429341 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:01.429313 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-mpzw6"] Jul 22 03:09:01.431672 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:09:01.431646 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48041ebc_ec47_4b82_86b1_50b154e9b453.slice/crio-5645c5f693dbf73756cf71c3132fe2a4947d8a381d7fd2443bea418374e0a9ca WatchSource:0}: Error finding container 5645c5f693dbf73756cf71c3132fe2a4947d8a381d7fd2443bea418374e0a9ca: Status 404 returned error can't find the container with id 5645c5f693dbf73756cf71c3132fe2a4947d8a381d7fd2443bea418374e0a9ca Jul 22 03:09:01.791159 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:01.791117 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:01.885925 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:01.885878 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:01.986037 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:01.985999 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:02.117244 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:02.117213 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:02.405246 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:02.405151 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-mpzw6" event={"ID":"48041ebc-ec47-4b82-86b1-50b154e9b453","Type":"ContainerStarted","Data":"5645c5f693dbf73756cf71c3132fe2a4947d8a381d7fd2443bea418374e0a9ca"} Jul 22 03:09:02.495072 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:02.494939 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:02.585994 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:02.585926 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:02.728805 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:02.728734 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:03.409760 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:03.409722 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-mpzw6" event={"ID":"48041ebc-ec47-4b82-86b1-50b154e9b453","Type":"ContainerStarted","Data":"9814e59eb1b8ae6664f187480e8e079921aef72e4a8d1c4324765c29bb563189"} Jul 22 03:09:03.410189 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:03.409919 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-mpzw6" Jul 22 03:09:03.414946 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:03.414922 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-mpzw6" Jul 22 03:09:03.426452 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:03.426402 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-mpzw6" podStartSLOduration=1.677673268 podStartE2EDuration="3.426384727s" totalImagesPullingTime="1.748711459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:09:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:09:03.426311183 +0000 UTC m=+83.875294437" watchObservedRunningTime="2026-07-22 03:09:03.426384727 +0000 UTC m=+83.875367978" Jul 22 03:09:03.661860 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:03.661833 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:03.763657 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:03.763626 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:03.882485 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:03.882453 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:04.087923 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:04.087886 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:04.162579 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:04.162548 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:05.300788 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:05.300761 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-8458747fff-b6gnb" Jul 22 03:09:05.466531 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:05.466501 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:05.560848 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:05.560780 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:08.022767 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:08.022728 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" podUID="0dbb2e21-4aa1-43df-8917-adc7034719b0" containerName="registry" containerID="cri-o://5d7ade5f0188a34d1f762c644d3541a13e88fbf27fc803040bd47449401ec18a" gracePeriod=30 Jul 22 03:09:08.260070 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:08.260047 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" Jul 22 03:09:08.374501 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:08.374466 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0dbb2e21-4aa1-43df-8917-adc7034719b0-registry-certificates\") pod \"0dbb2e21-4aa1-43df-8917-adc7034719b0\" (UID: \"0dbb2e21-4aa1-43df-8917-adc7034719b0\") " Jul 22 03:09:08.374663 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:08.374515 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0dbb2e21-4aa1-43df-8917-adc7034719b0-installation-pull-secrets\") pod \"0dbb2e21-4aa1-43df-8917-adc7034719b0\" (UID: \"0dbb2e21-4aa1-43df-8917-adc7034719b0\") " Jul 22 03:09:08.374663 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:08.374543 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0dbb2e21-4aa1-43df-8917-adc7034719b0-ca-trust-extracted\") pod \"0dbb2e21-4aa1-43df-8917-adc7034719b0\" (UID: \"0dbb2e21-4aa1-43df-8917-adc7034719b0\") " Jul 22 03:09:08.374663 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:08.374581 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0dbb2e21-4aa1-43df-8917-adc7034719b0-trusted-ca\") pod \"0dbb2e21-4aa1-43df-8917-adc7034719b0\" (UID: \"0dbb2e21-4aa1-43df-8917-adc7034719b0\") " Jul 22 03:09:08.374663 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:08.374612 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ff4g6\" (UniqueName: \"kubernetes.io/projected/0dbb2e21-4aa1-43df-8917-adc7034719b0-kube-api-access-ff4g6\") pod \"0dbb2e21-4aa1-43df-8917-adc7034719b0\" (UID: \"0dbb2e21-4aa1-43df-8917-adc7034719b0\") " Jul 22 03:09:08.374663 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:08.374642 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0dbb2e21-4aa1-43df-8917-adc7034719b0-registry-tls\") pod \"0dbb2e21-4aa1-43df-8917-adc7034719b0\" (UID: \"0dbb2e21-4aa1-43df-8917-adc7034719b0\") " Jul 22 03:09:08.374906 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:08.374672 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0dbb2e21-4aa1-43df-8917-adc7034719b0-bound-sa-token\") pod \"0dbb2e21-4aa1-43df-8917-adc7034719b0\" (UID: \"0dbb2e21-4aa1-43df-8917-adc7034719b0\") " Jul 22 03:09:08.374906 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:08.374716 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0dbb2e21-4aa1-43df-8917-adc7034719b0-image-registry-private-configuration\") pod \"0dbb2e21-4aa1-43df-8917-adc7034719b0\" (UID: \"0dbb2e21-4aa1-43df-8917-adc7034719b0\") " Jul 22 03:09:08.375042 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:08.374921 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0dbb2e21-4aa1-43df-8917-adc7034719b0-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "0dbb2e21-4aa1-43df-8917-adc7034719b0" (UID: "0dbb2e21-4aa1-43df-8917-adc7034719b0"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:09:08.375042 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:08.375025 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0dbb2e21-4aa1-43df-8917-adc7034719b0-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "0dbb2e21-4aa1-43df-8917-adc7034719b0" (UID: "0dbb2e21-4aa1-43df-8917-adc7034719b0"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:09:08.377295 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:08.377269 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0dbb2e21-4aa1-43df-8917-adc7034719b0-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "0dbb2e21-4aa1-43df-8917-adc7034719b0" (UID: "0dbb2e21-4aa1-43df-8917-adc7034719b0"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:09:08.377482 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:08.377456 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0dbb2e21-4aa1-43df-8917-adc7034719b0-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "0dbb2e21-4aa1-43df-8917-adc7034719b0" (UID: "0dbb2e21-4aa1-43df-8917-adc7034719b0"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:09:08.377555 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:08.377507 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0dbb2e21-4aa1-43df-8917-adc7034719b0-kube-api-access-ff4g6" (OuterVolumeSpecName: "kube-api-access-ff4g6") pod "0dbb2e21-4aa1-43df-8917-adc7034719b0" (UID: "0dbb2e21-4aa1-43df-8917-adc7034719b0"). InnerVolumeSpecName "kube-api-access-ff4g6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:09:08.377644 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:08.377615 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0dbb2e21-4aa1-43df-8917-adc7034719b0-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "0dbb2e21-4aa1-43df-8917-adc7034719b0" (UID: "0dbb2e21-4aa1-43df-8917-adc7034719b0"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:09:08.377734 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:08.377651 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0dbb2e21-4aa1-43df-8917-adc7034719b0-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "0dbb2e21-4aa1-43df-8917-adc7034719b0" (UID: "0dbb2e21-4aa1-43df-8917-adc7034719b0"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:09:08.382918 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:08.382893 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0dbb2e21-4aa1-43df-8917-adc7034719b0-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "0dbb2e21-4aa1-43df-8917-adc7034719b0" (UID: "0dbb2e21-4aa1-43df-8917-adc7034719b0"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:09:08.439028 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:08.438986 2556 generic.go:356] "Generic (PLEG): container finished" podID="0dbb2e21-4aa1-43df-8917-adc7034719b0" containerID="5d7ade5f0188a34d1f762c644d3541a13e88fbf27fc803040bd47449401ec18a" exitCode=0 Jul 22 03:09:08.439178 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:08.439052 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" Jul 22 03:09:08.439178 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:08.439057 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" event={"ID":"0dbb2e21-4aa1-43df-8917-adc7034719b0","Type":"ContainerDied","Data":"5d7ade5f0188a34d1f762c644d3541a13e88fbf27fc803040bd47449401ec18a"} Jul 22 03:09:08.439178 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:08.439131 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7fcbd97f78-7hd7h" event={"ID":"0dbb2e21-4aa1-43df-8917-adc7034719b0","Type":"ContainerDied","Data":"3d3fdbdf0ae75d9e5a06af3b1fab56edf85fd41c7112d269cb566e7605c1b7d2"} Jul 22 03:09:08.439178 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:08.439160 2556 scope.go:117] "RemoveContainer" containerID="5d7ade5f0188a34d1f762c644d3541a13e88fbf27fc803040bd47449401ec18a" Jul 22 03:09:08.448069 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:08.448052 2556 scope.go:117] "RemoveContainer" containerID="5d7ade5f0188a34d1f762c644d3541a13e88fbf27fc803040bd47449401ec18a" Jul 22 03:09:08.448345 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:09:08.448322 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d7ade5f0188a34d1f762c644d3541a13e88fbf27fc803040bd47449401ec18a\": container with ID starting with 5d7ade5f0188a34d1f762c644d3541a13e88fbf27fc803040bd47449401ec18a not found: ID does not exist" containerID="5d7ade5f0188a34d1f762c644d3541a13e88fbf27fc803040bd47449401ec18a" Jul 22 03:09:08.448405 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:08.448353 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d7ade5f0188a34d1f762c644d3541a13e88fbf27fc803040bd47449401ec18a"} err="failed to get container status \"5d7ade5f0188a34d1f762c644d3541a13e88fbf27fc803040bd47449401ec18a\": rpc error: code = NotFound desc = could not find container \"5d7ade5f0188a34d1f762c644d3541a13e88fbf27fc803040bd47449401ec18a\": container with ID starting with 5d7ade5f0188a34d1f762c644d3541a13e88fbf27fc803040bd47449401ec18a not found: ID does not exist" Jul 22 03:09:08.461978 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:08.461941 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7fcbd97f78-7hd7h"] Jul 22 03:09:08.465627 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:08.465609 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7fcbd97f78-7hd7h"] Jul 22 03:09:08.475977 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:08.475939 2556 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0dbb2e21-4aa1-43df-8917-adc7034719b0-registry-tls\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:09:08.476073 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:08.475982 2556 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0dbb2e21-4aa1-43df-8917-adc7034719b0-bound-sa-token\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:09:08.476073 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:08.475997 2556 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0dbb2e21-4aa1-43df-8917-adc7034719b0-image-registry-private-configuration\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:09:08.476073 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:08.476010 2556 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0dbb2e21-4aa1-43df-8917-adc7034719b0-registry-certificates\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:09:08.476073 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:08.476024 2556 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0dbb2e21-4aa1-43df-8917-adc7034719b0-installation-pull-secrets\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:09:08.476073 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:08.476037 2556 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0dbb2e21-4aa1-43df-8917-adc7034719b0-ca-trust-extracted\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:09:08.476073 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:08.476050 2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0dbb2e21-4aa1-43df-8917-adc7034719b0-trusted-ca\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:09:08.476073 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:08.476065 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ff4g6\" (UniqueName: \"kubernetes.io/projected/0dbb2e21-4aa1-43df-8917-adc7034719b0-kube-api-access-ff4g6\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:09:10.023815 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:10.023784 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0dbb2e21-4aa1-43df-8917-adc7034719b0" path="/var/lib/kubelet/pods/0dbb2e21-4aa1-43df-8917-adc7034719b0/volumes" Jul 22 03:09:10.146327 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:10.146297 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:10.249521 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:10.249467 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:11.098185 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:11.098151 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7b77548757-tj9kq"] Jul 22 03:09:11.098548 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:11.098411 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0dbb2e21-4aa1-43df-8917-adc7034719b0" containerName="registry" Jul 22 03:09:11.098548 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:11.098421 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0dbb2e21-4aa1-43df-8917-adc7034719b0" containerName="registry" Jul 22 03:09:11.098548 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:11.098462 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="0dbb2e21-4aa1-43df-8917-adc7034719b0" containerName="registry" Jul 22 03:09:11.103272 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:11.103255 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b77548757-tj9kq" Jul 22 03:09:11.108381 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:11.108359 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 22 03:09:11.108917 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:11.108894 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 22 03:09:11.109715 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:11.109691 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 22 03:09:11.109816 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:11.109729 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 22 03:09:11.109886 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:11.109836 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 22 03:09:11.109943 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:11.109856 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-xdrf9\"" Jul 22 03:09:11.109943 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:11.109837 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 22 03:09:11.110067 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:11.109980 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 22 03:09:11.113229 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:11.113209 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b77548757-tj9kq"] Jul 22 03:09:11.198101 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:11.198066 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fec7fc17-a2f1-4731-873f-73b5aba8d874-console-oauth-config\") pod \"console-7b77548757-tj9kq\" (UID: \"fec7fc17-a2f1-4731-873f-73b5aba8d874\") " pod="openshift-console/console-7b77548757-tj9kq" Jul 22 03:09:11.198101 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:11.198101 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fec7fc17-a2f1-4731-873f-73b5aba8d874-oauth-serving-cert\") pod \"console-7b77548757-tj9kq\" (UID: \"fec7fc17-a2f1-4731-873f-73b5aba8d874\") " pod="openshift-console/console-7b77548757-tj9kq" Jul 22 03:09:11.198339 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:11.198128 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g66bk\" (UniqueName: \"kubernetes.io/projected/fec7fc17-a2f1-4731-873f-73b5aba8d874-kube-api-access-g66bk\") pod \"console-7b77548757-tj9kq\" (UID: \"fec7fc17-a2f1-4731-873f-73b5aba8d874\") " pod="openshift-console/console-7b77548757-tj9kq" Jul 22 03:09:11.198339 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:11.198251 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fec7fc17-a2f1-4731-873f-73b5aba8d874-service-ca\") pod \"console-7b77548757-tj9kq\" (UID: \"fec7fc17-a2f1-4731-873f-73b5aba8d874\") " pod="openshift-console/console-7b77548757-tj9kq" Jul 22 03:09:11.198339 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:11.198309 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fec7fc17-a2f1-4731-873f-73b5aba8d874-console-config\") pod \"console-7b77548757-tj9kq\" (UID: \"fec7fc17-a2f1-4731-873f-73b5aba8d874\") " pod="openshift-console/console-7b77548757-tj9kq" Jul 22 03:09:11.198483 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:11.198364 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fec7fc17-a2f1-4731-873f-73b5aba8d874-console-serving-cert\") pod \"console-7b77548757-tj9kq\" (UID: \"fec7fc17-a2f1-4731-873f-73b5aba8d874\") " pod="openshift-console/console-7b77548757-tj9kq" Jul 22 03:09:11.299413 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:11.299377 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fec7fc17-a2f1-4731-873f-73b5aba8d874-console-serving-cert\") pod \"console-7b77548757-tj9kq\" (UID: \"fec7fc17-a2f1-4731-873f-73b5aba8d874\") " pod="openshift-console/console-7b77548757-tj9kq" Jul 22 03:09:11.299600 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:11.299418 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fec7fc17-a2f1-4731-873f-73b5aba8d874-console-oauth-config\") pod \"console-7b77548757-tj9kq\" (UID: \"fec7fc17-a2f1-4731-873f-73b5aba8d874\") " pod="openshift-console/console-7b77548757-tj9kq" Jul 22 03:09:11.299600 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:11.299443 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fec7fc17-a2f1-4731-873f-73b5aba8d874-oauth-serving-cert\") pod \"console-7b77548757-tj9kq\" (UID: \"fec7fc17-a2f1-4731-873f-73b5aba8d874\") " pod="openshift-console/console-7b77548757-tj9kq" Jul 22 03:09:11.299600 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:11.299487 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g66bk\" (UniqueName: \"kubernetes.io/projected/fec7fc17-a2f1-4731-873f-73b5aba8d874-kube-api-access-g66bk\") pod \"console-7b77548757-tj9kq\" (UID: \"fec7fc17-a2f1-4731-873f-73b5aba8d874\") " pod="openshift-console/console-7b77548757-tj9kq" Jul 22 03:09:11.299600 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:11.299518 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fec7fc17-a2f1-4731-873f-73b5aba8d874-service-ca\") pod \"console-7b77548757-tj9kq\" (UID: \"fec7fc17-a2f1-4731-873f-73b5aba8d874\") " pod="openshift-console/console-7b77548757-tj9kq" Jul 22 03:09:11.299600 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:11.299548 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fec7fc17-a2f1-4731-873f-73b5aba8d874-console-config\") pod \"console-7b77548757-tj9kq\" (UID: \"fec7fc17-a2f1-4731-873f-73b5aba8d874\") " pod="openshift-console/console-7b77548757-tj9kq" Jul 22 03:09:11.300255 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:11.300229 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fec7fc17-a2f1-4731-873f-73b5aba8d874-oauth-serving-cert\") pod \"console-7b77548757-tj9kq\" (UID: \"fec7fc17-a2f1-4731-873f-73b5aba8d874\") " pod="openshift-console/console-7b77548757-tj9kq" Jul 22 03:09:11.300387 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:11.300365 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fec7fc17-a2f1-4731-873f-73b5aba8d874-console-config\") pod \"console-7b77548757-tj9kq\" (UID: \"fec7fc17-a2f1-4731-873f-73b5aba8d874\") " pod="openshift-console/console-7b77548757-tj9kq" Jul 22 03:09:11.300444 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:11.300384 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fec7fc17-a2f1-4731-873f-73b5aba8d874-service-ca\") pod \"console-7b77548757-tj9kq\" (UID: \"fec7fc17-a2f1-4731-873f-73b5aba8d874\") " pod="openshift-console/console-7b77548757-tj9kq" Jul 22 03:09:11.301975 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:11.301924 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fec7fc17-a2f1-4731-873f-73b5aba8d874-console-serving-cert\") pod \"console-7b77548757-tj9kq\" (UID: \"fec7fc17-a2f1-4731-873f-73b5aba8d874\") " pod="openshift-console/console-7b77548757-tj9kq" Jul 22 03:09:11.301975 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:11.301928 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fec7fc17-a2f1-4731-873f-73b5aba8d874-console-oauth-config\") pod \"console-7b77548757-tj9kq\" (UID: \"fec7fc17-a2f1-4731-873f-73b5aba8d874\") " pod="openshift-console/console-7b77548757-tj9kq" Jul 22 03:09:11.309099 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:11.309081 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g66bk\" (UniqueName: \"kubernetes.io/projected/fec7fc17-a2f1-4731-873f-73b5aba8d874-kube-api-access-g66bk\") pod \"console-7b77548757-tj9kq\" (UID: \"fec7fc17-a2f1-4731-873f-73b5aba8d874\") " pod="openshift-console/console-7b77548757-tj9kq" Jul 22 03:09:11.412437 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:11.412357 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b77548757-tj9kq" Jul 22 03:09:11.481184 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:11.481151 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:11.545751 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:11.545728 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b77548757-tj9kq"] Jul 22 03:09:11.548022 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:09:11.547992 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfec7fc17_a2f1_4731_873f_73b5aba8d874.slice/crio-91be103cc1b7804959bc0aaea0f9b0b10d2c0d5d714b26f1044b1ea3ed96eba9 WatchSource:0}: Error finding container 91be103cc1b7804959bc0aaea0f9b0b10d2c0d5d714b26f1044b1ea3ed96eba9: Status 404 returned error can't find the container with id 91be103cc1b7804959bc0aaea0f9b0b10d2c0d5d714b26f1044b1ea3ed96eba9 Jul 22 03:09:11.550361 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:11.550337 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:12.456856 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:12.456816 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b77548757-tj9kq" event={"ID":"fec7fc17-a2f1-4731-873f-73b5aba8d874","Type":"ContainerStarted","Data":"91be103cc1b7804959bc0aaea0f9b0b10d2c0d5d714b26f1044b1ea3ed96eba9"} Jul 22 03:09:13.042004 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:13.041973 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:13.113725 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:13.113687 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:13.633325 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:13.633289 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:13.735942 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:13.735911 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:14.586355 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:14.586317 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:14.658647 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:14.658605 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:15.470901 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:15.470864 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b77548757-tj9kq" event={"ID":"fec7fc17-a2f1-4731-873f-73b5aba8d874","Type":"ContainerStarted","Data":"638501eaea19ea8b7334774e995aa0e8909a6f1fb5ef9560ad3ae8795d977122"} Jul 22 03:09:15.502970 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:15.502906 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7b77548757-tj9kq" podStartSLOduration=0.743170437 podStartE2EDuration="4.502891213s" totalImagesPullingTime="3.759720776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:09:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:09:15.501119608 +0000 UTC m=+95.950102858" watchObservedRunningTime="2026-07-22 03:09:15.502891213 +0000 UTC m=+95.951874462" Jul 22 03:09:15.505206 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:15.505182 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-ff5d4b79d-psddb"] Jul 22 03:09:15.508868 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:15.508844 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:15.522611 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:15.522590 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-ff5d4b79d-psddb" Jul 22 03:09:15.529148 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:15.529118 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-ff5d4b79d-psddb"] Jul 22 03:09:15.536715 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:15.536693 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 22 03:09:15.612805 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:15.612769 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:15.635462 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:15.635430 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4731dfe-1339-4b79-a3e2-3e904d27f53e-console-oauth-config\") pod \"console-ff5d4b79d-psddb\" (UID: \"c4731dfe-1339-4b79-a3e2-3e904d27f53e\") " pod="openshift-console/console-ff5d4b79d-psddb" Jul 22 03:09:15.635627 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:15.635543 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4731dfe-1339-4b79-a3e2-3e904d27f53e-service-ca\") pod \"console-ff5d4b79d-psddb\" (UID: \"c4731dfe-1339-4b79-a3e2-3e904d27f53e\") " pod="openshift-console/console-ff5d4b79d-psddb" Jul 22 03:09:15.635689 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:15.635670 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4731dfe-1339-4b79-a3e2-3e904d27f53e-console-serving-cert\") pod \"console-ff5d4b79d-psddb\" (UID: \"c4731dfe-1339-4b79-a3e2-3e904d27f53e\") " pod="openshift-console/console-ff5d4b79d-psddb" Jul 22 03:09:15.635735 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:15.635693 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4731dfe-1339-4b79-a3e2-3e904d27f53e-oauth-serving-cert\") pod \"console-ff5d4b79d-psddb\" (UID: \"c4731dfe-1339-4b79-a3e2-3e904d27f53e\") " pod="openshift-console/console-ff5d4b79d-psddb" Jul 22 03:09:15.635735 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:15.635722 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4731dfe-1339-4b79-a3e2-3e904d27f53e-console-config\") pod \"console-ff5d4b79d-psddb\" (UID: \"c4731dfe-1339-4b79-a3e2-3e904d27f53e\") " pod="openshift-console/console-ff5d4b79d-psddb" Jul 22 03:09:15.635825 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:15.635776 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4731dfe-1339-4b79-a3e2-3e904d27f53e-trusted-ca-bundle\") pod \"console-ff5d4b79d-psddb\" (UID: \"c4731dfe-1339-4b79-a3e2-3e904d27f53e\") " pod="openshift-console/console-ff5d4b79d-psddb" Jul 22 03:09:15.635908 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:15.635884 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fs94\" (UniqueName: \"kubernetes.io/projected/c4731dfe-1339-4b79-a3e2-3e904d27f53e-kube-api-access-2fs94\") pod \"console-ff5d4b79d-psddb\" (UID: \"c4731dfe-1339-4b79-a3e2-3e904d27f53e\") " pod="openshift-console/console-ff5d4b79d-psddb" Jul 22 03:09:15.737323 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:15.737289 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4731dfe-1339-4b79-a3e2-3e904d27f53e-service-ca\") pod \"console-ff5d4b79d-psddb\" (UID: \"c4731dfe-1339-4b79-a3e2-3e904d27f53e\") " pod="openshift-console/console-ff5d4b79d-psddb" Jul 22 03:09:15.737753 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:15.737333 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4731dfe-1339-4b79-a3e2-3e904d27f53e-console-serving-cert\") pod \"console-ff5d4b79d-psddb\" (UID: \"c4731dfe-1339-4b79-a3e2-3e904d27f53e\") " pod="openshift-console/console-ff5d4b79d-psddb" Jul 22 03:09:15.737753 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:15.737368 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4731dfe-1339-4b79-a3e2-3e904d27f53e-oauth-serving-cert\") pod \"console-ff5d4b79d-psddb\" (UID: \"c4731dfe-1339-4b79-a3e2-3e904d27f53e\") " pod="openshift-console/console-ff5d4b79d-psddb" Jul 22 03:09:15.737753 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:15.737387 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4731dfe-1339-4b79-a3e2-3e904d27f53e-console-config\") pod \"console-ff5d4b79d-psddb\" (UID: \"c4731dfe-1339-4b79-a3e2-3e904d27f53e\") " pod="openshift-console/console-ff5d4b79d-psddb" Jul 22 03:09:15.737753 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:15.737405 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4731dfe-1339-4b79-a3e2-3e904d27f53e-trusted-ca-bundle\") pod \"console-ff5d4b79d-psddb\" (UID: \"c4731dfe-1339-4b79-a3e2-3e904d27f53e\") " pod="openshift-console/console-ff5d4b79d-psddb" Jul 22 03:09:15.737753 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:15.737424 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fs94\" (UniqueName: \"kubernetes.io/projected/c4731dfe-1339-4b79-a3e2-3e904d27f53e-kube-api-access-2fs94\") pod \"console-ff5d4b79d-psddb\" (UID: \"c4731dfe-1339-4b79-a3e2-3e904d27f53e\") " pod="openshift-console/console-ff5d4b79d-psddb" Jul 22 03:09:15.737753 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:15.737444 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4731dfe-1339-4b79-a3e2-3e904d27f53e-console-oauth-config\") pod \"console-ff5d4b79d-psddb\" (UID: \"c4731dfe-1339-4b79-a3e2-3e904d27f53e\") " pod="openshift-console/console-ff5d4b79d-psddb" Jul 22 03:09:15.738215 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:15.738186 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4731dfe-1339-4b79-a3e2-3e904d27f53e-service-ca\") pod \"console-ff5d4b79d-psddb\" (UID: \"c4731dfe-1339-4b79-a3e2-3e904d27f53e\") " pod="openshift-console/console-ff5d4b79d-psddb" Jul 22 03:09:15.738284 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:15.738218 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4731dfe-1339-4b79-a3e2-3e904d27f53e-console-config\") pod \"console-ff5d4b79d-psddb\" (UID: \"c4731dfe-1339-4b79-a3e2-3e904d27f53e\") " pod="openshift-console/console-ff5d4b79d-psddb" Jul 22 03:09:15.738284 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:15.738190 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4731dfe-1339-4b79-a3e2-3e904d27f53e-oauth-serving-cert\") pod \"console-ff5d4b79d-psddb\" (UID: \"c4731dfe-1339-4b79-a3e2-3e904d27f53e\") " pod="openshift-console/console-ff5d4b79d-psddb" Jul 22 03:09:15.738423 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:15.738398 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4731dfe-1339-4b79-a3e2-3e904d27f53e-trusted-ca-bundle\") pod \"console-ff5d4b79d-psddb\" (UID: \"c4731dfe-1339-4b79-a3e2-3e904d27f53e\") " pod="openshift-console/console-ff5d4b79d-psddb" Jul 22 03:09:15.739869 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:15.739840 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4731dfe-1339-4b79-a3e2-3e904d27f53e-console-oauth-config\") pod \"console-ff5d4b79d-psddb\" (UID: \"c4731dfe-1339-4b79-a3e2-3e904d27f53e\") " pod="openshift-console/console-ff5d4b79d-psddb" Jul 22 03:09:15.739869 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:15.739804 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4731dfe-1339-4b79-a3e2-3e904d27f53e-console-serving-cert\") pod \"console-ff5d4b79d-psddb\" (UID: \"c4731dfe-1339-4b79-a3e2-3e904d27f53e\") " pod="openshift-console/console-ff5d4b79d-psddb" Jul 22 03:09:15.754117 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:15.754095 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fs94\" (UniqueName: \"kubernetes.io/projected/c4731dfe-1339-4b79-a3e2-3e904d27f53e-kube-api-access-2fs94\") pod \"console-ff5d4b79d-psddb\" (UID: \"c4731dfe-1339-4b79-a3e2-3e904d27f53e\") " pod="openshift-console/console-ff5d4b79d-psddb" Jul 22 03:09:15.837859 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:15.837821 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-ff5d4b79d-psddb" Jul 22 03:09:16.000883 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:16.000854 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-ff5d4b79d-psddb"] Jul 22 03:09:16.003272 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:09:16.003244 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4731dfe_1339_4b79_a3e2_3e904d27f53e.slice/crio-42640f4a84e8604483b6d3eec952cb0261a8aa8c8ca7c69e00fc591641207496 WatchSource:0}: Error finding container 42640f4a84e8604483b6d3eec952cb0261a8aa8c8ca7c69e00fc591641207496: Status 404 returned error can't find the container with id 42640f4a84e8604483b6d3eec952cb0261a8aa8c8ca7c69e00fc591641207496 Jul 22 03:09:16.476331 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:16.476281 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-ff5d4b79d-psddb" event={"ID":"c4731dfe-1339-4b79-a3e2-3e904d27f53e","Type":"ContainerStarted","Data":"a3706900e1ddc48e5ded94f82bb5b1ff8786e0092c2da6e48af4a6d5f5cffc33"} Jul 22 03:09:16.476502 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:16.476344 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-ff5d4b79d-psddb" event={"ID":"c4731dfe-1339-4b79-a3e2-3e904d27f53e","Type":"ContainerStarted","Data":"42640f4a84e8604483b6d3eec952cb0261a8aa8c8ca7c69e00fc591641207496"} Jul 22 03:09:16.498915 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:16.498870 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-ff5d4b79d-psddb" podStartSLOduration=1.498850803 podStartE2EDuration="1.498850803s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:09:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:09:16.497698712 +0000 UTC m=+96.946681958" watchObservedRunningTime="2026-07-22 03:09:16.498850803 +0000 UTC m=+96.947834052" Jul 22 03:09:16.887841 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:16.887806 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:16.993466 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:16.993425 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:21.102281 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:21.102246 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:21.172626 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:21.172596 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:21.413572 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:21.413482 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7b77548757-tj9kq" Jul 22 03:09:21.413572 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:21.413563 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7b77548757-tj9kq" Jul 22 03:09:21.418331 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:21.418311 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7b77548757-tj9kq" Jul 22 03:09:21.498745 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:21.498719 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7b77548757-tj9kq" Jul 22 03:09:21.865162 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:21.865129 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:21.965277 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:21.965248 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:23.783922 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:23.783875 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:23.920756 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:23.920721 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:23.996217 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:23.996181 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:25.364648 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:25.364614 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-k4rxt" Jul 22 03:09:25.838526 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:25.838493 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-ff5d4b79d-psddb" Jul 22 03:09:25.838687 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:25.838553 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-ff5d4b79d-psddb" Jul 22 03:09:25.843321 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:25.843296 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-ff5d4b79d-psddb" Jul 22 03:09:26.517825 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:26.517799 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-ff5d4b79d-psddb" Jul 22 03:09:26.572285 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:26.572250 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b77548757-tj9kq"] Jul 22 03:09:27.896196 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:27.896154 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:27.997380 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:27.997344 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:29.295888 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:29.295850 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:29.501281 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:29.501245 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:30.722373 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:30.722339 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:30.806239 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:30.806202 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:31.241600 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:31.241569 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:31.338730 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:31.338693 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:41.733764 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:41.733724 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:41.836081 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:41.836038 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:47.511012 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:47.510970 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:47.577870 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:47.577838 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:09:51.592914 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:51.592868 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7b77548757-tj9kq" podUID="fec7fc17-a2f1-4731-873f-73b5aba8d874" containerName="console" containerID="cri-o://638501eaea19ea8b7334774e995aa0e8909a6f1fb5ef9560ad3ae8795d977122" gracePeriod=15 Jul 22 03:09:51.839132 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:51.839104 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b77548757-tj9kq" Jul 22 03:09:51.926419 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:51.926341 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fec7fc17-a2f1-4731-873f-73b5aba8d874-console-oauth-config\") pod \"fec7fc17-a2f1-4731-873f-73b5aba8d874\" (UID: \"fec7fc17-a2f1-4731-873f-73b5aba8d874\") " Jul 22 03:09:51.926419 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:51.926376 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g66bk\" (UniqueName: \"kubernetes.io/projected/fec7fc17-a2f1-4731-873f-73b5aba8d874-kube-api-access-g66bk\") pod \"fec7fc17-a2f1-4731-873f-73b5aba8d874\" (UID: \"fec7fc17-a2f1-4731-873f-73b5aba8d874\") " Jul 22 03:09:51.926419 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:51.926396 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fec7fc17-a2f1-4731-873f-73b5aba8d874-console-config\") pod \"fec7fc17-a2f1-4731-873f-73b5aba8d874\" (UID: \"fec7fc17-a2f1-4731-873f-73b5aba8d874\") " Jul 22 03:09:51.926802 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:51.926428 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fec7fc17-a2f1-4731-873f-73b5aba8d874-oauth-serving-cert\") pod \"fec7fc17-a2f1-4731-873f-73b5aba8d874\" (UID: \"fec7fc17-a2f1-4731-873f-73b5aba8d874\") " Jul 22 03:09:51.926802 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:51.926451 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fec7fc17-a2f1-4731-873f-73b5aba8d874-service-ca\") pod \"fec7fc17-a2f1-4731-873f-73b5aba8d874\" (UID: \"fec7fc17-a2f1-4731-873f-73b5aba8d874\") " Jul 22 03:09:51.926802 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:51.926490 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fec7fc17-a2f1-4731-873f-73b5aba8d874-console-serving-cert\") pod \"fec7fc17-a2f1-4731-873f-73b5aba8d874\" (UID: \"fec7fc17-a2f1-4731-873f-73b5aba8d874\") " Jul 22 03:09:51.926920 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:51.926794 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fec7fc17-a2f1-4731-873f-73b5aba8d874-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fec7fc17-a2f1-4731-873f-73b5aba8d874" (UID: "fec7fc17-a2f1-4731-873f-73b5aba8d874"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:09:51.926920 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:51.926801 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fec7fc17-a2f1-4731-873f-73b5aba8d874-console-config" (OuterVolumeSpecName: "console-config") pod "fec7fc17-a2f1-4731-873f-73b5aba8d874" (UID: "fec7fc17-a2f1-4731-873f-73b5aba8d874"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:09:51.926920 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:51.926861 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fec7fc17-a2f1-4731-873f-73b5aba8d874-service-ca" (OuterVolumeSpecName: "service-ca") pod "fec7fc17-a2f1-4731-873f-73b5aba8d874" (UID: "fec7fc17-a2f1-4731-873f-73b5aba8d874"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:09:51.928735 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:51.928696 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fec7fc17-a2f1-4731-873f-73b5aba8d874-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fec7fc17-a2f1-4731-873f-73b5aba8d874" (UID: "fec7fc17-a2f1-4731-873f-73b5aba8d874"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:09:51.928837 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:51.928783 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fec7fc17-a2f1-4731-873f-73b5aba8d874-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fec7fc17-a2f1-4731-873f-73b5aba8d874" (UID: "fec7fc17-a2f1-4731-873f-73b5aba8d874"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:09:51.928837 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:51.928791 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fec7fc17-a2f1-4731-873f-73b5aba8d874-kube-api-access-g66bk" (OuterVolumeSpecName: "kube-api-access-g66bk") pod "fec7fc17-a2f1-4731-873f-73b5aba8d874" (UID: "fec7fc17-a2f1-4731-873f-73b5aba8d874"). InnerVolumeSpecName "kube-api-access-g66bk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:09:52.029879 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:52.029587 2556 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fec7fc17-a2f1-4731-873f-73b5aba8d874-oauth-serving-cert\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:09:52.030059 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:52.029887 2556 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fec7fc17-a2f1-4731-873f-73b5aba8d874-service-ca\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:09:52.030059 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:52.029900 2556 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fec7fc17-a2f1-4731-873f-73b5aba8d874-console-serving-cert\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:09:52.030059 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:52.029909 2556 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fec7fc17-a2f1-4731-873f-73b5aba8d874-console-oauth-config\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:09:52.030059 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:52.029918 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g66bk\" (UniqueName: \"kubernetes.io/projected/fec7fc17-a2f1-4731-873f-73b5aba8d874-kube-api-access-g66bk\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:09:52.030059 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:52.029927 2556 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fec7fc17-a2f1-4731-873f-73b5aba8d874-console-config\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:09:52.607718 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:52.607680 2556 generic.go:356] "Generic (PLEG): container finished" podID="fec7fc17-a2f1-4731-873f-73b5aba8d874" containerID="638501eaea19ea8b7334774e995aa0e8909a6f1fb5ef9560ad3ae8795d977122" exitCode=0 Jul 22 03:09:52.608221 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:52.607742 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b77548757-tj9kq" Jul 22 03:09:52.608221 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:52.607768 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b77548757-tj9kq" event={"ID":"fec7fc17-a2f1-4731-873f-73b5aba8d874","Type":"ContainerDied","Data":"638501eaea19ea8b7334774e995aa0e8909a6f1fb5ef9560ad3ae8795d977122"} Jul 22 03:09:52.608221 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:52.607850 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b77548757-tj9kq" event={"ID":"fec7fc17-a2f1-4731-873f-73b5aba8d874","Type":"ContainerDied","Data":"91be103cc1b7804959bc0aaea0f9b0b10d2c0d5d714b26f1044b1ea3ed96eba9"} Jul 22 03:09:52.608221 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:52.607876 2556 scope.go:117] "RemoveContainer" containerID="638501eaea19ea8b7334774e995aa0e8909a6f1fb5ef9560ad3ae8795d977122" Jul 22 03:09:52.615827 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:52.615809 2556 scope.go:117] "RemoveContainer" containerID="638501eaea19ea8b7334774e995aa0e8909a6f1fb5ef9560ad3ae8795d977122" Jul 22 03:09:52.616084 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:09:52.616066 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"638501eaea19ea8b7334774e995aa0e8909a6f1fb5ef9560ad3ae8795d977122\": container with ID starting with 638501eaea19ea8b7334774e995aa0e8909a6f1fb5ef9560ad3ae8795d977122 not found: ID does not exist" containerID="638501eaea19ea8b7334774e995aa0e8909a6f1fb5ef9560ad3ae8795d977122" Jul 22 03:09:52.616131 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:52.616093 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"638501eaea19ea8b7334774e995aa0e8909a6f1fb5ef9560ad3ae8795d977122"} err="failed to get container status \"638501eaea19ea8b7334774e995aa0e8909a6f1fb5ef9560ad3ae8795d977122\": rpc error: code = NotFound desc = could not find container \"638501eaea19ea8b7334774e995aa0e8909a6f1fb5ef9560ad3ae8795d977122\": container with ID starting with 638501eaea19ea8b7334774e995aa0e8909a6f1fb5ef9560ad3ae8795d977122 not found: ID does not exist" Jul 22 03:09:52.623235 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:52.623214 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b77548757-tj9kq"] Jul 22 03:09:52.629561 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:52.629541 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7b77548757-tj9kq"] Jul 22 03:09:54.022635 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:09:54.022603 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fec7fc17-a2f1-4731-873f-73b5aba8d874" path="/var/lib/kubelet/pods/fec7fc17-a2f1-4731-873f-73b5aba8d874/volumes" Jul 22 03:10:15.476039 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:15.475984 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:10:15.799148 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:15.799048 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:10:24.753114 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:24.753078 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b479d4584-tgg9g"] Jul 22 03:10:24.753569 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:24.753337 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fec7fc17-a2f1-4731-873f-73b5aba8d874" containerName="console" Jul 22 03:10:24.753569 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:24.753349 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="fec7fc17-a2f1-4731-873f-73b5aba8d874" containerName="console" Jul 22 03:10:24.753569 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:24.753402 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="fec7fc17-a2f1-4731-873f-73b5aba8d874" containerName="console" Jul 22 03:10:24.758264 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:24.758236 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b479d4584-tgg9g" Jul 22 03:10:24.770361 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:24.770335 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b479d4584-tgg9g"] Jul 22 03:10:24.817632 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:24.817591 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:10:24.884864 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:24.884831 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae275278-912c-454d-8b18-9bc619b8e3c1-trusted-ca-bundle\") pod \"console-6b479d4584-tgg9g\" (UID: \"ae275278-912c-454d-8b18-9bc619b8e3c1\") " pod="openshift-console/console-6b479d4584-tgg9g" Jul 22 03:10:24.884864 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:24.884865 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ae275278-912c-454d-8b18-9bc619b8e3c1-console-serving-cert\") pod \"console-6b479d4584-tgg9g\" (UID: \"ae275278-912c-454d-8b18-9bc619b8e3c1\") " pod="openshift-console/console-6b479d4584-tgg9g" Jul 22 03:10:24.885086 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:24.884893 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ae275278-912c-454d-8b18-9bc619b8e3c1-service-ca\") pod \"console-6b479d4584-tgg9g\" (UID: \"ae275278-912c-454d-8b18-9bc619b8e3c1\") " pod="openshift-console/console-6b479d4584-tgg9g" Jul 22 03:10:24.885086 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:24.884990 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qp6c\" (UniqueName: \"kubernetes.io/projected/ae275278-912c-454d-8b18-9bc619b8e3c1-kube-api-access-7qp6c\") pod \"console-6b479d4584-tgg9g\" (UID: \"ae275278-912c-454d-8b18-9bc619b8e3c1\") " pod="openshift-console/console-6b479d4584-tgg9g" Jul 22 03:10:24.885086 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:24.885025 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ae275278-912c-454d-8b18-9bc619b8e3c1-console-oauth-config\") pod \"console-6b479d4584-tgg9g\" (UID: \"ae275278-912c-454d-8b18-9bc619b8e3c1\") " pod="openshift-console/console-6b479d4584-tgg9g" Jul 22 03:10:24.885086 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:24.885083 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ae275278-912c-454d-8b18-9bc619b8e3c1-oauth-serving-cert\") pod \"console-6b479d4584-tgg9g\" (UID: \"ae275278-912c-454d-8b18-9bc619b8e3c1\") " pod="openshift-console/console-6b479d4584-tgg9g" Jul 22 03:10:24.885211 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:24.885099 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ae275278-912c-454d-8b18-9bc619b8e3c1-console-config\") pod \"console-6b479d4584-tgg9g\" (UID: \"ae275278-912c-454d-8b18-9bc619b8e3c1\") " pod="openshift-console/console-6b479d4584-tgg9g" Jul 22 03:10:24.918112 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:24.918073 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:10:24.971527 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:24.971496 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:10:24.985506 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:24.985467 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ae275278-912c-454d-8b18-9bc619b8e3c1-service-ca\") pod \"console-6b479d4584-tgg9g\" (UID: \"ae275278-912c-454d-8b18-9bc619b8e3c1\") " pod="openshift-console/console-6b479d4584-tgg9g" Jul 22 03:10:24.985693 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:24.985528 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qp6c\" (UniqueName: \"kubernetes.io/projected/ae275278-912c-454d-8b18-9bc619b8e3c1-kube-api-access-7qp6c\") pod \"console-6b479d4584-tgg9g\" (UID: \"ae275278-912c-454d-8b18-9bc619b8e3c1\") " pod="openshift-console/console-6b479d4584-tgg9g" Jul 22 03:10:24.985693 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:24.985560 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ae275278-912c-454d-8b18-9bc619b8e3c1-console-oauth-config\") pod \"console-6b479d4584-tgg9g\" (UID: \"ae275278-912c-454d-8b18-9bc619b8e3c1\") " pod="openshift-console/console-6b479d4584-tgg9g" Jul 22 03:10:24.985693 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:24.985621 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ae275278-912c-454d-8b18-9bc619b8e3c1-oauth-serving-cert\") pod \"console-6b479d4584-tgg9g\" (UID: \"ae275278-912c-454d-8b18-9bc619b8e3c1\") " pod="openshift-console/console-6b479d4584-tgg9g" Jul 22 03:10:24.985693 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:24.985646 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ae275278-912c-454d-8b18-9bc619b8e3c1-console-config\") pod \"console-6b479d4584-tgg9g\" (UID: \"ae275278-912c-454d-8b18-9bc619b8e3c1\") " pod="openshift-console/console-6b479d4584-tgg9g" Jul 22 03:10:24.985693 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:24.985682 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae275278-912c-454d-8b18-9bc619b8e3c1-trusted-ca-bundle\") pod \"console-6b479d4584-tgg9g\" (UID: \"ae275278-912c-454d-8b18-9bc619b8e3c1\") " pod="openshift-console/console-6b479d4584-tgg9g" Jul 22 03:10:24.985980 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:24.985709 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ae275278-912c-454d-8b18-9bc619b8e3c1-console-serving-cert\") pod \"console-6b479d4584-tgg9g\" (UID: \"ae275278-912c-454d-8b18-9bc619b8e3c1\") " pod="openshift-console/console-6b479d4584-tgg9g" Jul 22 03:10:24.986969 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:24.986858 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ae275278-912c-454d-8b18-9bc619b8e3c1-console-config\") pod \"console-6b479d4584-tgg9g\" (UID: \"ae275278-912c-454d-8b18-9bc619b8e3c1\") " pod="openshift-console/console-6b479d4584-tgg9g" Jul 22 03:10:24.987525 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:24.987503 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ae275278-912c-454d-8b18-9bc619b8e3c1-oauth-serving-cert\") pod \"console-6b479d4584-tgg9g\" (UID: \"ae275278-912c-454d-8b18-9bc619b8e3c1\") " pod="openshift-console/console-6b479d4584-tgg9g" Jul 22 03:10:24.987774 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:24.987749 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ae275278-912c-454d-8b18-9bc619b8e3c1-service-ca\") pod \"console-6b479d4584-tgg9g\" (UID: \"ae275278-912c-454d-8b18-9bc619b8e3c1\") " pod="openshift-console/console-6b479d4584-tgg9g" Jul 22 03:10:24.988186 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:24.988162 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae275278-912c-454d-8b18-9bc619b8e3c1-trusted-ca-bundle\") pod \"console-6b479d4584-tgg9g\" (UID: \"ae275278-912c-454d-8b18-9bc619b8e3c1\") " pod="openshift-console/console-6b479d4584-tgg9g" Jul 22 03:10:24.988917 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:24.988898 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ae275278-912c-454d-8b18-9bc619b8e3c1-console-oauth-config\") pod \"console-6b479d4584-tgg9g\" (UID: \"ae275278-912c-454d-8b18-9bc619b8e3c1\") " pod="openshift-console/console-6b479d4584-tgg9g" Jul 22 03:10:24.989138 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:24.989116 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ae275278-912c-454d-8b18-9bc619b8e3c1-console-serving-cert\") pod \"console-6b479d4584-tgg9g\" (UID: \"ae275278-912c-454d-8b18-9bc619b8e3c1\") " pod="openshift-console/console-6b479d4584-tgg9g" Jul 22 03:10:24.995683 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:24.995656 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qp6c\" (UniqueName: \"kubernetes.io/projected/ae275278-912c-454d-8b18-9bc619b8e3c1-kube-api-access-7qp6c\") pod \"console-6b479d4584-tgg9g\" (UID: \"ae275278-912c-454d-8b18-9bc619b8e3c1\") " pod="openshift-console/console-6b479d4584-tgg9g" Jul 22 03:10:25.068483 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:25.068445 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b479d4584-tgg9g" Jul 22 03:10:25.195816 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:25.195784 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b479d4584-tgg9g"] Jul 22 03:10:25.199164 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:10:25.199131 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae275278_912c_454d_8b18_9bc619b8e3c1.slice/crio-b917ce014968cbdd90fca121da4b49c1a072829437d3bba84c371cd953b5a36a WatchSource:0}: Error finding container b917ce014968cbdd90fca121da4b49c1a072829437d3bba84c371cd953b5a36a: Status 404 returned error can't find the container with id b917ce014968cbdd90fca121da4b49c1a072829437d3bba84c371cd953b5a36a Jul 22 03:10:25.574932 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:25.574884 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:10:25.675832 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:25.675798 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:10:25.725275 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:25.725245 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b479d4584-tgg9g" event={"ID":"ae275278-912c-454d-8b18-9bc619b8e3c1","Type":"ContainerStarted","Data":"e0c1164b4114022fd0d9e1142c30af23369da80c2dfdf955c7e8f38ee7ae4fa8"} Jul 22 03:10:25.725488 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:25.725300 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b479d4584-tgg9g" event={"ID":"ae275278-912c-454d-8b18-9bc619b8e3c1","Type":"ContainerStarted","Data":"b917ce014968cbdd90fca121da4b49c1a072829437d3bba84c371cd953b5a36a"} Jul 22 03:10:25.745637 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:25.745590 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b479d4584-tgg9g" podStartSLOduration=1.74557495 podStartE2EDuration="1.74557495s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:10:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:10:25.744005624 +0000 UTC m=+166.192988873" watchObservedRunningTime="2026-07-22 03:10:25.74557495 +0000 UTC m=+166.194558198" Jul 22 03:10:35.069179 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:35.069082 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b479d4584-tgg9g" Jul 22 03:10:35.069179 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:35.069149 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b479d4584-tgg9g" Jul 22 03:10:35.073675 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:35.073651 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b479d4584-tgg9g" Jul 22 03:10:35.762895 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:35.762864 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b479d4584-tgg9g" Jul 22 03:10:35.803554 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:35.803521 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-ff5d4b79d-psddb"] Jul 22 03:10:35.909860 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:35.909825 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:10:36.009915 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:36.009877 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:10:36.111313 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:10:36.111276 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:11:00.823825 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:00.823778 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-ff5d4b79d-psddb" podUID="c4731dfe-1339-4b79-a3e2-3e904d27f53e" containerName="console" containerID="cri-o://a3706900e1ddc48e5ded94f82bb5b1ff8786e0092c2da6e48af4a6d5f5cffc33" gracePeriod=15 Jul 22 03:11:01.058027 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:01.058004 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-ff5d4b79d-psddb" Jul 22 03:11:01.166448 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:01.166348 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4731dfe-1339-4b79-a3e2-3e904d27f53e-console-oauth-config\") pod \"c4731dfe-1339-4b79-a3e2-3e904d27f53e\" (UID: \"c4731dfe-1339-4b79-a3e2-3e904d27f53e\") " Jul 22 03:11:01.166448 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:01.166409 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4731dfe-1339-4b79-a3e2-3e904d27f53e-trusted-ca-bundle\") pod \"c4731dfe-1339-4b79-a3e2-3e904d27f53e\" (UID: \"c4731dfe-1339-4b79-a3e2-3e904d27f53e\") " Jul 22 03:11:01.166671 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:01.166458 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2fs94\" (UniqueName: \"kubernetes.io/projected/c4731dfe-1339-4b79-a3e2-3e904d27f53e-kube-api-access-2fs94\") pod \"c4731dfe-1339-4b79-a3e2-3e904d27f53e\" (UID: \"c4731dfe-1339-4b79-a3e2-3e904d27f53e\") " Jul 22 03:11:01.166671 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:01.166487 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4731dfe-1339-4b79-a3e2-3e904d27f53e-service-ca\") pod \"c4731dfe-1339-4b79-a3e2-3e904d27f53e\" (UID: \"c4731dfe-1339-4b79-a3e2-3e904d27f53e\") " Jul 22 03:11:01.166671 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:01.166512 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4731dfe-1339-4b79-a3e2-3e904d27f53e-oauth-serving-cert\") pod \"c4731dfe-1339-4b79-a3e2-3e904d27f53e\" (UID: \"c4731dfe-1339-4b79-a3e2-3e904d27f53e\") " Jul 22 03:11:01.166671 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:01.166550 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4731dfe-1339-4b79-a3e2-3e904d27f53e-console-serving-cert\") pod \"c4731dfe-1339-4b79-a3e2-3e904d27f53e\" (UID: \"c4731dfe-1339-4b79-a3e2-3e904d27f53e\") " Jul 22 03:11:01.166671 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:01.166600 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4731dfe-1339-4b79-a3e2-3e904d27f53e-console-config\") pod \"c4731dfe-1339-4b79-a3e2-3e904d27f53e\" (UID: \"c4731dfe-1339-4b79-a3e2-3e904d27f53e\") " Jul 22 03:11:01.167033 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:01.166989 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4731dfe-1339-4b79-a3e2-3e904d27f53e-service-ca" (OuterVolumeSpecName: "service-ca") pod "c4731dfe-1339-4b79-a3e2-3e904d27f53e" (UID: "c4731dfe-1339-4b79-a3e2-3e904d27f53e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:11:01.167172 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:01.167069 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4731dfe-1339-4b79-a3e2-3e904d27f53e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c4731dfe-1339-4b79-a3e2-3e904d27f53e" (UID: "c4731dfe-1339-4b79-a3e2-3e904d27f53e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:11:01.167172 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:01.167129 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4731dfe-1339-4b79-a3e2-3e904d27f53e-console-config" (OuterVolumeSpecName: "console-config") pod "c4731dfe-1339-4b79-a3e2-3e904d27f53e" (UID: "c4731dfe-1339-4b79-a3e2-3e904d27f53e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:11:01.167172 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:01.167160 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4731dfe-1339-4b79-a3e2-3e904d27f53e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c4731dfe-1339-4b79-a3e2-3e904d27f53e" (UID: "c4731dfe-1339-4b79-a3e2-3e904d27f53e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:11:01.168498 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:01.168469 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4731dfe-1339-4b79-a3e2-3e904d27f53e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c4731dfe-1339-4b79-a3e2-3e904d27f53e" (UID: "c4731dfe-1339-4b79-a3e2-3e904d27f53e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:11:01.168591 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:01.168546 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4731dfe-1339-4b79-a3e2-3e904d27f53e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c4731dfe-1339-4b79-a3e2-3e904d27f53e" (UID: "c4731dfe-1339-4b79-a3e2-3e904d27f53e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:11:01.168634 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:01.168601 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4731dfe-1339-4b79-a3e2-3e904d27f53e-kube-api-access-2fs94" (OuterVolumeSpecName: "kube-api-access-2fs94") pod "c4731dfe-1339-4b79-a3e2-3e904d27f53e" (UID: "c4731dfe-1339-4b79-a3e2-3e904d27f53e"). InnerVolumeSpecName "kube-api-access-2fs94". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:11:01.267362 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:01.267329 2556 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4731dfe-1339-4b79-a3e2-3e904d27f53e-console-config\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:11:01.267362 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:01.267355 2556 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4731dfe-1339-4b79-a3e2-3e904d27f53e-console-oauth-config\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:11:01.267362 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:01.267365 2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4731dfe-1339-4b79-a3e2-3e904d27f53e-trusted-ca-bundle\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:11:01.267576 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:01.267375 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2fs94\" (UniqueName: \"kubernetes.io/projected/c4731dfe-1339-4b79-a3e2-3e904d27f53e-kube-api-access-2fs94\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:11:01.267576 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:01.267385 2556 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4731dfe-1339-4b79-a3e2-3e904d27f53e-service-ca\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:11:01.267576 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:01.267394 2556 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4731dfe-1339-4b79-a3e2-3e904d27f53e-oauth-serving-cert\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:11:01.267576 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:01.267402 2556 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4731dfe-1339-4b79-a3e2-3e904d27f53e-console-serving-cert\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:11:01.855804 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:01.855769 2556 generic.go:356] "Generic (PLEG): container finished" podID="c4731dfe-1339-4b79-a3e2-3e904d27f53e" containerID="a3706900e1ddc48e5ded94f82bb5b1ff8786e0092c2da6e48af4a6d5f5cffc33" exitCode=0 Jul 22 03:11:01.856213 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:01.855818 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-ff5d4b79d-psddb" Jul 22 03:11:01.856213 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:01.855860 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-ff5d4b79d-psddb" event={"ID":"c4731dfe-1339-4b79-a3e2-3e904d27f53e","Type":"ContainerDied","Data":"a3706900e1ddc48e5ded94f82bb5b1ff8786e0092c2da6e48af4a6d5f5cffc33"} Jul 22 03:11:01.856213 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:01.855911 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-ff5d4b79d-psddb" event={"ID":"c4731dfe-1339-4b79-a3e2-3e904d27f53e","Type":"ContainerDied","Data":"42640f4a84e8604483b6d3eec952cb0261a8aa8c8ca7c69e00fc591641207496"} Jul 22 03:11:01.856213 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:01.855933 2556 scope.go:117] "RemoveContainer" containerID="a3706900e1ddc48e5ded94f82bb5b1ff8786e0092c2da6e48af4a6d5f5cffc33" Jul 22 03:11:01.864749 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:01.864731 2556 scope.go:117] "RemoveContainer" containerID="a3706900e1ddc48e5ded94f82bb5b1ff8786e0092c2da6e48af4a6d5f5cffc33" Jul 22 03:11:01.865030 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:11:01.865012 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3706900e1ddc48e5ded94f82bb5b1ff8786e0092c2da6e48af4a6d5f5cffc33\": container with ID starting with a3706900e1ddc48e5ded94f82bb5b1ff8786e0092c2da6e48af4a6d5f5cffc33 not found: ID does not exist" containerID="a3706900e1ddc48e5ded94f82bb5b1ff8786e0092c2da6e48af4a6d5f5cffc33" Jul 22 03:11:01.865097 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:01.865044 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3706900e1ddc48e5ded94f82bb5b1ff8786e0092c2da6e48af4a6d5f5cffc33"} err="failed to get container status \"a3706900e1ddc48e5ded94f82bb5b1ff8786e0092c2da6e48af4a6d5f5cffc33\": rpc error: code = NotFound desc = could not find container \"a3706900e1ddc48e5ded94f82bb5b1ff8786e0092c2da6e48af4a6d5f5cffc33\": container with ID starting with a3706900e1ddc48e5ded94f82bb5b1ff8786e0092c2da6e48af4a6d5f5cffc33 not found: ID does not exist" Jul 22 03:11:01.876998 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:01.876975 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-ff5d4b79d-psddb"] Jul 22 03:11:01.881114 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:01.881094 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-ff5d4b79d-psddb"] Jul 22 03:11:02.022917 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:02.022887 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4731dfe-1339-4b79-a3e2-3e904d27f53e" path="/var/lib/kubelet/pods/c4731dfe-1339-4b79-a3e2-3e904d27f53e/volumes" Jul 22 03:11:12.431152 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:12.431118 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:11:17.655075 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:17.655034 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:11:17.709133 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:17.709092 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:11:23.201992 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:23.201936 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:11:30.712712 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:30.712669 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:11:30.812970 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:30.812919 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:11:42.699278 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:42.699236 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:11:42.765753 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:42.765719 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:11:46.230225 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:46.230185 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:11:46.331239 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:46.331210 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:11:58.572018 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:58.571985 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-57fd7bf74d-kpksw"] Jul 22 03:11:58.572383 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:58.572249 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4731dfe-1339-4b79-a3e2-3e904d27f53e" containerName="console" Jul 22 03:11:58.572383 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:58.572260 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4731dfe-1339-4b79-a3e2-3e904d27f53e" containerName="console" Jul 22 03:11:58.572383 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:58.572316 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4731dfe-1339-4b79-a3e2-3e904d27f53e" containerName="console" Jul 22 03:11:58.576328 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:58.576307 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57fd7bf74d-kpksw" Jul 22 03:11:58.602891 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:58.602862 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57fd7bf74d-kpksw"] Jul 22 03:11:58.661760 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:58.661727 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:11:58.686970 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:58.686930 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-service-ca\") pod \"console-57fd7bf74d-kpksw\" (UID: \"d2b3082a-2729-43ca-ba14-8ca41d1c83c1\") " pod="openshift-console/console-57fd7bf74d-kpksw" Jul 22 03:11:58.687113 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:58.686973 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-console-serving-cert\") pod \"console-57fd7bf74d-kpksw\" (UID: \"d2b3082a-2729-43ca-ba14-8ca41d1c83c1\") " pod="openshift-console/console-57fd7bf74d-kpksw" Jul 22 03:11:58.687113 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:58.687009 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-trusted-ca-bundle\") pod \"console-57fd7bf74d-kpksw\" (UID: \"d2b3082a-2729-43ca-ba14-8ca41d1c83c1\") " pod="openshift-console/console-57fd7bf74d-kpksw" Jul 22 03:11:58.687113 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:58.687046 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-console-oauth-config\") pod \"console-57fd7bf74d-kpksw\" (UID: \"d2b3082a-2729-43ca-ba14-8ca41d1c83c1\") " pod="openshift-console/console-57fd7bf74d-kpksw" Jul 22 03:11:58.687113 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:58.687070 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-console-config\") pod \"console-57fd7bf74d-kpksw\" (UID: \"d2b3082a-2729-43ca-ba14-8ca41d1c83c1\") " pod="openshift-console/console-57fd7bf74d-kpksw" Jul 22 03:11:58.687241 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:58.687121 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55rzv\" (UniqueName: \"kubernetes.io/projected/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-kube-api-access-55rzv\") pod \"console-57fd7bf74d-kpksw\" (UID: \"d2b3082a-2729-43ca-ba14-8ca41d1c83c1\") " pod="openshift-console/console-57fd7bf74d-kpksw" Jul 22 03:11:58.687241 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:58.687175 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-oauth-serving-cert\") pod \"console-57fd7bf74d-kpksw\" (UID: \"d2b3082a-2729-43ca-ba14-8ca41d1c83c1\") " pod="openshift-console/console-57fd7bf74d-kpksw" Jul 22 03:11:58.732435 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:58.732400 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:11:58.788276 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:58.788245 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-oauth-serving-cert\") pod \"console-57fd7bf74d-kpksw\" (UID: \"d2b3082a-2729-43ca-ba14-8ca41d1c83c1\") " pod="openshift-console/console-57fd7bf74d-kpksw" Jul 22 03:11:58.788393 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:58.788286 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-service-ca\") pod \"console-57fd7bf74d-kpksw\" (UID: \"d2b3082a-2729-43ca-ba14-8ca41d1c83c1\") " pod="openshift-console/console-57fd7bf74d-kpksw" Jul 22 03:11:58.788393 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:58.788310 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-console-serving-cert\") pod \"console-57fd7bf74d-kpksw\" (UID: \"d2b3082a-2729-43ca-ba14-8ca41d1c83c1\") " pod="openshift-console/console-57fd7bf74d-kpksw" Jul 22 03:11:58.788393 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:58.788343 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-trusted-ca-bundle\") pod \"console-57fd7bf74d-kpksw\" (UID: \"d2b3082a-2729-43ca-ba14-8ca41d1c83c1\") " pod="openshift-console/console-57fd7bf74d-kpksw" Jul 22 03:11:58.788393 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:58.788373 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-console-oauth-config\") pod \"console-57fd7bf74d-kpksw\" (UID: \"d2b3082a-2729-43ca-ba14-8ca41d1c83c1\") " pod="openshift-console/console-57fd7bf74d-kpksw" Jul 22 03:11:58.788607 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:58.788401 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-console-config\") pod \"console-57fd7bf74d-kpksw\" (UID: \"d2b3082a-2729-43ca-ba14-8ca41d1c83c1\") " pod="openshift-console/console-57fd7bf74d-kpksw" Jul 22 03:11:58.788607 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:58.788461 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55rzv\" (UniqueName: \"kubernetes.io/projected/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-kube-api-access-55rzv\") pod \"console-57fd7bf74d-kpksw\" (UID: \"d2b3082a-2729-43ca-ba14-8ca41d1c83c1\") " pod="openshift-console/console-57fd7bf74d-kpksw" Jul 22 03:11:58.789127 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:58.789100 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-service-ca\") pod \"console-57fd7bf74d-kpksw\" (UID: \"d2b3082a-2729-43ca-ba14-8ca41d1c83c1\") " pod="openshift-console/console-57fd7bf74d-kpksw" Jul 22 03:11:58.789258 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:58.789128 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-oauth-serving-cert\") pod \"console-57fd7bf74d-kpksw\" (UID: \"d2b3082a-2729-43ca-ba14-8ca41d1c83c1\") " pod="openshift-console/console-57fd7bf74d-kpksw" Jul 22 03:11:58.789258 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:58.789194 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-console-config\") pod \"console-57fd7bf74d-kpksw\" (UID: \"d2b3082a-2729-43ca-ba14-8ca41d1c83c1\") " pod="openshift-console/console-57fd7bf74d-kpksw" Jul 22 03:11:58.789258 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:58.789251 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-trusted-ca-bundle\") pod \"console-57fd7bf74d-kpksw\" (UID: \"d2b3082a-2729-43ca-ba14-8ca41d1c83c1\") " pod="openshift-console/console-57fd7bf74d-kpksw" Jul 22 03:11:58.790751 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:58.790723 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-console-oauth-config\") pod \"console-57fd7bf74d-kpksw\" (UID: \"d2b3082a-2729-43ca-ba14-8ca41d1c83c1\") " pod="openshift-console/console-57fd7bf74d-kpksw" Jul 22 03:11:58.790940 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:58.790923 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-console-serving-cert\") pod \"console-57fd7bf74d-kpksw\" (UID: \"d2b3082a-2729-43ca-ba14-8ca41d1c83c1\") " pod="openshift-console/console-57fd7bf74d-kpksw" Jul 22 03:11:58.796834 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:58.796806 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55rzv\" (UniqueName: \"kubernetes.io/projected/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-kube-api-access-55rzv\") pod \"console-57fd7bf74d-kpksw\" (UID: \"d2b3082a-2729-43ca-ba14-8ca41d1c83c1\") " pod="openshift-console/console-57fd7bf74d-kpksw" Jul 22 03:11:58.885344 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:58.885264 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57fd7bf74d-kpksw" Jul 22 03:11:59.010864 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:59.010753 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57fd7bf74d-kpksw"] Jul 22 03:11:59.013656 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:11:59.013630 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2b3082a_2729_43ca_ba14_8ca41d1c83c1.slice/crio-d3fe2e596a214ae5d070580094578d5f5779156f6c1c1e2c7555dfd639b6d1ed WatchSource:0}: Error finding container d3fe2e596a214ae5d070580094578d5f5779156f6c1c1e2c7555dfd639b6d1ed: Status 404 returned error can't find the container with id d3fe2e596a214ae5d070580094578d5f5779156f6c1c1e2c7555dfd639b6d1ed Jul 22 03:11:59.053634 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:11:59.053609 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57fd7bf74d-kpksw" event={"ID":"d2b3082a-2729-43ca-ba14-8ca41d1c83c1","Type":"ContainerStarted","Data":"d3fe2e596a214ae5d070580094578d5f5779156f6c1c1e2c7555dfd639b6d1ed"} Jul 22 03:12:00.058857 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:00.058776 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57fd7bf74d-kpksw" event={"ID":"d2b3082a-2729-43ca-ba14-8ca41d1c83c1","Type":"ContainerStarted","Data":"0a0ccdde824b44ff873d7bd84aac660813ec9532caee83ef1e7e03d29fbbad27"} Jul 22 03:12:00.076715 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:00.076663 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-57fd7bf74d-kpksw" podStartSLOduration=2.076647499 podStartE2EDuration="2.076647499s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:11:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:12:00.0760096 +0000 UTC m=+260.524992848" watchObservedRunningTime="2026-07-22 03:12:00.076647499 +0000 UTC m=+260.525630748" Jul 22 03:12:08.885722 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:08.885682 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-57fd7bf74d-kpksw" Jul 22 03:12:08.886120 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:08.885748 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-57fd7bf74d-kpksw" Jul 22 03:12:08.890402 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:08.890378 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-57fd7bf74d-kpksw" Jul 22 03:12:09.094926 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:09.094900 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-57fd7bf74d-kpksw" Jul 22 03:12:09.143080 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:09.142996 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b479d4584-tgg9g"] Jul 22 03:12:09.246554 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:09.246522 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:12:09.306965 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:09.306917 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:12:09.361697 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:09.361662 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:12:09.732078 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:09.732043 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:12:09.824221 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:09.824182 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:12:10.384587 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:10.384554 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l2wz5"] Jul 22 03:12:10.388011 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:10.387993 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l2wz5" Jul 22 03:12:10.390601 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:10.390583 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 22 03:12:10.390692 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:10.390627 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 22 03:12:10.391639 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:10.391625 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kxg97\"" Jul 22 03:12:10.397394 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:10.397373 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l2wz5"] Jul 22 03:12:10.476920 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:10.476882 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0adeb483-f50d-49a2-8532-dfc2ece0d4f2-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l2wz5\" (UID: \"0adeb483-f50d-49a2-8532-dfc2ece0d4f2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l2wz5" Jul 22 03:12:10.477140 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:10.476994 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0adeb483-f50d-49a2-8532-dfc2ece0d4f2-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l2wz5\" (UID: \"0adeb483-f50d-49a2-8532-dfc2ece0d4f2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l2wz5" Jul 22 03:12:10.477140 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:10.477017 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bw4x5\" (UniqueName: \"kubernetes.io/projected/0adeb483-f50d-49a2-8532-dfc2ece0d4f2-kube-api-access-bw4x5\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l2wz5\" (UID: \"0adeb483-f50d-49a2-8532-dfc2ece0d4f2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l2wz5" Jul 22 03:12:10.577389 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:10.577357 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0adeb483-f50d-49a2-8532-dfc2ece0d4f2-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l2wz5\" (UID: \"0adeb483-f50d-49a2-8532-dfc2ece0d4f2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l2wz5" Jul 22 03:12:10.577389 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:10.577392 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bw4x5\" (UniqueName: \"kubernetes.io/projected/0adeb483-f50d-49a2-8532-dfc2ece0d4f2-kube-api-access-bw4x5\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l2wz5\" (UID: \"0adeb483-f50d-49a2-8532-dfc2ece0d4f2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l2wz5" Jul 22 03:12:10.577608 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:10.577422 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0adeb483-f50d-49a2-8532-dfc2ece0d4f2-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l2wz5\" (UID: \"0adeb483-f50d-49a2-8532-dfc2ece0d4f2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l2wz5" Jul 22 03:12:10.577752 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:10.577735 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0adeb483-f50d-49a2-8532-dfc2ece0d4f2-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l2wz5\" (UID: \"0adeb483-f50d-49a2-8532-dfc2ece0d4f2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l2wz5" Jul 22 03:12:10.577788 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:10.577758 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0adeb483-f50d-49a2-8532-dfc2ece0d4f2-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l2wz5\" (UID: \"0adeb483-f50d-49a2-8532-dfc2ece0d4f2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l2wz5" Jul 22 03:12:10.586035 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:10.586004 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bw4x5\" (UniqueName: \"kubernetes.io/projected/0adeb483-f50d-49a2-8532-dfc2ece0d4f2-kube-api-access-bw4x5\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l2wz5\" (UID: \"0adeb483-f50d-49a2-8532-dfc2ece0d4f2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l2wz5" Jul 22 03:12:10.697725 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:10.697631 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l2wz5" Jul 22 03:12:10.822374 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:10.822344 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l2wz5"] Jul 22 03:12:10.825329 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:12:10.825303 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0adeb483_f50d_49a2_8532_dfc2ece0d4f2.slice/crio-c12391fd8983955efb9b80a436e8e9b9451bf1caf536ee6fd03dc7d09d377d8e WatchSource:0}: Error finding container c12391fd8983955efb9b80a436e8e9b9451bf1caf536ee6fd03dc7d09d377d8e: Status 404 returned error can't find the container with id c12391fd8983955efb9b80a436e8e9b9451bf1caf536ee6fd03dc7d09d377d8e Jul 22 03:12:11.098775 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:11.098739 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l2wz5" event={"ID":"0adeb483-f50d-49a2-8532-dfc2ece0d4f2","Type":"ContainerStarted","Data":"c12391fd8983955efb9b80a436e8e9b9451bf1caf536ee6fd03dc7d09d377d8e"} Jul 22 03:12:18.128574 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:18.128536 2556 generic.go:356] "Generic (PLEG): container finished" podID="0adeb483-f50d-49a2-8532-dfc2ece0d4f2" containerID="a4c5d65c226a38c36ab1f3deb6d2d6bf6153a1c1da60e437d021a50c8f56a600" exitCode=0 Jul 22 03:12:18.129047 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:18.128618 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l2wz5" event={"ID":"0adeb483-f50d-49a2-8532-dfc2ece0d4f2","Type":"ContainerDied","Data":"a4c5d65c226a38c36ab1f3deb6d2d6bf6153a1c1da60e437d021a50c8f56a600"} Jul 22 03:12:21.141797 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:21.141760 2556 generic.go:356] "Generic (PLEG): container finished" podID="0adeb483-f50d-49a2-8532-dfc2ece0d4f2" containerID="f78f54e4c00b4049cbcb7f17e3645ed9a2a68d69456564010013262fb57f649e" exitCode=0 Jul 22 03:12:21.142182 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:21.141842 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l2wz5" event={"ID":"0adeb483-f50d-49a2-8532-dfc2ece0d4f2","Type":"ContainerDied","Data":"f78f54e4c00b4049cbcb7f17e3645ed9a2a68d69456564010013262fb57f649e"} Jul 22 03:12:28.170593 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:28.170557 2556 generic.go:356] "Generic (PLEG): container finished" podID="0adeb483-f50d-49a2-8532-dfc2ece0d4f2" containerID="7202b72aaca9f2a580d39a72653d8d61de7fdbdd173b7405af9e430e507757e0" exitCode=0 Jul 22 03:12:28.170944 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:28.170605 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l2wz5" event={"ID":"0adeb483-f50d-49a2-8532-dfc2ece0d4f2","Type":"ContainerDied","Data":"7202b72aaca9f2a580d39a72653d8d61de7fdbdd173b7405af9e430e507757e0"} Jul 22 03:12:29.299705 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:29.299680 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l2wz5" Jul 22 03:12:29.443520 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:29.443417 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bw4x5\" (UniqueName: \"kubernetes.io/projected/0adeb483-f50d-49a2-8532-dfc2ece0d4f2-kube-api-access-bw4x5\") pod \"0adeb483-f50d-49a2-8532-dfc2ece0d4f2\" (UID: \"0adeb483-f50d-49a2-8532-dfc2ece0d4f2\") " Jul 22 03:12:29.443520 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:29.443494 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0adeb483-f50d-49a2-8532-dfc2ece0d4f2-bundle\") pod \"0adeb483-f50d-49a2-8532-dfc2ece0d4f2\" (UID: \"0adeb483-f50d-49a2-8532-dfc2ece0d4f2\") " Jul 22 03:12:29.443744 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:29.443602 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0adeb483-f50d-49a2-8532-dfc2ece0d4f2-util\") pod \"0adeb483-f50d-49a2-8532-dfc2ece0d4f2\" (UID: \"0adeb483-f50d-49a2-8532-dfc2ece0d4f2\") " Jul 22 03:12:29.444237 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:29.444205 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0adeb483-f50d-49a2-8532-dfc2ece0d4f2-bundle" (OuterVolumeSpecName: "bundle") pod "0adeb483-f50d-49a2-8532-dfc2ece0d4f2" (UID: "0adeb483-f50d-49a2-8532-dfc2ece0d4f2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:12:29.445682 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:29.445646 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0adeb483-f50d-49a2-8532-dfc2ece0d4f2-kube-api-access-bw4x5" (OuterVolumeSpecName: "kube-api-access-bw4x5") pod "0adeb483-f50d-49a2-8532-dfc2ece0d4f2" (UID: "0adeb483-f50d-49a2-8532-dfc2ece0d4f2"). InnerVolumeSpecName "kube-api-access-bw4x5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:12:29.447787 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:29.447764 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0adeb483-f50d-49a2-8532-dfc2ece0d4f2-util" (OuterVolumeSpecName: "util") pod "0adeb483-f50d-49a2-8532-dfc2ece0d4f2" (UID: "0adeb483-f50d-49a2-8532-dfc2ece0d4f2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:12:29.544380 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:29.544331 2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0adeb483-f50d-49a2-8532-dfc2ece0d4f2-util\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:12:29.544380 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:29.544375 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bw4x5\" (UniqueName: \"kubernetes.io/projected/0adeb483-f50d-49a2-8532-dfc2ece0d4f2-kube-api-access-bw4x5\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:12:29.544380 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:29.544386 2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0adeb483-f50d-49a2-8532-dfc2ece0d4f2-bundle\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:12:30.179423 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:30.179393 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l2wz5" Jul 22 03:12:30.179604 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:30.179393 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l2wz5" event={"ID":"0adeb483-f50d-49a2-8532-dfc2ece0d4f2","Type":"ContainerDied","Data":"c12391fd8983955efb9b80a436e8e9b9451bf1caf536ee6fd03dc7d09d377d8e"} Jul 22 03:12:30.179604 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:30.179534 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c12391fd8983955efb9b80a436e8e9b9451bf1caf536ee6fd03dc7d09d377d8e" Jul 22 03:12:31.434145 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:31.434107 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:12:32.548111 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:32.548073 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:12:33.419517 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:33.419485 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4f95p"] Jul 22 03:12:33.419768 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:33.419754 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0adeb483-f50d-49a2-8532-dfc2ece0d4f2" containerName="pull" Jul 22 03:12:33.419810 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:33.419769 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0adeb483-f50d-49a2-8532-dfc2ece0d4f2" containerName="pull" Jul 22 03:12:33.419810 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:33.419787 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0adeb483-f50d-49a2-8532-dfc2ece0d4f2" containerName="extract" Jul 22 03:12:33.419810 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:33.419793 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0adeb483-f50d-49a2-8532-dfc2ece0d4f2" containerName="extract" Jul 22 03:12:33.419810 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:33.419800 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0adeb483-f50d-49a2-8532-dfc2ece0d4f2" containerName="util" Jul 22 03:12:33.419810 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:33.419805 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0adeb483-f50d-49a2-8532-dfc2ece0d4f2" containerName="util" Jul 22 03:12:33.419977 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:33.419855 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="0adeb483-f50d-49a2-8532-dfc2ece0d4f2" containerName="extract" Jul 22 03:12:33.452325 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:33.452300 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4f95p"] Jul 22 03:12:33.452461 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:33.452395 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4f95p" Jul 22 03:12:33.456385 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:33.456362 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 22 03:12:33.456517 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:33.456372 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-8vxtr\"" Jul 22 03:12:33.456707 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:33.456693 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 22 03:12:33.577797 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:33.577763 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnnsx\" (UniqueName: \"kubernetes.io/projected/4453c718-64ae-4d46-b593-fe5c7a9f7247-kube-api-access-qnnsx\") pod \"cert-manager-operator-controller-manager-55c444bf86-4f95p\" (UID: \"4453c718-64ae-4d46-b593-fe5c7a9f7247\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4f95p" Jul 22 03:12:33.578187 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:33.577867 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4453c718-64ae-4d46-b593-fe5c7a9f7247-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4f95p\" (UID: \"4453c718-64ae-4d46-b593-fe5c7a9f7247\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4f95p" Jul 22 03:12:33.679026 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:33.678946 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnnsx\" (UniqueName: \"kubernetes.io/projected/4453c718-64ae-4d46-b593-fe5c7a9f7247-kube-api-access-qnnsx\") pod \"cert-manager-operator-controller-manager-55c444bf86-4f95p\" (UID: \"4453c718-64ae-4d46-b593-fe5c7a9f7247\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4f95p" Jul 22 03:12:33.679141 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:33.679064 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4453c718-64ae-4d46-b593-fe5c7a9f7247-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4f95p\" (UID: \"4453c718-64ae-4d46-b593-fe5c7a9f7247\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4f95p" Jul 22 03:12:33.679456 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:33.679438 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4453c718-64ae-4d46-b593-fe5c7a9f7247-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4f95p\" (UID: \"4453c718-64ae-4d46-b593-fe5c7a9f7247\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4f95p" Jul 22 03:12:33.687639 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:33.687608 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnnsx\" (UniqueName: \"kubernetes.io/projected/4453c718-64ae-4d46-b593-fe5c7a9f7247-kube-api-access-qnnsx\") pod \"cert-manager-operator-controller-manager-55c444bf86-4f95p\" (UID: \"4453c718-64ae-4d46-b593-fe5c7a9f7247\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4f95p" Jul 22 03:12:33.761643 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:33.761612 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4f95p" Jul 22 03:12:33.915727 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:33.915696 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4f95p"] Jul 22 03:12:33.916695 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:12:33.916675 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4453c718_64ae_4d46_b593_fe5c7a9f7247.slice/crio-6ad9865e4ba7aebb1ce7d38c9a804a84b8a2de3050e5ed870732b5dad277e20a WatchSource:0}: Error finding container 6ad9865e4ba7aebb1ce7d38c9a804a84b8a2de3050e5ed870732b5dad277e20a: Status 404 returned error can't find the container with id 6ad9865e4ba7aebb1ce7d38c9a804a84b8a2de3050e5ed870732b5dad277e20a Jul 22 03:12:34.164626 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:34.164582 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6b479d4584-tgg9g" podUID="ae275278-912c-454d-8b18-9bc619b8e3c1" containerName="console" containerID="cri-o://e0c1164b4114022fd0d9e1142c30af23369da80c2dfdf955c7e8f38ee7ae4fa8" gracePeriod=15 Jul 22 03:12:34.195600 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:34.195490 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4f95p" event={"ID":"4453c718-64ae-4d46-b593-fe5c7a9f7247","Type":"ContainerStarted","Data":"6ad9865e4ba7aebb1ce7d38c9a804a84b8a2de3050e5ed870732b5dad277e20a"} Jul 22 03:12:34.420391 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:34.420325 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b479d4584-tgg9g" Jul 22 03:12:34.588037 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:34.588001 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ae275278-912c-454d-8b18-9bc619b8e3c1-console-serving-cert\") pod \"ae275278-912c-454d-8b18-9bc619b8e3c1\" (UID: \"ae275278-912c-454d-8b18-9bc619b8e3c1\") " Jul 22 03:12:34.588037 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:34.588044 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ae275278-912c-454d-8b18-9bc619b8e3c1-console-config\") pod \"ae275278-912c-454d-8b18-9bc619b8e3c1\" (UID: \"ae275278-912c-454d-8b18-9bc619b8e3c1\") " Jul 22 03:12:34.588520 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:34.588074 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7qp6c\" (UniqueName: \"kubernetes.io/projected/ae275278-912c-454d-8b18-9bc619b8e3c1-kube-api-access-7qp6c\") pod \"ae275278-912c-454d-8b18-9bc619b8e3c1\" (UID: \"ae275278-912c-454d-8b18-9bc619b8e3c1\") " Jul 22 03:12:34.588520 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:34.588102 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ae275278-912c-454d-8b18-9bc619b8e3c1-oauth-serving-cert\") pod \"ae275278-912c-454d-8b18-9bc619b8e3c1\" (UID: \"ae275278-912c-454d-8b18-9bc619b8e3c1\") " Jul 22 03:12:34.588520 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:34.588156 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ae275278-912c-454d-8b18-9bc619b8e3c1-console-oauth-config\") pod \"ae275278-912c-454d-8b18-9bc619b8e3c1\" (UID: \"ae275278-912c-454d-8b18-9bc619b8e3c1\") " Jul 22 03:12:34.588520 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:34.588215 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae275278-912c-454d-8b18-9bc619b8e3c1-trusted-ca-bundle\") pod \"ae275278-912c-454d-8b18-9bc619b8e3c1\" (UID: \"ae275278-912c-454d-8b18-9bc619b8e3c1\") " Jul 22 03:12:34.588520 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:34.588247 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ae275278-912c-454d-8b18-9bc619b8e3c1-service-ca\") pod \"ae275278-912c-454d-8b18-9bc619b8e3c1\" (UID: \"ae275278-912c-454d-8b18-9bc619b8e3c1\") " Jul 22 03:12:34.588520 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:34.588412 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae275278-912c-454d-8b18-9bc619b8e3c1-console-config" (OuterVolumeSpecName: "console-config") pod "ae275278-912c-454d-8b18-9bc619b8e3c1" (UID: "ae275278-912c-454d-8b18-9bc619b8e3c1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:12:34.588766 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:34.588570 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae275278-912c-454d-8b18-9bc619b8e3c1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ae275278-912c-454d-8b18-9bc619b8e3c1" (UID: "ae275278-912c-454d-8b18-9bc619b8e3c1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:12:34.588766 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:34.588654 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae275278-912c-454d-8b18-9bc619b8e3c1-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ae275278-912c-454d-8b18-9bc619b8e3c1" (UID: "ae275278-912c-454d-8b18-9bc619b8e3c1"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:12:34.588766 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:34.588711 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae275278-912c-454d-8b18-9bc619b8e3c1-service-ca" (OuterVolumeSpecName: "service-ca") pod "ae275278-912c-454d-8b18-9bc619b8e3c1" (UID: "ae275278-912c-454d-8b18-9bc619b8e3c1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:12:34.590227 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:34.590207 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae275278-912c-454d-8b18-9bc619b8e3c1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ae275278-912c-454d-8b18-9bc619b8e3c1" (UID: "ae275278-912c-454d-8b18-9bc619b8e3c1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:12:34.590308 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:34.590294 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae275278-912c-454d-8b18-9bc619b8e3c1-kube-api-access-7qp6c" (OuterVolumeSpecName: "kube-api-access-7qp6c") pod "ae275278-912c-454d-8b18-9bc619b8e3c1" (UID: "ae275278-912c-454d-8b18-9bc619b8e3c1"). InnerVolumeSpecName "kube-api-access-7qp6c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:12:34.590343 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:34.590293 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae275278-912c-454d-8b18-9bc619b8e3c1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ae275278-912c-454d-8b18-9bc619b8e3c1" (UID: "ae275278-912c-454d-8b18-9bc619b8e3c1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:12:34.689050 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:34.688970 2556 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ae275278-912c-454d-8b18-9bc619b8e3c1-console-serving-cert\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:12:34.689050 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:34.689006 2556 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ae275278-912c-454d-8b18-9bc619b8e3c1-console-config\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:12:34.689050 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:34.689024 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qp6c\" (UniqueName: \"kubernetes.io/projected/ae275278-912c-454d-8b18-9bc619b8e3c1-kube-api-access-7qp6c\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:12:34.689050 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:34.689039 2556 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ae275278-912c-454d-8b18-9bc619b8e3c1-oauth-serving-cert\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:12:34.689373 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:34.689056 2556 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ae275278-912c-454d-8b18-9bc619b8e3c1-console-oauth-config\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:12:34.689373 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:34.689069 2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae275278-912c-454d-8b18-9bc619b8e3c1-trusted-ca-bundle\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:12:34.689373 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:34.689077 2556 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ae275278-912c-454d-8b18-9bc619b8e3c1-service-ca\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:12:35.201702 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:35.201662 2556 generic.go:356] "Generic (PLEG): container finished" podID="ae275278-912c-454d-8b18-9bc619b8e3c1" containerID="e0c1164b4114022fd0d9e1142c30af23369da80c2dfdf955c7e8f38ee7ae4fa8" exitCode=0 Jul 22 03:12:35.201885 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:35.201721 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b479d4584-tgg9g" Jul 22 03:12:35.201885 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:35.201744 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b479d4584-tgg9g" event={"ID":"ae275278-912c-454d-8b18-9bc619b8e3c1","Type":"ContainerDied","Data":"e0c1164b4114022fd0d9e1142c30af23369da80c2dfdf955c7e8f38ee7ae4fa8"} Jul 22 03:12:35.201885 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:35.201801 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b479d4584-tgg9g" event={"ID":"ae275278-912c-454d-8b18-9bc619b8e3c1","Type":"ContainerDied","Data":"b917ce014968cbdd90fca121da4b49c1a072829437d3bba84c371cd953b5a36a"} Jul 22 03:12:35.201885 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:35.201821 2556 scope.go:117] "RemoveContainer" containerID="e0c1164b4114022fd0d9e1142c30af23369da80c2dfdf955c7e8f38ee7ae4fa8" Jul 22 03:12:35.226850 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:35.226827 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b479d4584-tgg9g"] Jul 22 03:12:35.230831 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:35.230808 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b479d4584-tgg9g"] Jul 22 03:12:35.244817 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:35.244789 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:12:35.348105 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:35.348057 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:12:35.592916 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:35.592893 2556 scope.go:117] "RemoveContainer" containerID="e0c1164b4114022fd0d9e1142c30af23369da80c2dfdf955c7e8f38ee7ae4fa8" Jul 22 03:12:35.593274 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:12:35.593252 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0c1164b4114022fd0d9e1142c30af23369da80c2dfdf955c7e8f38ee7ae4fa8\": container with ID starting with e0c1164b4114022fd0d9e1142c30af23369da80c2dfdf955c7e8f38ee7ae4fa8 not found: ID does not exist" containerID="e0c1164b4114022fd0d9e1142c30af23369da80c2dfdf955c7e8f38ee7ae4fa8" Jul 22 03:12:35.593311 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:35.593284 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0c1164b4114022fd0d9e1142c30af23369da80c2dfdf955c7e8f38ee7ae4fa8"} err="failed to get container status \"e0c1164b4114022fd0d9e1142c30af23369da80c2dfdf955c7e8f38ee7ae4fa8\": rpc error: code = NotFound desc = could not find container \"e0c1164b4114022fd0d9e1142c30af23369da80c2dfdf955c7e8f38ee7ae4fa8\": container with ID starting with e0c1164b4114022fd0d9e1142c30af23369da80c2dfdf955c7e8f38ee7ae4fa8 not found: ID does not exist" Jul 22 03:12:36.023200 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:36.023127 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae275278-912c-454d-8b18-9bc619b8e3c1" path="/var/lib/kubelet/pods/ae275278-912c-454d-8b18-9bc619b8e3c1/volumes" Jul 22 03:12:36.208030 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:36.207989 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4f95p" event={"ID":"4453c718-64ae-4d46-b593-fe5c7a9f7247","Type":"ContainerStarted","Data":"a579269c869af23158d7feada6f3d86b71ea8183932e70fde9f2de55e57bc7ed"} Jul 22 03:12:36.232127 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:36.232072 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4f95p" podStartSLOduration=1.100791642 podStartE2EDuration="3.232055401s" totalImagesPullingTime="2.131263759s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:12:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:12:36.231147238 +0000 UTC m=+296.680130489" watchObservedRunningTime="2026-07-22 03:12:36.232055401 +0000 UTC m=+296.681038650" Jul 22 03:12:36.690448 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:36.690414 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:12:36.814237 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:36.814202 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:12:37.003555 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:37.003469 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:12:37.103068 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:37.103029 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:12:37.367450 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:37.367418 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9pmx8"] Jul 22 03:12:37.367722 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:37.367709 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae275278-912c-454d-8b18-9bc619b8e3c1" containerName="console" Jul 22 03:12:37.367800 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:37.367733 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae275278-912c-454d-8b18-9bc619b8e3c1" containerName="console" Jul 22 03:12:37.367860 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:37.367814 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae275278-912c-454d-8b18-9bc619b8e3c1" containerName="console" Jul 22 03:12:37.369771 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:37.369752 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9pmx8" Jul 22 03:12:37.372408 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:37.372388 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 22 03:12:37.372528 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:37.372460 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kxg97\"" Jul 22 03:12:37.373492 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:37.373470 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 22 03:12:37.388897 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:37.388875 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9pmx8"] Jul 22 03:12:37.512832 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:37.512795 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sk5lp\" (UniqueName: \"kubernetes.io/projected/52acc26f-5c44-4109-9320-c06e1dbeb4d2-kube-api-access-sk5lp\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9pmx8\" (UID: \"52acc26f-5c44-4109-9320-c06e1dbeb4d2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9pmx8" Jul 22 03:12:37.513027 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:37.512842 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/52acc26f-5c44-4109-9320-c06e1dbeb4d2-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9pmx8\" (UID: \"52acc26f-5c44-4109-9320-c06e1dbeb4d2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9pmx8" Jul 22 03:12:37.513027 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:37.512893 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/52acc26f-5c44-4109-9320-c06e1dbeb4d2-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9pmx8\" (UID: \"52acc26f-5c44-4109-9320-c06e1dbeb4d2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9pmx8" Jul 22 03:12:37.613944 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:37.613910 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk5lp\" (UniqueName: \"kubernetes.io/projected/52acc26f-5c44-4109-9320-c06e1dbeb4d2-kube-api-access-sk5lp\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9pmx8\" (UID: \"52acc26f-5c44-4109-9320-c06e1dbeb4d2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9pmx8" Jul 22 03:12:37.614123 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:37.613969 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/52acc26f-5c44-4109-9320-c06e1dbeb4d2-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9pmx8\" (UID: \"52acc26f-5c44-4109-9320-c06e1dbeb4d2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9pmx8" Jul 22 03:12:37.614123 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:37.614006 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/52acc26f-5c44-4109-9320-c06e1dbeb4d2-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9pmx8\" (UID: \"52acc26f-5c44-4109-9320-c06e1dbeb4d2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9pmx8" Jul 22 03:12:37.614358 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:37.614342 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/52acc26f-5c44-4109-9320-c06e1dbeb4d2-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9pmx8\" (UID: \"52acc26f-5c44-4109-9320-c06e1dbeb4d2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9pmx8" Jul 22 03:12:37.614406 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:37.614389 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/52acc26f-5c44-4109-9320-c06e1dbeb4d2-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9pmx8\" (UID: \"52acc26f-5c44-4109-9320-c06e1dbeb4d2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9pmx8" Jul 22 03:12:37.622075 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:37.622010 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sk5lp\" (UniqueName: \"kubernetes.io/projected/52acc26f-5c44-4109-9320-c06e1dbeb4d2-kube-api-access-sk5lp\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9pmx8\" (UID: \"52acc26f-5c44-4109-9320-c06e1dbeb4d2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9pmx8" Jul 22 03:12:37.678716 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:37.678690 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9pmx8" Jul 22 03:12:37.806380 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:37.806353 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9pmx8"] Jul 22 03:12:37.808141 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:12:37.808109 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52acc26f_5c44_4109_9320_c06e1dbeb4d2.slice/crio-6186b9d29adeda886652b3aa12e9e72499d47e2e20c8b011e934a42385b01cee WatchSource:0}: Error finding container 6186b9d29adeda886652b3aa12e9e72499d47e2e20c8b011e934a42385b01cee: Status 404 returned error can't find the container with id 6186b9d29adeda886652b3aa12e9e72499d47e2e20c8b011e934a42385b01cee Jul 22 03:12:38.217825 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:38.217739 2556 generic.go:356] "Generic (PLEG): container finished" podID="52acc26f-5c44-4109-9320-c06e1dbeb4d2" containerID="a0dc242970d1299af0e64e66a326de1723fc717d519731fea84c940fb9955455" exitCode=0 Jul 22 03:12:38.217825 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:38.217801 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9pmx8" event={"ID":"52acc26f-5c44-4109-9320-c06e1dbeb4d2","Type":"ContainerDied","Data":"a0dc242970d1299af0e64e66a326de1723fc717d519731fea84c940fb9955455"} Jul 22 03:12:38.218040 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:38.217861 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9pmx8" event={"ID":"52acc26f-5c44-4109-9320-c06e1dbeb4d2","Type":"ContainerStarted","Data":"6186b9d29adeda886652b3aa12e9e72499d47e2e20c8b011e934a42385b01cee"} Jul 22 03:12:39.528048 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:39.528004 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:12:39.748752 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:39.748707 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:12:39.909341 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:39.909312 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-t4vvn"] Jul 22 03:12:39.916158 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:39.916136 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-t4vvn" Jul 22 03:12:39.918984 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:39.918947 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 22 03:12:39.919911 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:39.919892 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 22 03:12:39.920042 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:39.919893 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-hlwwq\"" Jul 22 03:12:39.922495 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:39.922472 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-t4vvn"] Jul 22 03:12:39.933823 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:39.933796 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/488a5d7d-99c5-4249-bd59-1fea4ccefbef-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-t4vvn\" (UID: \"488a5d7d-99c5-4249-bd59-1fea4ccefbef\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-t4vvn" Jul 22 03:12:39.933939 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:39.933842 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bp2qd\" (UniqueName: \"kubernetes.io/projected/488a5d7d-99c5-4249-bd59-1fea4ccefbef-kube-api-access-bp2qd\") pod \"cert-manager-cainjector-559cdd8c87-t4vvn\" (UID: \"488a5d7d-99c5-4249-bd59-1fea4ccefbef\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-t4vvn" Jul 22 03:12:39.996598 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:39.996577 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscf8_a01dfba4-f590-412a-8438-1646c51bb1a1/ovn-acl-logging/0.log" Jul 22 03:12:39.996716 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:39.996582 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscf8_a01dfba4-f590-412a-8438-1646c51bb1a1/ovn-acl-logging/0.log" Jul 22 03:12:39.999533 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:39.999517 2556 kubelet.go:1648] "Image garbage collection succeeded" Jul 22 03:12:40.036795 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:40.034927 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bp2qd\" (UniqueName: \"kubernetes.io/projected/488a5d7d-99c5-4249-bd59-1fea4ccefbef-kube-api-access-bp2qd\") pod \"cert-manager-cainjector-559cdd8c87-t4vvn\" (UID: \"488a5d7d-99c5-4249-bd59-1fea4ccefbef\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-t4vvn" Jul 22 03:12:40.036795 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:40.035022 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/488a5d7d-99c5-4249-bd59-1fea4ccefbef-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-t4vvn\" (UID: \"488a5d7d-99c5-4249-bd59-1fea4ccefbef\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-t4vvn" Jul 22 03:12:40.043262 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:40.043241 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/488a5d7d-99c5-4249-bd59-1fea4ccefbef-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-t4vvn\" (UID: \"488a5d7d-99c5-4249-bd59-1fea4ccefbef\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-t4vvn" Jul 22 03:12:40.043518 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:40.043501 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bp2qd\" (UniqueName: \"kubernetes.io/projected/488a5d7d-99c5-4249-bd59-1fea4ccefbef-kube-api-access-bp2qd\") pod \"cert-manager-cainjector-559cdd8c87-t4vvn\" (UID: \"488a5d7d-99c5-4249-bd59-1fea4ccefbef\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-t4vvn" Jul 22 03:12:40.229064 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:40.228978 2556 generic.go:356] "Generic (PLEG): container finished" podID="52acc26f-5c44-4109-9320-c06e1dbeb4d2" containerID="52890fc5e63bbd0ef5be349c0248e6a118f306197589ab3a12b59fe2fcfa7cf4" exitCode=0 Jul 22 03:12:40.229064 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:40.229036 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9pmx8" event={"ID":"52acc26f-5c44-4109-9320-c06e1dbeb4d2","Type":"ContainerDied","Data":"52890fc5e63bbd0ef5be349c0248e6a118f306197589ab3a12b59fe2fcfa7cf4"} Jul 22 03:12:40.237201 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:40.237181 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-t4vvn" Jul 22 03:12:40.369745 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:40.369715 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-t4vvn"] Jul 22 03:12:40.387761 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:12:40.387731 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod488a5d7d_99c5_4249_bd59_1fea4ccefbef.slice/crio-133d7f3dcd5b90b452eae8a089e04f964c20ee1ef742fae1527679639f4a199c WatchSource:0}: Error finding container 133d7f3dcd5b90b452eae8a089e04f964c20ee1ef742fae1527679639f4a199c: Status 404 returned error can't find the container with id 133d7f3dcd5b90b452eae8a089e04f964c20ee1ef742fae1527679639f4a199c Jul 22 03:12:41.236669 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:41.236637 2556 generic.go:356] "Generic (PLEG): container finished" podID="52acc26f-5c44-4109-9320-c06e1dbeb4d2" containerID="d08763757c0cb43ff71793ea944be34010172ce6aece85aaccd7d2b9dac0738a" exitCode=0 Jul 22 03:12:41.237128 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:41.236725 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9pmx8" event={"ID":"52acc26f-5c44-4109-9320-c06e1dbeb4d2","Type":"ContainerDied","Data":"d08763757c0cb43ff71793ea944be34010172ce6aece85aaccd7d2b9dac0738a"} Jul 22 03:12:41.238315 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:41.238286 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-t4vvn" event={"ID":"488a5d7d-99c5-4249-bd59-1fea4ccefbef","Type":"ContainerStarted","Data":"133d7f3dcd5b90b452eae8a089e04f964c20ee1ef742fae1527679639f4a199c"} Jul 22 03:12:41.829598 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:41.829565 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:12:41.929787 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:41.929725 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:12:43.011385 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:43.011358 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9pmx8" Jul 22 03:12:43.055892 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:43.055866 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/52acc26f-5c44-4109-9320-c06e1dbeb4d2-util\") pod \"52acc26f-5c44-4109-9320-c06e1dbeb4d2\" (UID: \"52acc26f-5c44-4109-9320-c06e1dbeb4d2\") " Jul 22 03:12:43.056042 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:43.055908 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/52acc26f-5c44-4109-9320-c06e1dbeb4d2-bundle\") pod \"52acc26f-5c44-4109-9320-c06e1dbeb4d2\" (UID: \"52acc26f-5c44-4109-9320-c06e1dbeb4d2\") " Jul 22 03:12:43.056042 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:43.055943 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sk5lp\" (UniqueName: \"kubernetes.io/projected/52acc26f-5c44-4109-9320-c06e1dbeb4d2-kube-api-access-sk5lp\") pod \"52acc26f-5c44-4109-9320-c06e1dbeb4d2\" (UID: \"52acc26f-5c44-4109-9320-c06e1dbeb4d2\") " Jul 22 03:12:43.056507 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:43.056480 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52acc26f-5c44-4109-9320-c06e1dbeb4d2-bundle" (OuterVolumeSpecName: "bundle") pod "52acc26f-5c44-4109-9320-c06e1dbeb4d2" (UID: "52acc26f-5c44-4109-9320-c06e1dbeb4d2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:12:43.058168 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:43.058146 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52acc26f-5c44-4109-9320-c06e1dbeb4d2-kube-api-access-sk5lp" (OuterVolumeSpecName: "kube-api-access-sk5lp") pod "52acc26f-5c44-4109-9320-c06e1dbeb4d2" (UID: "52acc26f-5c44-4109-9320-c06e1dbeb4d2"). InnerVolumeSpecName "kube-api-access-sk5lp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:12:43.060746 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:43.060723 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52acc26f-5c44-4109-9320-c06e1dbeb4d2-util" (OuterVolumeSpecName: "util") pod "52acc26f-5c44-4109-9320-c06e1dbeb4d2" (UID: "52acc26f-5c44-4109-9320-c06e1dbeb4d2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:12:43.156839 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:43.156804 2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/52acc26f-5c44-4109-9320-c06e1dbeb4d2-bundle\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:12:43.156839 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:43.156837 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sk5lp\" (UniqueName: \"kubernetes.io/projected/52acc26f-5c44-4109-9320-c06e1dbeb4d2-kube-api-access-sk5lp\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:12:43.156839 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:43.156848 2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/52acc26f-5c44-4109-9320-c06e1dbeb4d2-util\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:12:43.250014 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:43.249919 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9pmx8" event={"ID":"52acc26f-5c44-4109-9320-c06e1dbeb4d2","Type":"ContainerDied","Data":"6186b9d29adeda886652b3aa12e9e72499d47e2e20c8b011e934a42385b01cee"} Jul 22 03:12:43.250014 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:43.249993 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6186b9d29adeda886652b3aa12e9e72499d47e2e20c8b011e934a42385b01cee" Jul 22 03:12:43.250212 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:43.249926 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9pmx8" Jul 22 03:12:43.251602 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:43.251575 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-t4vvn" event={"ID":"488a5d7d-99c5-4249-bd59-1fea4ccefbef","Type":"ContainerStarted","Data":"3a6661b21e34f4cbf7760e95c74f321f1140fbd6b52023146ff1e569d49ce5fb"} Jul 22 03:12:43.273198 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:43.273150 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-t4vvn" podStartSLOduration=1.637291252 podStartE2EDuration="4.273133589s" totalImagesPullingTime="2.635842337s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:12:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:12:43.271671013 +0000 UTC m=+303.720654265" watchObservedRunningTime="2026-07-22 03:12:43.273133589 +0000 UTC m=+303.722116842" Jul 22 03:12:46.933407 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:46.933359 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:12:48.248577 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:48.248539 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:12:48.347659 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:48.347626 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:12:50.479330 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:50.479293 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2nzz6"] Jul 22 03:12:50.479705 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:50.479558 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52acc26f-5c44-4109-9320-c06e1dbeb4d2" containerName="util" Jul 22 03:12:50.479705 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:50.479569 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="52acc26f-5c44-4109-9320-c06e1dbeb4d2" containerName="util" Jul 22 03:12:50.479705 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:50.479575 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52acc26f-5c44-4109-9320-c06e1dbeb4d2" containerName="pull" Jul 22 03:12:50.479705 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:50.479580 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="52acc26f-5c44-4109-9320-c06e1dbeb4d2" containerName="pull" Jul 22 03:12:50.479705 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:50.479589 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52acc26f-5c44-4109-9320-c06e1dbeb4d2" containerName="extract" Jul 22 03:12:50.479705 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:50.479594 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="52acc26f-5c44-4109-9320-c06e1dbeb4d2" containerName="extract" Jul 22 03:12:50.479705 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:50.479643 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="52acc26f-5c44-4109-9320-c06e1dbeb4d2" containerName="extract" Jul 22 03:12:50.481268 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:50.481254 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2nzz6" Jul 22 03:12:50.484557 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:50.484536 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-qrc8b\"" Jul 22 03:12:50.485746 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:50.485726 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 22 03:12:50.485795 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:50.485754 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 22 03:12:50.492835 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:50.492813 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2nzz6"] Jul 22 03:12:50.513918 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:50.513890 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9scr\" (UniqueName: \"kubernetes.io/projected/3c165226-b386-4f76-a5c7-4c9c2204daae-kube-api-access-m9scr\") pod \"openshift-lws-operator-fd8ccff4c-2nzz6\" (UID: \"3c165226-b386-4f76-a5c7-4c9c2204daae\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2nzz6" Jul 22 03:12:50.514065 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:50.513929 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3c165226-b386-4f76-a5c7-4c9c2204daae-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2nzz6\" (UID: \"3c165226-b386-4f76-a5c7-4c9c2204daae\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2nzz6" Jul 22 03:12:50.614727 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:50.614700 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9scr\" (UniqueName: \"kubernetes.io/projected/3c165226-b386-4f76-a5c7-4c9c2204daae-kube-api-access-m9scr\") pod \"openshift-lws-operator-fd8ccff4c-2nzz6\" (UID: \"3c165226-b386-4f76-a5c7-4c9c2204daae\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2nzz6" Jul 22 03:12:50.614868 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:50.614736 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3c165226-b386-4f76-a5c7-4c9c2204daae-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2nzz6\" (UID: \"3c165226-b386-4f76-a5c7-4c9c2204daae\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2nzz6" Jul 22 03:12:50.615200 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:50.615183 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3c165226-b386-4f76-a5c7-4c9c2204daae-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2nzz6\" (UID: \"3c165226-b386-4f76-a5c7-4c9c2204daae\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2nzz6" Jul 22 03:12:50.623353 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:50.623326 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9scr\" (UniqueName: \"kubernetes.io/projected/3c165226-b386-4f76-a5c7-4c9c2204daae-kube-api-access-m9scr\") pod \"openshift-lws-operator-fd8ccff4c-2nzz6\" (UID: \"3c165226-b386-4f76-a5c7-4c9c2204daae\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2nzz6" Jul 22 03:12:50.789924 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:50.789834 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2nzz6" Jul 22 03:12:50.913862 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:50.913838 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2nzz6"] Jul 22 03:12:50.916483 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:12:50.916453 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3c165226_b386_4f76_a5c7_4c9c2204daae.slice/crio-e0d8f7acf35a2da4d4ae2a678f1e4f829bcaaca406c56265108bbaa89491e799 WatchSource:0}: Error finding container e0d8f7acf35a2da4d4ae2a678f1e4f829bcaaca406c56265108bbaa89491e799: Status 404 returned error can't find the container with id e0d8f7acf35a2da4d4ae2a678f1e4f829bcaaca406c56265108bbaa89491e799 Jul 22 03:12:50.918170 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:50.918153 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 03:12:51.286473 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:51.286439 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2nzz6" event={"ID":"3c165226-b386-4f76-a5c7-4c9c2204daae","Type":"ContainerStarted","Data":"e0d8f7acf35a2da4d4ae2a678f1e4f829bcaaca406c56265108bbaa89491e799"} Jul 22 03:12:53.295628 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:53.295595 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2nzz6" event={"ID":"3c165226-b386-4f76-a5c7-4c9c2204daae","Type":"ContainerStarted","Data":"c2468a03b23ea88bd7fed78256e61128ac7dd90b8c88d0c5736a775388cca599"} Jul 22 03:12:53.313772 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:53.313721 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2nzz6" podStartSLOduration=1.759173192 podStartE2EDuration="3.313707216s" totalImagesPullingTime="1.554534024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:12:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:12:53.3131798 +0000 UTC m=+313.762163070" watchObservedRunningTime="2026-07-22 03:12:53.313707216 +0000 UTC m=+313.762690465" Jul 22 03:12:53.961869 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:53.961838 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:12:54.076640 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:54.076605 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:12:56.129324 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:56.129290 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:12:57.923553 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:57.923518 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4xdj9"] Jul 22 03:12:57.944714 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:57.944688 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4xdj9"] Jul 22 03:12:57.944857 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:57.944786 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4xdj9" Jul 22 03:12:57.947621 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:57.947592 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 22 03:12:57.948746 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:57.948724 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 22 03:12:57.948868 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:57.948790 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kxg97\"" Jul 22 03:12:57.975033 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:57.975006 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/57066c43-511c-4fbc-9a4b-6f69aac878ba-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4xdj9\" (UID: \"57066c43-511c-4fbc-9a4b-6f69aac878ba\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4xdj9" Jul 22 03:12:57.975142 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:57.975070 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/57066c43-511c-4fbc-9a4b-6f69aac878ba-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4xdj9\" (UID: \"57066c43-511c-4fbc-9a4b-6f69aac878ba\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4xdj9" Jul 22 03:12:57.975142 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:57.975112 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5ngm\" (UniqueName: \"kubernetes.io/projected/57066c43-511c-4fbc-9a4b-6f69aac878ba-kube-api-access-j5ngm\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4xdj9\" (UID: \"57066c43-511c-4fbc-9a4b-6f69aac878ba\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4xdj9" Jul 22 03:12:58.075524 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:58.075489 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5ngm\" (UniqueName: \"kubernetes.io/projected/57066c43-511c-4fbc-9a4b-6f69aac878ba-kube-api-access-j5ngm\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4xdj9\" (UID: \"57066c43-511c-4fbc-9a4b-6f69aac878ba\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4xdj9" Jul 22 03:12:58.075524 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:58.075533 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/57066c43-511c-4fbc-9a4b-6f69aac878ba-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4xdj9\" (UID: \"57066c43-511c-4fbc-9a4b-6f69aac878ba\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4xdj9" Jul 22 03:12:58.075767 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:58.075721 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/57066c43-511c-4fbc-9a4b-6f69aac878ba-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4xdj9\" (UID: \"57066c43-511c-4fbc-9a4b-6f69aac878ba\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4xdj9" Jul 22 03:12:58.075916 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:58.075895 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/57066c43-511c-4fbc-9a4b-6f69aac878ba-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4xdj9\" (UID: \"57066c43-511c-4fbc-9a4b-6f69aac878ba\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4xdj9" Jul 22 03:12:58.076079 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:58.076062 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/57066c43-511c-4fbc-9a4b-6f69aac878ba-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4xdj9\" (UID: \"57066c43-511c-4fbc-9a4b-6f69aac878ba\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4xdj9" Jul 22 03:12:58.084588 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:58.084559 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5ngm\" (UniqueName: \"kubernetes.io/projected/57066c43-511c-4fbc-9a4b-6f69aac878ba-kube-api-access-j5ngm\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4xdj9\" (UID: \"57066c43-511c-4fbc-9a4b-6f69aac878ba\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4xdj9" Jul 22 03:12:58.253431 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:58.253347 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4xdj9" Jul 22 03:12:58.398463 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:58.398439 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4xdj9"] Jul 22 03:12:58.400291 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:12:58.400265 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57066c43_511c_4fbc_9a4b_6f69aac878ba.slice/crio-dbe9997cf98b0a87360ec8f10836d1a4c1a605a017f0070c95679845e4959555 WatchSource:0}: Error finding container dbe9997cf98b0a87360ec8f10836d1a4c1a605a017f0070c95679845e4959555: Status 404 returned error can't find the container with id dbe9997cf98b0a87360ec8f10836d1a4c1a605a017f0070c95679845e4959555 Jul 22 03:12:59.320174 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:59.320145 2556 generic.go:356] "Generic (PLEG): container finished" podID="57066c43-511c-4fbc-9a4b-6f69aac878ba" containerID="ffbe3432156865d624081fa8d377e58c1764af88acb44571d692eb3c7330b2b9" exitCode=0 Jul 22 03:12:59.320537 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:59.320235 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4xdj9" event={"ID":"57066c43-511c-4fbc-9a4b-6f69aac878ba","Type":"ContainerDied","Data":"ffbe3432156865d624081fa8d377e58c1764af88acb44571d692eb3c7330b2b9"} Jul 22 03:12:59.320537 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:12:59.320305 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4xdj9" event={"ID":"57066c43-511c-4fbc-9a4b-6f69aac878ba","Type":"ContainerStarted","Data":"dbe9997cf98b0a87360ec8f10836d1a4c1a605a017f0070c95679845e4959555"} Jul 22 03:13:00.326664 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:00.326629 2556 generic.go:356] "Generic (PLEG): container finished" podID="57066c43-511c-4fbc-9a4b-6f69aac878ba" containerID="670443229dba134978b9029d614b78c455101d0fb6c95f1f5df4f5fff0b704c7" exitCode=0 Jul 22 03:13:00.327157 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:00.326684 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4xdj9" event={"ID":"57066c43-511c-4fbc-9a4b-6f69aac878ba","Type":"ContainerDied","Data":"670443229dba134978b9029d614b78c455101d0fb6c95f1f5df4f5fff0b704c7"} Jul 22 03:13:01.332613 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:01.332577 2556 generic.go:356] "Generic (PLEG): container finished" podID="57066c43-511c-4fbc-9a4b-6f69aac878ba" containerID="8127735b5e268ee1f1ff551f9a84556c5d71b4dd365b516cf5bc9960b5ae00d4" exitCode=0 Jul 22 03:13:01.333003 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:01.332616 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4xdj9" event={"ID":"57066c43-511c-4fbc-9a4b-6f69aac878ba","Type":"ContainerDied","Data":"8127735b5e268ee1f1ff551f9a84556c5d71b4dd365b516cf5bc9960b5ae00d4"} Jul 22 03:13:02.455243 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:02.455217 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4xdj9" Jul 22 03:13:02.509822 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:02.509787 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j5ngm\" (UniqueName: \"kubernetes.io/projected/57066c43-511c-4fbc-9a4b-6f69aac878ba-kube-api-access-j5ngm\") pod \"57066c43-511c-4fbc-9a4b-6f69aac878ba\" (UID: \"57066c43-511c-4fbc-9a4b-6f69aac878ba\") " Jul 22 03:13:02.510024 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:02.509842 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/57066c43-511c-4fbc-9a4b-6f69aac878ba-bundle\") pod \"57066c43-511c-4fbc-9a4b-6f69aac878ba\" (UID: \"57066c43-511c-4fbc-9a4b-6f69aac878ba\") " Jul 22 03:13:02.510024 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:02.509905 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/57066c43-511c-4fbc-9a4b-6f69aac878ba-util\") pod \"57066c43-511c-4fbc-9a4b-6f69aac878ba\" (UID: \"57066c43-511c-4fbc-9a4b-6f69aac878ba\") " Jul 22 03:13:02.510633 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:02.510601 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57066c43-511c-4fbc-9a4b-6f69aac878ba-bundle" (OuterVolumeSpecName: "bundle") pod "57066c43-511c-4fbc-9a4b-6f69aac878ba" (UID: "57066c43-511c-4fbc-9a4b-6f69aac878ba"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:13:02.511925 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:02.511900 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57066c43-511c-4fbc-9a4b-6f69aac878ba-kube-api-access-j5ngm" (OuterVolumeSpecName: "kube-api-access-j5ngm") pod "57066c43-511c-4fbc-9a4b-6f69aac878ba" (UID: "57066c43-511c-4fbc-9a4b-6f69aac878ba"). InnerVolumeSpecName "kube-api-access-j5ngm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:13:02.515586 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:02.515563 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57066c43-511c-4fbc-9a4b-6f69aac878ba-util" (OuterVolumeSpecName: "util") pod "57066c43-511c-4fbc-9a4b-6f69aac878ba" (UID: "57066c43-511c-4fbc-9a4b-6f69aac878ba"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:13:02.610539 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:02.610436 2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/57066c43-511c-4fbc-9a4b-6f69aac878ba-util\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:13:02.610539 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:02.610483 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5ngm\" (UniqueName: \"kubernetes.io/projected/57066c43-511c-4fbc-9a4b-6f69aac878ba-kube-api-access-j5ngm\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:13:02.610539 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:02.610495 2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/57066c43-511c-4fbc-9a4b-6f69aac878ba-bundle\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:13:03.343288 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:03.343251 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4xdj9" event={"ID":"57066c43-511c-4fbc-9a4b-6f69aac878ba","Type":"ContainerDied","Data":"dbe9997cf98b0a87360ec8f10836d1a4c1a605a017f0070c95679845e4959555"} Jul 22 03:13:03.343288 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:03.343285 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4xdj9" Jul 22 03:13:03.343492 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:03.343306 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dbe9997cf98b0a87360ec8f10836d1a4c1a605a017f0070c95679845e4959555" Jul 22 03:13:10.455319 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:10.455278 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:10.550218 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:10.550180 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:11.062136 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.062105 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:11.332752 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.332714 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:11.436242 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.436194 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:11.489428 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.489396 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5fb7f869bf-gsrx5"] Jul 22 03:13:11.489846 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.489677 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57066c43-511c-4fbc-9a4b-6f69aac878ba" containerName="pull" Jul 22 03:13:11.489846 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.489687 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="57066c43-511c-4fbc-9a4b-6f69aac878ba" containerName="pull" Jul 22 03:13:11.489846 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.489696 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57066c43-511c-4fbc-9a4b-6f69aac878ba" containerName="extract" Jul 22 03:13:11.489846 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.489702 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="57066c43-511c-4fbc-9a4b-6f69aac878ba" containerName="extract" Jul 22 03:13:11.489846 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.489711 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57066c43-511c-4fbc-9a4b-6f69aac878ba" containerName="util" Jul 22 03:13:11.489846 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.489717 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="57066c43-511c-4fbc-9a4b-6f69aac878ba" containerName="util" Jul 22 03:13:11.489846 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.489776 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="57066c43-511c-4fbc-9a4b-6f69aac878ba" containerName="extract" Jul 22 03:13:11.495352 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.495286 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5fb7f869bf-gsrx5" Jul 22 03:13:11.497747 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.497720 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 22 03:13:11.500647 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.500629 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 22 03:13:11.500783 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.500758 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 22 03:13:11.500783 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.500766 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-mmj5l\"" Jul 22 03:13:11.500880 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.500823 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 22 03:13:11.511759 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.511736 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5fb7f869bf-gsrx5"] Jul 22 03:13:11.580344 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.580311 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4c37b2ca-8721-4147-bf19-eb95bf2d1f4d-manifests\") pod \"opendatahub-operator-controller-manager-5fb7f869bf-gsrx5\" (UID: \"4c37b2ca-8721-4147-bf19-eb95bf2d1f4d\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb7f869bf-gsrx5" Jul 22 03:13:11.580344 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.580344 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c37b2ca-8721-4147-bf19-eb95bf2d1f4d-tmp\") pod \"opendatahub-operator-controller-manager-5fb7f869bf-gsrx5\" (UID: \"4c37b2ca-8721-4147-bf19-eb95bf2d1f4d\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb7f869bf-gsrx5" Jul 22 03:13:11.580584 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.580361 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4c37b2ca-8721-4147-bf19-eb95bf2d1f4d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5fb7f869bf-gsrx5\" (UID: \"4c37b2ca-8721-4147-bf19-eb95bf2d1f4d\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb7f869bf-gsrx5" Jul 22 03:13:11.580584 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.580395 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4c37b2ca-8721-4147-bf19-eb95bf2d1f4d-webhook-cert\") pod \"opendatahub-operator-controller-manager-5fb7f869bf-gsrx5\" (UID: \"4c37b2ca-8721-4147-bf19-eb95bf2d1f4d\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb7f869bf-gsrx5" Jul 22 03:13:11.580584 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.580476 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rs2rj\" (UniqueName: \"kubernetes.io/projected/4c37b2ca-8721-4147-bf19-eb95bf2d1f4d-kube-api-access-rs2rj\") pod \"opendatahub-operator-controller-manager-5fb7f869bf-gsrx5\" (UID: \"4c37b2ca-8721-4147-bf19-eb95bf2d1f4d\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb7f869bf-gsrx5" Jul 22 03:13:11.673666 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.673574 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:11.681566 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.681540 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4c37b2ca-8721-4147-bf19-eb95bf2d1f4d-manifests\") pod \"opendatahub-operator-controller-manager-5fb7f869bf-gsrx5\" (UID: \"4c37b2ca-8721-4147-bf19-eb95bf2d1f4d\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb7f869bf-gsrx5" Jul 22 03:13:11.681654 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.681576 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c37b2ca-8721-4147-bf19-eb95bf2d1f4d-tmp\") pod \"opendatahub-operator-controller-manager-5fb7f869bf-gsrx5\" (UID: \"4c37b2ca-8721-4147-bf19-eb95bf2d1f4d\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb7f869bf-gsrx5" Jul 22 03:13:11.681654 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.681598 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4c37b2ca-8721-4147-bf19-eb95bf2d1f4d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5fb7f869bf-gsrx5\" (UID: \"4c37b2ca-8721-4147-bf19-eb95bf2d1f4d\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb7f869bf-gsrx5" Jul 22 03:13:11.681654 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.681628 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4c37b2ca-8721-4147-bf19-eb95bf2d1f4d-webhook-cert\") pod \"opendatahub-operator-controller-manager-5fb7f869bf-gsrx5\" (UID: \"4c37b2ca-8721-4147-bf19-eb95bf2d1f4d\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb7f869bf-gsrx5" Jul 22 03:13:11.681783 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.681659 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rs2rj\" (UniqueName: \"kubernetes.io/projected/4c37b2ca-8721-4147-bf19-eb95bf2d1f4d-kube-api-access-rs2rj\") pod \"opendatahub-operator-controller-manager-5fb7f869bf-gsrx5\" (UID: \"4c37b2ca-8721-4147-bf19-eb95bf2d1f4d\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb7f869bf-gsrx5" Jul 22 03:13:11.682043 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.682024 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4c37b2ca-8721-4147-bf19-eb95bf2d1f4d-manifests\") pod \"opendatahub-operator-controller-manager-5fb7f869bf-gsrx5\" (UID: \"4c37b2ca-8721-4147-bf19-eb95bf2d1f4d\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb7f869bf-gsrx5" Jul 22 03:13:11.683970 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.683913 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c37b2ca-8721-4147-bf19-eb95bf2d1f4d-tmp\") pod \"opendatahub-operator-controller-manager-5fb7f869bf-gsrx5\" (UID: \"4c37b2ca-8721-4147-bf19-eb95bf2d1f4d\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb7f869bf-gsrx5" Jul 22 03:13:11.684122 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.684102 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4c37b2ca-8721-4147-bf19-eb95bf2d1f4d-webhook-cert\") pod \"opendatahub-operator-controller-manager-5fb7f869bf-gsrx5\" (UID: \"4c37b2ca-8721-4147-bf19-eb95bf2d1f4d\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb7f869bf-gsrx5" Jul 22 03:13:11.684240 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.684220 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4c37b2ca-8721-4147-bf19-eb95bf2d1f4d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5fb7f869bf-gsrx5\" (UID: \"4c37b2ca-8721-4147-bf19-eb95bf2d1f4d\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb7f869bf-gsrx5" Jul 22 03:13:11.689807 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.689789 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rs2rj\" (UniqueName: \"kubernetes.io/projected/4c37b2ca-8721-4147-bf19-eb95bf2d1f4d-kube-api-access-rs2rj\") pod \"opendatahub-operator-controller-manager-5fb7f869bf-gsrx5\" (UID: \"4c37b2ca-8721-4147-bf19-eb95bf2d1f4d\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb7f869bf-gsrx5" Jul 22 03:13:11.774424 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.774384 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:11.806562 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.806537 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5fb7f869bf-gsrx5" Jul 22 03:13:11.874587 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.874543 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:11.941029 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.940919 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5fb7f869bf-gsrx5"] Jul 22 03:13:11.944961 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:13:11.944921 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c37b2ca_8721_4147_bf19_eb95bf2d1f4d.slice/crio-e494e82b10ee397c99374bfee0f0d285692e0e619c40ad47b38ba8f30b08b1d1 WatchSource:0}: Error finding container e494e82b10ee397c99374bfee0f0d285692e0e619c40ad47b38ba8f30b08b1d1: Status 404 returned error can't find the container with id e494e82b10ee397c99374bfee0f0d285692e0e619c40ad47b38ba8f30b08b1d1 Jul 22 03:13:11.974234 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:11.974196 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:12.081963 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:12.081897 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:12.174659 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:12.174630 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:12.379579 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:12.379541 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5fb7f869bf-gsrx5" event={"ID":"4c37b2ca-8721-4147-bf19-eb95bf2d1f4d","Type":"ContainerStarted","Data":"e494e82b10ee397c99374bfee0f0d285692e0e619c40ad47b38ba8f30b08b1d1"} Jul 22 03:13:13.913173 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:13.913138 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:14.015798 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:14.015686 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:16.400035 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:16.399929 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5fb7f869bf-gsrx5" event={"ID":"4c37b2ca-8721-4147-bf19-eb95bf2d1f4d","Type":"ContainerStarted","Data":"8130b47e8cfc0ddd971af8a4f872e721f06d7b5c3c2df7750d227c4739f56f48"} Jul 22 03:13:17.405306 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:17.405272 2556 generic.go:356] "Generic (PLEG): container finished" podID="4c37b2ca-8721-4147-bf19-eb95bf2d1f4d" containerID="8130b47e8cfc0ddd971af8a4f872e721f06d7b5c3c2df7750d227c4739f56f48" exitCode=0 Jul 22 03:13:17.405733 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:17.405351 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5fb7f869bf-gsrx5" event={"ID":"4c37b2ca-8721-4147-bf19-eb95bf2d1f4d","Type":"ContainerDied","Data":"8130b47e8cfc0ddd971af8a4f872e721f06d7b5c3c2df7750d227c4739f56f48"} Jul 22 03:13:18.416254 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:18.416221 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5fb7f869bf-gsrx5" event={"ID":"4c37b2ca-8721-4147-bf19-eb95bf2d1f4d","Type":"ContainerStarted","Data":"ffccbcce6b8c88e31add53d9dffb2f734b5c8cf4f2cd7f361d2c8de4bdd31415"} Jul 22 03:13:18.416638 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:18.416344 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5fb7f869bf-gsrx5" Jul 22 03:13:18.440345 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:18.440299 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5fb7f869bf-gsrx5" podStartSLOduration=2.478159449 podStartE2EDuration="7.440283384s" totalImagesPullingTime="3.943752129s" totalInitContainerRuntime="1.018371806s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:13:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:13:18.43767845 +0000 UTC m=+338.886661700" watchObservedRunningTime="2026-07-22 03:13:18.440283384 +0000 UTC m=+338.889266633" Jul 22 03:13:22.726270 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:22.726230 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:22.798460 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:22.798421 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:24.711656 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:24.711626 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:24.816375 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:24.816341 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:29.423214 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:29.423178 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5fb7f869bf-gsrx5" Jul 22 03:13:31.123374 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:31.123290 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:31.250352 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:31.250303 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:31.912269 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:31.912235 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:32.017433 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:32.017399 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:32.117553 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:32.117517 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:32.273873 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:32.273780 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:32.601064 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:32.601028 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:32.937557 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:32.937459 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:34.537255 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:34.537220 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:34.555316 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:34.555284 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-59676447d6-27d27"] Jul 22 03:13:34.558831 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:34.558811 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-59676447d6-27d27" Jul 22 03:13:34.561445 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:34.561420 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 22 03:13:34.561445 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:34.561431 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 22 03:13:34.562440 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:34.562422 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 22 03:13:34.562565 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:34.562486 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 22 03:13:34.568816 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:34.568796 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-59676447d6-27d27"] Jul 22 03:13:34.609290 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:34.609255 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:34.677294 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:34.677260 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5pf2\" (UniqueName: \"kubernetes.io/projected/4b59b506-ebea-4869-8530-ff57fdee114b-kube-api-access-x5pf2\") pod \"kube-auth-proxy-59676447d6-27d27\" (UID: \"4b59b506-ebea-4869-8530-ff57fdee114b\") " pod="openshift-ingress/kube-auth-proxy-59676447d6-27d27" Jul 22 03:13:34.677470 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:34.677309 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b59b506-ebea-4869-8530-ff57fdee114b-tls-certs\") pod \"kube-auth-proxy-59676447d6-27d27\" (UID: \"4b59b506-ebea-4869-8530-ff57fdee114b\") " pod="openshift-ingress/kube-auth-proxy-59676447d6-27d27" Jul 22 03:13:34.677470 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:34.677383 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b59b506-ebea-4869-8530-ff57fdee114b-tmp\") pod \"kube-auth-proxy-59676447d6-27d27\" (UID: \"4b59b506-ebea-4869-8530-ff57fdee114b\") " pod="openshift-ingress/kube-auth-proxy-59676447d6-27d27" Jul 22 03:13:34.778502 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:34.778468 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5pf2\" (UniqueName: \"kubernetes.io/projected/4b59b506-ebea-4869-8530-ff57fdee114b-kube-api-access-x5pf2\") pod \"kube-auth-proxy-59676447d6-27d27\" (UID: \"4b59b506-ebea-4869-8530-ff57fdee114b\") " pod="openshift-ingress/kube-auth-proxy-59676447d6-27d27" Jul 22 03:13:34.778502 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:34.778514 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b59b506-ebea-4869-8530-ff57fdee114b-tls-certs\") pod \"kube-auth-proxy-59676447d6-27d27\" (UID: \"4b59b506-ebea-4869-8530-ff57fdee114b\") " pod="openshift-ingress/kube-auth-proxy-59676447d6-27d27" Jul 22 03:13:34.778730 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:34.778562 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b59b506-ebea-4869-8530-ff57fdee114b-tmp\") pod \"kube-auth-proxy-59676447d6-27d27\" (UID: \"4b59b506-ebea-4869-8530-ff57fdee114b\") " pod="openshift-ingress/kube-auth-proxy-59676447d6-27d27" Jul 22 03:13:34.780894 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:34.780871 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b59b506-ebea-4869-8530-ff57fdee114b-tmp\") pod \"kube-auth-proxy-59676447d6-27d27\" (UID: \"4b59b506-ebea-4869-8530-ff57fdee114b\") " pod="openshift-ingress/kube-auth-proxy-59676447d6-27d27" Jul 22 03:13:34.781041 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:34.781024 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b59b506-ebea-4869-8530-ff57fdee114b-tls-certs\") pod \"kube-auth-proxy-59676447d6-27d27\" (UID: \"4b59b506-ebea-4869-8530-ff57fdee114b\") " pod="openshift-ingress/kube-auth-proxy-59676447d6-27d27" Jul 22 03:13:34.787182 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:34.787159 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5pf2\" (UniqueName: \"kubernetes.io/projected/4b59b506-ebea-4869-8530-ff57fdee114b-kube-api-access-x5pf2\") pod \"kube-auth-proxy-59676447d6-27d27\" (UID: \"4b59b506-ebea-4869-8530-ff57fdee114b\") " pod="openshift-ingress/kube-auth-proxy-59676447d6-27d27" Jul 22 03:13:34.868827 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:34.868796 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-59676447d6-27d27" Jul 22 03:13:34.997120 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:34.997099 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-59676447d6-27d27"] Jul 22 03:13:34.999699 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:13:34.999673 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b59b506_ebea_4869_8530_ff57fdee114b.slice/crio-67833ac0e9c5273552b760a2cffe738eb5fef075303b667792bdb9ee8e87d131 WatchSource:0}: Error finding container 67833ac0e9c5273552b760a2cffe738eb5fef075303b667792bdb9ee8e87d131: Status 404 returned error can't find the container with id 67833ac0e9c5273552b760a2cffe738eb5fef075303b667792bdb9ee8e87d131 Jul 22 03:13:35.493410 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:35.493372 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-59676447d6-27d27" event={"ID":"4b59b506-ebea-4869-8530-ff57fdee114b","Type":"ContainerStarted","Data":"67833ac0e9c5273552b760a2cffe738eb5fef075303b667792bdb9ee8e87d131"} Jul 22 03:13:35.861025 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:35.860988 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-h9v54"] Jul 22 03:13:35.863685 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:35.863653 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:35.865579 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:35.865564 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-h9v54" Jul 22 03:13:35.868141 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:35.868117 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 22 03:13:35.868249 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:35.868117 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 22 03:13:35.868340 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:35.868324 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 22 03:13:35.868378 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:35.868322 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-2778r\"" Jul 22 03:13:35.868411 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:35.868325 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 22 03:13:35.886058 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:35.886030 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-h9v54"] Jul 22 03:13:35.946654 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:35.946615 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:35.989466 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:35.989425 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f9a4732-a2e2-4290-a889-7f7a53fd1536-local-certs\") pod \"istiod-openshift-gateway-7574459656-h9v54\" (UID: \"9f9a4732-a2e2-4290-a889-7f7a53fd1536\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h9v54" Jul 22 03:13:35.989645 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:35.989474 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9f9a4732-a2e2-4290-a889-7f7a53fd1536-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-h9v54\" (UID: \"9f9a4732-a2e2-4290-a889-7f7a53fd1536\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h9v54" Jul 22 03:13:35.989645 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:35.989543 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4dlq\" (UniqueName: \"kubernetes.io/projected/9f9a4732-a2e2-4290-a889-7f7a53fd1536-kube-api-access-r4dlq\") pod \"istiod-openshift-gateway-7574459656-h9v54\" (UID: \"9f9a4732-a2e2-4290-a889-7f7a53fd1536\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h9v54" Jul 22 03:13:35.989731 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:35.989706 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f9a4732-a2e2-4290-a889-7f7a53fd1536-istio-token\") pod \"istiod-openshift-gateway-7574459656-h9v54\" (UID: \"9f9a4732-a2e2-4290-a889-7f7a53fd1536\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h9v54" Jul 22 03:13:35.989796 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:35.989771 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9f9a4732-a2e2-4290-a889-7f7a53fd1536-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-h9v54\" (UID: \"9f9a4732-a2e2-4290-a889-7f7a53fd1536\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h9v54" Jul 22 03:13:35.989850 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:35.989807 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9f9a4732-a2e2-4290-a889-7f7a53fd1536-cacerts\") pod \"istiod-openshift-gateway-7574459656-h9v54\" (UID: \"9f9a4732-a2e2-4290-a889-7f7a53fd1536\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h9v54" Jul 22 03:13:35.989850 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:35.989829 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9f9a4732-a2e2-4290-a889-7f7a53fd1536-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-h9v54\" (UID: \"9f9a4732-a2e2-4290-a889-7f7a53fd1536\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h9v54" Jul 22 03:13:36.091243 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:36.091207 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9f9a4732-a2e2-4290-a889-7f7a53fd1536-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-h9v54\" (UID: \"9f9a4732-a2e2-4290-a889-7f7a53fd1536\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h9v54" Jul 22 03:13:36.091425 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:36.091256 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9f9a4732-a2e2-4290-a889-7f7a53fd1536-cacerts\") pod \"istiod-openshift-gateway-7574459656-h9v54\" (UID: \"9f9a4732-a2e2-4290-a889-7f7a53fd1536\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h9v54" Jul 22 03:13:36.091494 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:36.091437 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9f9a4732-a2e2-4290-a889-7f7a53fd1536-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-h9v54\" (UID: \"9f9a4732-a2e2-4290-a889-7f7a53fd1536\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h9v54" Jul 22 03:13:36.091554 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:36.091502 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f9a4732-a2e2-4290-a889-7f7a53fd1536-local-certs\") pod \"istiod-openshift-gateway-7574459656-h9v54\" (UID: \"9f9a4732-a2e2-4290-a889-7f7a53fd1536\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h9v54" Jul 22 03:13:36.091606 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:36.091548 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9f9a4732-a2e2-4290-a889-7f7a53fd1536-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-h9v54\" (UID: \"9f9a4732-a2e2-4290-a889-7f7a53fd1536\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h9v54" Jul 22 03:13:36.091606 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:36.091575 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4dlq\" (UniqueName: \"kubernetes.io/projected/9f9a4732-a2e2-4290-a889-7f7a53fd1536-kube-api-access-r4dlq\") pod \"istiod-openshift-gateway-7574459656-h9v54\" (UID: \"9f9a4732-a2e2-4290-a889-7f7a53fd1536\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h9v54" Jul 22 03:13:36.091715 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:36.091642 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f9a4732-a2e2-4290-a889-7f7a53fd1536-istio-token\") pod \"istiod-openshift-gateway-7574459656-h9v54\" (UID: \"9f9a4732-a2e2-4290-a889-7f7a53fd1536\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h9v54" Jul 22 03:13:36.092354 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:36.092301 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9f9a4732-a2e2-4290-a889-7f7a53fd1536-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-h9v54\" (UID: \"9f9a4732-a2e2-4290-a889-7f7a53fd1536\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h9v54" Jul 22 03:13:36.094234 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:36.094205 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9f9a4732-a2e2-4290-a889-7f7a53fd1536-cacerts\") pod \"istiod-openshift-gateway-7574459656-h9v54\" (UID: \"9f9a4732-a2e2-4290-a889-7f7a53fd1536\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h9v54" Jul 22 03:13:36.094399 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:36.094377 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f9a4732-a2e2-4290-a889-7f7a53fd1536-local-certs\") pod \"istiod-openshift-gateway-7574459656-h9v54\" (UID: \"9f9a4732-a2e2-4290-a889-7f7a53fd1536\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h9v54" Jul 22 03:13:36.094456 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:36.094381 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9f9a4732-a2e2-4290-a889-7f7a53fd1536-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-h9v54\" (UID: \"9f9a4732-a2e2-4290-a889-7f7a53fd1536\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h9v54" Jul 22 03:13:36.094572 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:36.094552 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9f9a4732-a2e2-4290-a889-7f7a53fd1536-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-h9v54\" (UID: \"9f9a4732-a2e2-4290-a889-7f7a53fd1536\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h9v54" Jul 22 03:13:36.102168 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:36.102143 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f9a4732-a2e2-4290-a889-7f7a53fd1536-istio-token\") pod \"istiod-openshift-gateway-7574459656-h9v54\" (UID: \"9f9a4732-a2e2-4290-a889-7f7a53fd1536\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h9v54" Jul 22 03:13:36.102168 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:36.102159 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4dlq\" (UniqueName: \"kubernetes.io/projected/9f9a4732-a2e2-4290-a889-7f7a53fd1536-kube-api-access-r4dlq\") pod \"istiod-openshift-gateway-7574459656-h9v54\" (UID: \"9f9a4732-a2e2-4290-a889-7f7a53fd1536\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h9v54" Jul 22 03:13:36.174995 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:36.174906 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-h9v54" Jul 22 03:13:36.323241 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:36.323218 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-h9v54"] Jul 22 03:13:36.324902 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:13:36.324857 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f9a4732_a2e2_4290_a889_7f7a53fd1536.slice/crio-25bd9825a54935d4ef4b45d454f8b7196310e3b2f6a69d45d96319cac7f63ae6 WatchSource:0}: Error finding container 25bd9825a54935d4ef4b45d454f8b7196310e3b2f6a69d45d96319cac7f63ae6: Status 404 returned error can't find the container with id 25bd9825a54935d4ef4b45d454f8b7196310e3b2f6a69d45d96319cac7f63ae6 Jul 22 03:13:36.501685 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:36.501588 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-h9v54" event={"ID":"9f9a4732-a2e2-4290-a889-7f7a53fd1536","Type":"ContainerStarted","Data":"25bd9825a54935d4ef4b45d454f8b7196310e3b2f6a69d45d96319cac7f63ae6"} Jul 22 03:13:36.980116 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:36.980074 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:38.515816 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:38.515726 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-59676447d6-27d27" event={"ID":"4b59b506-ebea-4869-8530-ff57fdee114b","Type":"ContainerStarted","Data":"761524d4138d753f01ff2b78e00b58e37930e39dc54874c3b26c22d20fe4a0ee"} Jul 22 03:13:38.554547 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:38.554491 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-59676447d6-27d27" podStartSLOduration=1.348501997 podStartE2EDuration="4.554475207s" totalImagesPullingTime="3.20597321s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:13:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:13:38.552376316 +0000 UTC m=+359.001359566" watchObservedRunningTime="2026-07-22 03:13:38.554475207 +0000 UTC m=+359.003458459" Jul 22 03:13:39.261272 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:39.261225 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:13:39.261408 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:39.261310 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:13:39.522467 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:39.522386 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-h9v54" event={"ID":"9f9a4732-a2e2-4290-a889-7f7a53fd1536","Type":"ContainerStarted","Data":"4287067cae22859ab2545572f52f030fc23631b8321fbb7f25f5b72e354cd049"} Jul 22 03:13:39.522467 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:39.522463 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-h9v54" Jul 22 03:13:39.558853 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:39.558771 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-h9v54" podStartSLOduration=1.624774483 podStartE2EDuration="4.558749565s" totalImagesPullingTime="2.933975082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:13:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:13:39.557246592 +0000 UTC m=+360.006229842" watchObservedRunningTime="2026-07-22 03:13:39.558749565 +0000 UTC m=+360.007732815" Jul 22 03:13:40.221710 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:40.221668 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:40.336682 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:40.336639 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:40.528903 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:40.528823 2556 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-h9v54 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 03:13:40.528903 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:40.528884 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-h9v54" podUID="9f9a4732-a2e2-4290-a889-7f7a53fd1536" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 03:13:40.779557 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:40.779475 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n"] Jul 22 03:13:40.784595 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:40.784573 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n" Jul 22 03:13:40.787822 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:40.787796 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-s49r2\"" Jul 22 03:13:40.796278 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:40.796233 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n"] Jul 22 03:13:40.808729 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:40.808696 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:40.930342 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:40.930312 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e03f9ba-5161-404d-998b-c0dd42f50790-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45mh7n\" (UID: \"7e03f9ba-5161-404d-998b-c0dd42f50790\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n" Jul 22 03:13:40.930525 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:40.930368 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lcxb\" (UniqueName: \"kubernetes.io/projected/7e03f9ba-5161-404d-998b-c0dd42f50790-kube-api-access-7lcxb\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45mh7n\" (UID: \"7e03f9ba-5161-404d-998b-c0dd42f50790\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n" Jul 22 03:13:40.930525 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:40.930397 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e03f9ba-5161-404d-998b-c0dd42f50790-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45mh7n\" (UID: \"7e03f9ba-5161-404d-998b-c0dd42f50790\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n" Jul 22 03:13:40.930525 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:40.930420 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e03f9ba-5161-404d-998b-c0dd42f50790-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45mh7n\" (UID: \"7e03f9ba-5161-404d-998b-c0dd42f50790\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n" Jul 22 03:13:40.930525 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:40.930474 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e03f9ba-5161-404d-998b-c0dd42f50790-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45mh7n\" (UID: \"7e03f9ba-5161-404d-998b-c0dd42f50790\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n" Jul 22 03:13:40.930525 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:40.930501 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e03f9ba-5161-404d-998b-c0dd42f50790-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45mh7n\" (UID: \"7e03f9ba-5161-404d-998b-c0dd42f50790\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n" Jul 22 03:13:40.930758 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:40.930533 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e03f9ba-5161-404d-998b-c0dd42f50790-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45mh7n\" (UID: \"7e03f9ba-5161-404d-998b-c0dd42f50790\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n" Jul 22 03:13:40.930758 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:40.930577 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e03f9ba-5161-404d-998b-c0dd42f50790-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45mh7n\" (UID: \"7e03f9ba-5161-404d-998b-c0dd42f50790\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n" Jul 22 03:13:40.930758 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:40.930607 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e03f9ba-5161-404d-998b-c0dd42f50790-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45mh7n\" (UID: \"7e03f9ba-5161-404d-998b-c0dd42f50790\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n" Jul 22 03:13:41.031377 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:41.031292 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e03f9ba-5161-404d-998b-c0dd42f50790-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45mh7n\" (UID: \"7e03f9ba-5161-404d-998b-c0dd42f50790\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n" Jul 22 03:13:41.031377 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:41.031333 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lcxb\" (UniqueName: \"kubernetes.io/projected/7e03f9ba-5161-404d-998b-c0dd42f50790-kube-api-access-7lcxb\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45mh7n\" (UID: \"7e03f9ba-5161-404d-998b-c0dd42f50790\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n" Jul 22 03:13:41.031377 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:41.031353 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e03f9ba-5161-404d-998b-c0dd42f50790-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45mh7n\" (UID: \"7e03f9ba-5161-404d-998b-c0dd42f50790\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n" Jul 22 03:13:41.031645 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:41.031444 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e03f9ba-5161-404d-998b-c0dd42f50790-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45mh7n\" (UID: \"7e03f9ba-5161-404d-998b-c0dd42f50790\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n" Jul 22 03:13:41.031645 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:41.031473 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e03f9ba-5161-404d-998b-c0dd42f50790-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45mh7n\" (UID: \"7e03f9ba-5161-404d-998b-c0dd42f50790\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n" Jul 22 03:13:41.031645 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:41.031496 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e03f9ba-5161-404d-998b-c0dd42f50790-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45mh7n\" (UID: \"7e03f9ba-5161-404d-998b-c0dd42f50790\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n" Jul 22 03:13:41.031645 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:41.031522 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e03f9ba-5161-404d-998b-c0dd42f50790-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45mh7n\" (UID: \"7e03f9ba-5161-404d-998b-c0dd42f50790\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n" Jul 22 03:13:41.031645 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:41.031588 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e03f9ba-5161-404d-998b-c0dd42f50790-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45mh7n\" (UID: \"7e03f9ba-5161-404d-998b-c0dd42f50790\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n" Jul 22 03:13:41.031645 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:41.031617 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e03f9ba-5161-404d-998b-c0dd42f50790-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45mh7n\" (UID: \"7e03f9ba-5161-404d-998b-c0dd42f50790\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n" Jul 22 03:13:41.031930 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:41.031907 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e03f9ba-5161-404d-998b-c0dd42f50790-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45mh7n\" (UID: \"7e03f9ba-5161-404d-998b-c0dd42f50790\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n" Jul 22 03:13:41.031930 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:41.031916 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e03f9ba-5161-404d-998b-c0dd42f50790-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45mh7n\" (UID: \"7e03f9ba-5161-404d-998b-c0dd42f50790\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n" Jul 22 03:13:41.032087 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:41.032061 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e03f9ba-5161-404d-998b-c0dd42f50790-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45mh7n\" (UID: \"7e03f9ba-5161-404d-998b-c0dd42f50790\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n" Jul 22 03:13:41.032229 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:41.032205 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e03f9ba-5161-404d-998b-c0dd42f50790-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45mh7n\" (UID: \"7e03f9ba-5161-404d-998b-c0dd42f50790\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n" Jul 22 03:13:41.032678 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:41.032657 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e03f9ba-5161-404d-998b-c0dd42f50790-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45mh7n\" (UID: \"7e03f9ba-5161-404d-998b-c0dd42f50790\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n" Jul 22 03:13:41.033629 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:41.033613 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e03f9ba-5161-404d-998b-c0dd42f50790-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45mh7n\" (UID: \"7e03f9ba-5161-404d-998b-c0dd42f50790\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n" Jul 22 03:13:41.033910 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:41.033891 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e03f9ba-5161-404d-998b-c0dd42f50790-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45mh7n\" (UID: \"7e03f9ba-5161-404d-998b-c0dd42f50790\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n" Jul 22 03:13:41.048638 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:41.048615 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e03f9ba-5161-404d-998b-c0dd42f50790-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45mh7n\" (UID: \"7e03f9ba-5161-404d-998b-c0dd42f50790\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n" Jul 22 03:13:41.049026 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:41.049005 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lcxb\" (UniqueName: \"kubernetes.io/projected/7e03f9ba-5161-404d-998b-c0dd42f50790-kube-api-access-7lcxb\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45mh7n\" (UID: \"7e03f9ba-5161-404d-998b-c0dd42f50790\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n" Jul 22 03:13:41.087307 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:41.087283 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:41.097132 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:41.097106 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n" Jul 22 03:13:41.247834 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:41.247801 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n"] Jul 22 03:13:41.251386 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:13:41.251349 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e03f9ba_5161_404d_998b_c0dd42f50790.slice/crio-8d5a4ccc197ee00640e5a017e5975a4e9ee0d685f19c02e8a6ae7e3743b8a6c2 WatchSource:0}: Error finding container 8d5a4ccc197ee00640e5a017e5975a4e9ee0d685f19c02e8a6ae7e3743b8a6c2: Status 404 returned error can't find the container with id 8d5a4ccc197ee00640e5a017e5975a4e9ee0d685f19c02e8a6ae7e3743b8a6c2 Jul 22 03:13:41.345814 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:41.345647 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:41.532633 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:41.532603 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n" event={"ID":"7e03f9ba-5161-404d-998b-c0dd42f50790","Type":"ContainerStarted","Data":"8d5a4ccc197ee00640e5a017e5975a4e9ee0d685f19c02e8a6ae7e3743b8a6c2"} Jul 22 03:13:42.110529 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:42.110488 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:42.181222 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:42.181188 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4prjh"] Jul 22 03:13:42.184077 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:42.184054 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4prjh" Jul 22 03:13:42.187969 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:42.187894 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-nnrgm"] Jul 22 03:13:42.188405 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:42.188189 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 22 03:13:42.191089 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:42.191067 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nnrgm" Jul 22 03:13:42.197173 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:42.197156 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-gmfgg\"" Jul 22 03:13:42.198105 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:42.198080 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-qths6\"" Jul 22 03:13:42.203345 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:42.203322 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 22 03:13:42.209210 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:42.209095 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4prjh"] Jul 22 03:13:42.211289 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:42.211265 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-nnrgm"] Jul 22 03:13:42.244695 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:42.244653 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/63db064d-1b10-4979-a48c-0e93223b5304-cert\") pod \"kserve-controller-manager-d6f4b65b8-4prjh\" (UID: \"63db064d-1b10-4979-a48c-0e93223b5304\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4prjh" Jul 22 03:13:42.244870 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:42.244747 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62jjg\" (UniqueName: \"kubernetes.io/projected/63db064d-1b10-4979-a48c-0e93223b5304-kube-api-access-62jjg\") pod \"kserve-controller-manager-d6f4b65b8-4prjh\" (UID: \"63db064d-1b10-4979-a48c-0e93223b5304\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4prjh" Jul 22 03:13:42.244870 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:42.244808 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqnzt\" (UniqueName: \"kubernetes.io/projected/612328a2-521a-44d1-aa8a-31785bdd3998-kube-api-access-cqnzt\") pod \"llmisvc-controller-manager-777c6bd7d5-nnrgm\" (UID: \"612328a2-521a-44d1-aa8a-31785bdd3998\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nnrgm" Jul 22 03:13:42.244870 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:42.244841 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/612328a2-521a-44d1-aa8a-31785bdd3998-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-nnrgm\" (UID: \"612328a2-521a-44d1-aa8a-31785bdd3998\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nnrgm" Jul 22 03:13:42.330931 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:42.330890 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:42.345818 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:42.345781 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqnzt\" (UniqueName: \"kubernetes.io/projected/612328a2-521a-44d1-aa8a-31785bdd3998-kube-api-access-cqnzt\") pod \"llmisvc-controller-manager-777c6bd7d5-nnrgm\" (UID: \"612328a2-521a-44d1-aa8a-31785bdd3998\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nnrgm" Jul 22 03:13:42.346021 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:42.345839 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/612328a2-521a-44d1-aa8a-31785bdd3998-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-nnrgm\" (UID: \"612328a2-521a-44d1-aa8a-31785bdd3998\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nnrgm" Jul 22 03:13:42.346021 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:42.345883 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/63db064d-1b10-4979-a48c-0e93223b5304-cert\") pod \"kserve-controller-manager-d6f4b65b8-4prjh\" (UID: \"63db064d-1b10-4979-a48c-0e93223b5304\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4prjh" Jul 22 03:13:42.346021 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:42.345994 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62jjg\" (UniqueName: \"kubernetes.io/projected/63db064d-1b10-4979-a48c-0e93223b5304-kube-api-access-62jjg\") pod \"kserve-controller-manager-d6f4b65b8-4prjh\" (UID: \"63db064d-1b10-4979-a48c-0e93223b5304\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4prjh" Jul 22 03:13:42.346210 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:13:42.346055 2556 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 22 03:13:42.346210 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:13:42.346134 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63db064d-1b10-4979-a48c-0e93223b5304-cert podName:63db064d-1b10-4979-a48c-0e93223b5304 nodeName:}" failed. No retries permitted until 2026-07-22 03:13:42.846112045 +0000 UTC m=+363.295095278 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/63db064d-1b10-4979-a48c-0e93223b5304-cert") pod "kserve-controller-manager-d6f4b65b8-4prjh" (UID: "63db064d-1b10-4979-a48c-0e93223b5304") : secret "kserve-webhook-server-cert" not found Jul 22 03:13:42.348929 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:42.348902 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/612328a2-521a-44d1-aa8a-31785bdd3998-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-nnrgm\" (UID: \"612328a2-521a-44d1-aa8a-31785bdd3998\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nnrgm" Jul 22 03:13:42.364061 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:42.363981 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62jjg\" (UniqueName: \"kubernetes.io/projected/63db064d-1b10-4979-a48c-0e93223b5304-kube-api-access-62jjg\") pod \"kserve-controller-manager-d6f4b65b8-4prjh\" (UID: \"63db064d-1b10-4979-a48c-0e93223b5304\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4prjh" Jul 22 03:13:42.364425 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:42.364377 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqnzt\" (UniqueName: \"kubernetes.io/projected/612328a2-521a-44d1-aa8a-31785bdd3998-kube-api-access-cqnzt\") pod \"llmisvc-controller-manager-777c6bd7d5-nnrgm\" (UID: \"612328a2-521a-44d1-aa8a-31785bdd3998\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nnrgm" Jul 22 03:13:42.442287 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:42.442202 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:42.505924 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:42.505888 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nnrgm" Jul 22 03:13:42.706295 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:42.706271 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-nnrgm"] Jul 22 03:13:42.708970 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:13:42.708917 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod612328a2_521a_44d1_aa8a_31785bdd3998.slice/crio-8d4d06f824e1ff100a46f494be04ccaef6f21c512c50ebd2946f25e2b8d47d6f WatchSource:0}: Error finding container 8d4d06f824e1ff100a46f494be04ccaef6f21c512c50ebd2946f25e2b8d47d6f: Status 404 returned error can't find the container with id 8d4d06f824e1ff100a46f494be04ccaef6f21c512c50ebd2946f25e2b8d47d6f Jul 22 03:13:42.850834 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:42.850780 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/63db064d-1b10-4979-a48c-0e93223b5304-cert\") pod \"kserve-controller-manager-d6f4b65b8-4prjh\" (UID: \"63db064d-1b10-4979-a48c-0e93223b5304\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4prjh" Jul 22 03:13:42.853601 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:42.853574 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/63db064d-1b10-4979-a48c-0e93223b5304-cert\") pod \"kserve-controller-manager-d6f4b65b8-4prjh\" (UID: \"63db064d-1b10-4979-a48c-0e93223b5304\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4prjh" Jul 22 03:13:42.860081 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:42.860053 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:43.098079 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:43.098045 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4prjh" Jul 22 03:13:43.256791 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:43.256769 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4prjh"] Jul 22 03:13:43.431201 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:43.431126 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:43.528918 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:43.528881 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-h9v54" Jul 22 03:13:43.546099 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:43.546062 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nnrgm" event={"ID":"612328a2-521a-44d1-aa8a-31785bdd3998","Type":"ContainerStarted","Data":"8d4d06f824e1ff100a46f494be04ccaef6f21c512c50ebd2946f25e2b8d47d6f"} Jul 22 03:13:43.607524 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:43.607487 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:43.717987 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:43.717869 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:44.166903 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:13:44.166862 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63db064d_1b10_4979_a48c_0e93223b5304.slice/crio-18a8d28509542bc431e6034c4a1bd39251a064ce8a572e18ba657f38631d6bc4 WatchSource:0}: Error finding container 18a8d28509542bc431e6034c4a1bd39251a064ce8a572e18ba657f38631d6bc4: Status 404 returned error can't find the container with id 18a8d28509542bc431e6034c4a1bd39251a064ce8a572e18ba657f38631d6bc4 Jul 22 03:13:44.334711 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:44.334665 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:13:44.334890 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:44.334758 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:13:44.334890 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:44.334812 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:13:44.552058 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:44.551988 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4prjh" event={"ID":"63db064d-1b10-4979-a48c-0e93223b5304","Type":"ContainerStarted","Data":"18a8d28509542bc431e6034c4a1bd39251a064ce8a572e18ba657f38631d6bc4"} Jul 22 03:13:44.680743 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:44.680712 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-h9v54"] Jul 22 03:13:44.680971 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:44.680927 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-h9v54" podUID="9f9a4732-a2e2-4290-a889-7f7a53fd1536" containerName="discovery" containerID="cri-o://4287067cae22859ab2545572f52f030fc23631b8321fbb7f25f5b72e354cd049" gracePeriod=30 Jul 22 03:13:45.126965 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:45.126928 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-h9v54" Jul 22 03:13:45.171248 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:45.171205 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r4dlq\" (UniqueName: \"kubernetes.io/projected/9f9a4732-a2e2-4290-a889-7f7a53fd1536-kube-api-access-r4dlq\") pod \"9f9a4732-a2e2-4290-a889-7f7a53fd1536\" (UID: \"9f9a4732-a2e2-4290-a889-7f7a53fd1536\") " Jul 22 03:13:45.171405 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:45.171268 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9f9a4732-a2e2-4290-a889-7f7a53fd1536-istio-kubeconfig\") pod \"9f9a4732-a2e2-4290-a889-7f7a53fd1536\" (UID: \"9f9a4732-a2e2-4290-a889-7f7a53fd1536\") " Jul 22 03:13:45.171405 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:45.171329 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9f9a4732-a2e2-4290-a889-7f7a53fd1536-cacerts\") pod \"9f9a4732-a2e2-4290-a889-7f7a53fd1536\" (UID: \"9f9a4732-a2e2-4290-a889-7f7a53fd1536\") " Jul 22 03:13:45.171405 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:45.171389 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9f9a4732-a2e2-4290-a889-7f7a53fd1536-istio-csr-dns-cert\") pod \"9f9a4732-a2e2-4290-a889-7f7a53fd1536\" (UID: \"9f9a4732-a2e2-4290-a889-7f7a53fd1536\") " Jul 22 03:13:45.171579 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:45.171438 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f9a4732-a2e2-4290-a889-7f7a53fd1536-istio-token\") pod \"9f9a4732-a2e2-4290-a889-7f7a53fd1536\" (UID: \"9f9a4732-a2e2-4290-a889-7f7a53fd1536\") " Jul 22 03:13:45.171579 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:45.171473 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f9a4732-a2e2-4290-a889-7f7a53fd1536-local-certs\") pod \"9f9a4732-a2e2-4290-a889-7f7a53fd1536\" (UID: \"9f9a4732-a2e2-4290-a889-7f7a53fd1536\") " Jul 22 03:13:45.171579 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:45.171498 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9f9a4732-a2e2-4290-a889-7f7a53fd1536-istio-csr-ca-configmap\") pod \"9f9a4732-a2e2-4290-a889-7f7a53fd1536\" (UID: \"9f9a4732-a2e2-4290-a889-7f7a53fd1536\") " Jul 22 03:13:45.173792 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:45.173749 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f9a4732-a2e2-4290-a889-7f7a53fd1536-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "9f9a4732-a2e2-4290-a889-7f7a53fd1536" (UID: "9f9a4732-a2e2-4290-a889-7f7a53fd1536"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:13:45.178046 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:45.178017 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f9a4732-a2e2-4290-a889-7f7a53fd1536-cacerts" (OuterVolumeSpecName: "cacerts") pod "9f9a4732-a2e2-4290-a889-7f7a53fd1536" (UID: "9f9a4732-a2e2-4290-a889-7f7a53fd1536"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:13:45.178345 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:45.178288 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f9a4732-a2e2-4290-a889-7f7a53fd1536-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "9f9a4732-a2e2-4290-a889-7f7a53fd1536" (UID: "9f9a4732-a2e2-4290-a889-7f7a53fd1536"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:13:45.178345 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:45.178309 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f9a4732-a2e2-4290-a889-7f7a53fd1536-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "9f9a4732-a2e2-4290-a889-7f7a53fd1536" (UID: "9f9a4732-a2e2-4290-a889-7f7a53fd1536"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:13:45.179890 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:45.179850 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f9a4732-a2e2-4290-a889-7f7a53fd1536-istio-token" (OuterVolumeSpecName: "istio-token") pod "9f9a4732-a2e2-4290-a889-7f7a53fd1536" (UID: "9f9a4732-a2e2-4290-a889-7f7a53fd1536"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:13:45.181870 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:45.181811 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f9a4732-a2e2-4290-a889-7f7a53fd1536-kube-api-access-r4dlq" (OuterVolumeSpecName: "kube-api-access-r4dlq") pod "9f9a4732-a2e2-4290-a889-7f7a53fd1536" (UID: "9f9a4732-a2e2-4290-a889-7f7a53fd1536"). InnerVolumeSpecName "kube-api-access-r4dlq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:13:45.181870 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:45.181840 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f9a4732-a2e2-4290-a889-7f7a53fd1536-local-certs" (OuterVolumeSpecName: "local-certs") pod "9f9a4732-a2e2-4290-a889-7f7a53fd1536" (UID: "9f9a4732-a2e2-4290-a889-7f7a53fd1536"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:13:45.272730 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:45.272696 2556 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9f9a4732-a2e2-4290-a889-7f7a53fd1536-cacerts\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:13:45.272730 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:45.272728 2556 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9f9a4732-a2e2-4290-a889-7f7a53fd1536-istio-csr-dns-cert\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:13:45.272941 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:45.272742 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f9a4732-a2e2-4290-a889-7f7a53fd1536-istio-token\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:13:45.272941 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:45.272756 2556 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f9a4732-a2e2-4290-a889-7f7a53fd1536-local-certs\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:13:45.272941 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:45.272771 2556 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9f9a4732-a2e2-4290-a889-7f7a53fd1536-istio-csr-ca-configmap\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:13:45.272941 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:45.272786 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4dlq\" (UniqueName: \"kubernetes.io/projected/9f9a4732-a2e2-4290-a889-7f7a53fd1536-kube-api-access-r4dlq\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:13:45.272941 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:45.272802 2556 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9f9a4732-a2e2-4290-a889-7f7a53fd1536-istio-kubeconfig\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:13:45.558700 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:45.558616 2556 generic.go:356] "Generic (PLEG): container finished" podID="9f9a4732-a2e2-4290-a889-7f7a53fd1536" containerID="4287067cae22859ab2545572f52f030fc23631b8321fbb7f25f5b72e354cd049" exitCode=0 Jul 22 03:13:45.558700 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:45.558656 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-h9v54" Jul 22 03:13:45.558920 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:45.558692 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-h9v54" event={"ID":"9f9a4732-a2e2-4290-a889-7f7a53fd1536","Type":"ContainerDied","Data":"4287067cae22859ab2545572f52f030fc23631b8321fbb7f25f5b72e354cd049"} Jul 22 03:13:45.558920 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:45.558744 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-h9v54" event={"ID":"9f9a4732-a2e2-4290-a889-7f7a53fd1536","Type":"ContainerDied","Data":"25bd9825a54935d4ef4b45d454f8b7196310e3b2f6a69d45d96319cac7f63ae6"} Jul 22 03:13:45.558920 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:45.558770 2556 scope.go:117] "RemoveContainer" containerID="4287067cae22859ab2545572f52f030fc23631b8321fbb7f25f5b72e354cd049" Jul 22 03:13:45.560592 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:45.560567 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n" event={"ID":"7e03f9ba-5161-404d-998b-c0dd42f50790","Type":"ContainerStarted","Data":"d91376c2dc43be9487542dac712c6c5ce69be0b51e042a199bc809363b7fa9e8"} Jul 22 03:13:45.570907 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:45.570891 2556 scope.go:117] "RemoveContainer" containerID="4287067cae22859ab2545572f52f030fc23631b8321fbb7f25f5b72e354cd049" Jul 22 03:13:45.571425 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:13:45.571396 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4287067cae22859ab2545572f52f030fc23631b8321fbb7f25f5b72e354cd049\": container with ID starting with 4287067cae22859ab2545572f52f030fc23631b8321fbb7f25f5b72e354cd049 not found: ID does not exist" containerID="4287067cae22859ab2545572f52f030fc23631b8321fbb7f25f5b72e354cd049" Jul 22 03:13:45.571530 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:45.571431 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4287067cae22859ab2545572f52f030fc23631b8321fbb7f25f5b72e354cd049"} err="failed to get container status \"4287067cae22859ab2545572f52f030fc23631b8321fbb7f25f5b72e354cd049\": rpc error: code = NotFound desc = could not find container \"4287067cae22859ab2545572f52f030fc23631b8321fbb7f25f5b72e354cd049\": container with ID starting with 4287067cae22859ab2545572f52f030fc23631b8321fbb7f25f5b72e354cd049 not found: ID does not exist" Jul 22 03:13:45.612554 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:45.612494 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n" podStartSLOduration=2.532748512 podStartE2EDuration="5.612471364s" totalImagesPullingTime="3.079722852s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:13:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:13:45.610025316 +0000 UTC m=+366.059008584" watchObservedRunningTime="2026-07-22 03:13:45.612471364 +0000 UTC m=+366.061454614" Jul 22 03:13:45.679916 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:45.676732 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-h9v54"] Jul 22 03:13:45.689965 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:45.689917 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-h9v54"] Jul 22 03:13:46.026753 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:46.026719 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f9a4732-a2e2-4290-a889-7f7a53fd1536" path="/var/lib/kubelet/pods/9f9a4732-a2e2-4290-a889-7f7a53fd1536/volumes" Jul 22 03:13:46.097685 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:46.097649 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n" Jul 22 03:13:46.102670 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:46.102646 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n" Jul 22 03:13:46.567455 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:46.567424 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n" Jul 22 03:13:46.568451 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:46.568429 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45mh7n" Jul 22 03:13:47.573338 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:47.573304 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nnrgm" event={"ID":"612328a2-521a-44d1-aa8a-31785bdd3998","Type":"ContainerStarted","Data":"df1f839399928bc2b50aeb000aa14f9ee836057464b1ecb6659119922af68a51"} Jul 22 03:13:47.573768 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:47.573371 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nnrgm" Jul 22 03:13:47.574888 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:47.574831 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4prjh" event={"ID":"63db064d-1b10-4979-a48c-0e93223b5304","Type":"ContainerStarted","Data":"826566357f482948592418a5b20c2c1956049d3fd434565944145d7e16b7e011"} Jul 22 03:13:47.575169 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:47.575150 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4prjh" Jul 22 03:13:47.594441 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:47.594403 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nnrgm" podStartSLOduration=0.950269993 podStartE2EDuration="5.594389592s" totalImagesPullingTime="4.644119599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:13:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:13:47.593004042 +0000 UTC m=+368.041987316" watchObservedRunningTime="2026-07-22 03:13:47.594389592 +0000 UTC m=+368.043372841" Jul 22 03:13:47.616241 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:47.616139 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4prjh" podStartSLOduration=2.433504998 podStartE2EDuration="5.616117794s" totalImagesPullingTime="3.182612796s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:13:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:13:47.614305934 +0000 UTC m=+368.063289176" watchObservedRunningTime="2026-07-22 03:13:47.616117794 +0000 UTC m=+368.065101044" Jul 22 03:13:48.328006 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:48.327947 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:48.439722 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:13:48.439670 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:14:12.728083 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:12.728049 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:14:12.829134 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:12.829098 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:14:18.585436 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:18.585402 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nnrgm" Jul 22 03:14:18.588492 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:18.588469 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4prjh" Jul 22 03:14:32.518226 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:32.518172 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:14:32.734199 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:32.734162 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7r6p8"] Jul 22 03:14:32.734501 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:32.734488 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f9a4732-a2e2-4290-a889-7f7a53fd1536" containerName="discovery" Jul 22 03:14:32.734549 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:32.734502 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f9a4732-a2e2-4290-a889-7f7a53fd1536" containerName="discovery" Jul 22 03:14:32.734583 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:32.734570 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f9a4732-a2e2-4290-a889-7f7a53fd1536" containerName="discovery" Jul 22 03:14:32.736519 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:32.736501 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7r6p8" Jul 22 03:14:32.747359 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:32.747339 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-qr6f4\"" Jul 22 03:14:32.759776 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:32.759752 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 22 03:14:32.762472 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:32.762452 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 22 03:14:32.773448 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:32.773392 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfv85\" (UniqueName: \"kubernetes.io/projected/3d6b6f44-7aa1-4bcf-81dc-3233ac01e7aa-kube-api-access-hfv85\") pod \"kuadrant-operator-catalog-7r6p8\" (UID: \"3d6b6f44-7aa1-4bcf-81dc-3233ac01e7aa\") " pod="kuadrant-system/kuadrant-operator-catalog-7r6p8" Jul 22 03:14:32.855081 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:32.855017 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:14:32.855232 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:32.855047 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7r6p8"] Jul 22 03:14:32.873995 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:32.873941 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfv85\" (UniqueName: \"kubernetes.io/projected/3d6b6f44-7aa1-4bcf-81dc-3233ac01e7aa-kube-api-access-hfv85\") pod \"kuadrant-operator-catalog-7r6p8\" (UID: \"3d6b6f44-7aa1-4bcf-81dc-3233ac01e7aa\") " pod="kuadrant-system/kuadrant-operator-catalog-7r6p8" Jul 22 03:14:32.894405 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:32.894371 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfv85\" (UniqueName: \"kubernetes.io/projected/3d6b6f44-7aa1-4bcf-81dc-3233ac01e7aa-kube-api-access-hfv85\") pod \"kuadrant-operator-catalog-7r6p8\" (UID: \"3d6b6f44-7aa1-4bcf-81dc-3233ac01e7aa\") " pod="kuadrant-system/kuadrant-operator-catalog-7r6p8" Jul 22 03:14:32.971733 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:32.971699 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:14:32.999001 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:32.998968 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7r6p8"] Jul 22 03:14:32.999220 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:32.999208 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7r6p8" Jul 22 03:14:33.126429 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:33.126402 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7r6p8"] Jul 22 03:14:33.129503 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:14:33.129478 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d6b6f44_7aa1_4bcf_81dc_3233ac01e7aa.slice/crio-ad97d613105363a3b138478ad29ddae22de3dc19326bf32843854f397503617a WatchSource:0}: Error finding container ad97d613105363a3b138478ad29ddae22de3dc19326bf32843854f397503617a: Status 404 returned error can't find the container with id ad97d613105363a3b138478ad29ddae22de3dc19326bf32843854f397503617a Jul 22 03:14:33.207385 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:33.207353 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wmxqz"] Jul 22 03:14:33.210077 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:33.210061 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-wmxqz" Jul 22 03:14:33.219622 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:33.219596 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wmxqz"] Jul 22 03:14:33.276325 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:33.276293 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cmp9\" (UniqueName: \"kubernetes.io/projected/8094bb22-572a-46e3-8c00-2e3ee71387e4-kube-api-access-2cmp9\") pod \"kuadrant-operator-catalog-wmxqz\" (UID: \"8094bb22-572a-46e3-8c00-2e3ee71387e4\") " pod="kuadrant-system/kuadrant-operator-catalog-wmxqz" Jul 22 03:14:33.376671 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:33.376644 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cmp9\" (UniqueName: \"kubernetes.io/projected/8094bb22-572a-46e3-8c00-2e3ee71387e4-kube-api-access-2cmp9\") pod \"kuadrant-operator-catalog-wmxqz\" (UID: \"8094bb22-572a-46e3-8c00-2e3ee71387e4\") " pod="kuadrant-system/kuadrant-operator-catalog-wmxqz" Jul 22 03:14:33.386452 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:33.386421 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cmp9\" (UniqueName: \"kubernetes.io/projected/8094bb22-572a-46e3-8c00-2e3ee71387e4-kube-api-access-2cmp9\") pod \"kuadrant-operator-catalog-wmxqz\" (UID: \"8094bb22-572a-46e3-8c00-2e3ee71387e4\") " pod="kuadrant-system/kuadrant-operator-catalog-wmxqz" Jul 22 03:14:33.519926 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:33.519897 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-wmxqz" Jul 22 03:14:33.653356 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:33.653307 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wmxqz"] Jul 22 03:14:33.655385 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:14:33.655359 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8094bb22_572a_46e3_8c00_2e3ee71387e4.slice/crio-9364b3ab512c2d2ce433297ddb9f22f3f02aefff6b2f7152f3ab5f3563ef62bd WatchSource:0}: Error finding container 9364b3ab512c2d2ce433297ddb9f22f3f02aefff6b2f7152f3ab5f3563ef62bd: Status 404 returned error can't find the container with id 9364b3ab512c2d2ce433297ddb9f22f3f02aefff6b2f7152f3ab5f3563ef62bd Jul 22 03:14:33.790900 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:33.790860 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-wmxqz" event={"ID":"8094bb22-572a-46e3-8c00-2e3ee71387e4","Type":"ContainerStarted","Data":"9364b3ab512c2d2ce433297ddb9f22f3f02aefff6b2f7152f3ab5f3563ef62bd"} Jul 22 03:14:33.792014 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:33.791988 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7r6p8" event={"ID":"3d6b6f44-7aa1-4bcf-81dc-3233ac01e7aa","Type":"ContainerStarted","Data":"ad97d613105363a3b138478ad29ddae22de3dc19326bf32843854f397503617a"} Jul 22 03:14:33.877259 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:33.877226 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:14:33.958429 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:33.958328 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:14:35.804454 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:35.804414 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7r6p8" event={"ID":"3d6b6f44-7aa1-4bcf-81dc-3233ac01e7aa","Type":"ContainerStarted","Data":"20be0a6f13082c1d9ecb25b740f899a6578229c7784d6e33f4ad0a624f8ee95c"} Jul 22 03:14:35.804891 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:35.804426 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-7r6p8" podUID="3d6b6f44-7aa1-4bcf-81dc-3233ac01e7aa" containerName="registry-server" containerID="cri-o://20be0a6f13082c1d9ecb25b740f899a6578229c7784d6e33f4ad0a624f8ee95c" gracePeriod=2 Jul 22 03:14:35.805920 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:35.805893 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-wmxqz" event={"ID":"8094bb22-572a-46e3-8c00-2e3ee71387e4","Type":"ContainerStarted","Data":"3c644900ab98e349313980f09394c53f01ec22bbedd5c0d1548e2736db97669b"} Jul 22 03:14:35.821879 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:35.821837 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-7r6p8" podStartSLOduration=1.663576441 podStartE2EDuration="3.821822862s" totalImagesPullingTime="2.158246421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:14:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:14:35.82057541 +0000 UTC m=+416.269558660" watchObservedRunningTime="2026-07-22 03:14:35.821822862 +0000 UTC m=+416.270806153" Jul 22 03:14:36.040619 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:36.040595 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7r6p8" Jul 22 03:14:36.060878 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:36.060787 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-wmxqz" podStartSLOduration=1.425756756 podStartE2EDuration="3.060768244s" totalImagesPullingTime="1.635011488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:14:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:14:35.836321229 +0000 UTC m=+416.285304479" watchObservedRunningTime="2026-07-22 03:14:36.060768244 +0000 UTC m=+416.509751493" Jul 22 03:14:36.101437 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:36.101411 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfv85\" (UniqueName: \"kubernetes.io/projected/3d6b6f44-7aa1-4bcf-81dc-3233ac01e7aa-kube-api-access-hfv85\") pod \"3d6b6f44-7aa1-4bcf-81dc-3233ac01e7aa\" (UID: \"3d6b6f44-7aa1-4bcf-81dc-3233ac01e7aa\") " Jul 22 03:14:36.103529 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:36.103504 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d6b6f44-7aa1-4bcf-81dc-3233ac01e7aa-kube-api-access-hfv85" (OuterVolumeSpecName: "kube-api-access-hfv85") pod "3d6b6f44-7aa1-4bcf-81dc-3233ac01e7aa" (UID: "3d6b6f44-7aa1-4bcf-81dc-3233ac01e7aa"). InnerVolumeSpecName "kube-api-access-hfv85". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:14:36.202094 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:36.202068 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfv85\" (UniqueName: \"kubernetes.io/projected/3d6b6f44-7aa1-4bcf-81dc-3233ac01e7aa-kube-api-access-hfv85\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:14:36.811936 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:36.811897 2556 generic.go:356] "Generic (PLEG): container finished" podID="3d6b6f44-7aa1-4bcf-81dc-3233ac01e7aa" containerID="20be0a6f13082c1d9ecb25b740f899a6578229c7784d6e33f4ad0a624f8ee95c" exitCode=0 Jul 22 03:14:36.812354 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:36.811941 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7r6p8" event={"ID":"3d6b6f44-7aa1-4bcf-81dc-3233ac01e7aa","Type":"ContainerDied","Data":"20be0a6f13082c1d9ecb25b740f899a6578229c7784d6e33f4ad0a624f8ee95c"} Jul 22 03:14:36.812354 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:36.811976 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7r6p8" Jul 22 03:14:36.812354 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:36.812034 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7r6p8" event={"ID":"3d6b6f44-7aa1-4bcf-81dc-3233ac01e7aa","Type":"ContainerDied","Data":"ad97d613105363a3b138478ad29ddae22de3dc19326bf32843854f397503617a"} Jul 22 03:14:36.812354 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:36.812057 2556 scope.go:117] "RemoveContainer" containerID="20be0a6f13082c1d9ecb25b740f899a6578229c7784d6e33f4ad0a624f8ee95c" Jul 22 03:14:36.821009 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:36.820994 2556 scope.go:117] "RemoveContainer" containerID="20be0a6f13082c1d9ecb25b740f899a6578229c7784d6e33f4ad0a624f8ee95c" Jul 22 03:14:36.821260 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:14:36.821239 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20be0a6f13082c1d9ecb25b740f899a6578229c7784d6e33f4ad0a624f8ee95c\": container with ID starting with 20be0a6f13082c1d9ecb25b740f899a6578229c7784d6e33f4ad0a624f8ee95c not found: ID does not exist" containerID="20be0a6f13082c1d9ecb25b740f899a6578229c7784d6e33f4ad0a624f8ee95c" Jul 22 03:14:36.821310 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:36.821267 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20be0a6f13082c1d9ecb25b740f899a6578229c7784d6e33f4ad0a624f8ee95c"} err="failed to get container status \"20be0a6f13082c1d9ecb25b740f899a6578229c7784d6e33f4ad0a624f8ee95c\": rpc error: code = NotFound desc = could not find container \"20be0a6f13082c1d9ecb25b740f899a6578229c7784d6e33f4ad0a624f8ee95c\": container with ID starting with 20be0a6f13082c1d9ecb25b740f899a6578229c7784d6e33f4ad0a624f8ee95c not found: ID does not exist" Jul 22 03:14:36.836385 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:36.836360 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7r6p8"] Jul 22 03:14:36.848197 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:36.848173 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7r6p8"] Jul 22 03:14:38.023484 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:38.023450 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d6b6f44-7aa1-4bcf-81dc-3233ac01e7aa" path="/var/lib/kubelet/pods/3d6b6f44-7aa1-4bcf-81dc-3233ac01e7aa/volumes" Jul 22 03:14:43.520586 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:43.520541 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-wmxqz" Jul 22 03:14:43.521191 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:43.520615 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-wmxqz" Jul 22 03:14:43.543278 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:43.543254 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-wmxqz" Jul 22 03:14:43.867248 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:43.867218 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-wmxqz" Jul 22 03:14:48.444914 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:48.444878 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rwqdw"] Jul 22 03:14:48.445296 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:48.445208 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d6b6f44-7aa1-4bcf-81dc-3233ac01e7aa" containerName="registry-server" Jul 22 03:14:48.445296 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:48.445220 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d6b6f44-7aa1-4bcf-81dc-3233ac01e7aa" containerName="registry-server" Jul 22 03:14:48.445296 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:48.445286 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d6b6f44-7aa1-4bcf-81dc-3233ac01e7aa" containerName="registry-server" Jul 22 03:14:48.448174 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:48.448152 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rwqdw" Jul 22 03:14:48.450873 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:48.450850 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-2pb9b\"" Jul 22 03:14:48.457570 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:48.457547 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rwqdw"] Jul 22 03:14:48.501977 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:48.501928 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d7095a2-1171-4232-8838-b98a0848a484-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rwqdw\" (UID: \"4d7095a2-1171-4232-8838-b98a0848a484\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rwqdw" Jul 22 03:14:48.502122 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:48.501987 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d7095a2-1171-4232-8838-b98a0848a484-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rwqdw\" (UID: \"4d7095a2-1171-4232-8838-b98a0848a484\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rwqdw" Jul 22 03:14:48.502122 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:48.502089 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcdr7\" (UniqueName: \"kubernetes.io/projected/4d7095a2-1171-4232-8838-b98a0848a484-kube-api-access-qcdr7\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rwqdw\" (UID: \"4d7095a2-1171-4232-8838-b98a0848a484\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rwqdw" Jul 22 03:14:48.602540 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:48.602504 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d7095a2-1171-4232-8838-b98a0848a484-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rwqdw\" (UID: \"4d7095a2-1171-4232-8838-b98a0848a484\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rwqdw" Jul 22 03:14:48.602540 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:48.602548 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d7095a2-1171-4232-8838-b98a0848a484-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rwqdw\" (UID: \"4d7095a2-1171-4232-8838-b98a0848a484\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rwqdw" Jul 22 03:14:48.602781 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:48.602611 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcdr7\" (UniqueName: \"kubernetes.io/projected/4d7095a2-1171-4232-8838-b98a0848a484-kube-api-access-qcdr7\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rwqdw\" (UID: \"4d7095a2-1171-4232-8838-b98a0848a484\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rwqdw" Jul 22 03:14:48.602972 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:48.602926 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d7095a2-1171-4232-8838-b98a0848a484-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rwqdw\" (UID: \"4d7095a2-1171-4232-8838-b98a0848a484\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rwqdw" Jul 22 03:14:48.603059 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:48.603017 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d7095a2-1171-4232-8838-b98a0848a484-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rwqdw\" (UID: \"4d7095a2-1171-4232-8838-b98a0848a484\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rwqdw" Jul 22 03:14:48.611509 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:48.611476 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcdr7\" (UniqueName: \"kubernetes.io/projected/4d7095a2-1171-4232-8838-b98a0848a484-kube-api-access-qcdr7\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rwqdw\" (UID: \"4d7095a2-1171-4232-8838-b98a0848a484\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rwqdw" Jul 22 03:14:48.758178 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:48.758104 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rwqdw" Jul 22 03:14:48.905545 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:48.905517 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rwqdw"] Jul 22 03:14:48.906872 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:14:48.906846 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d7095a2_1171_4232_8838_b98a0848a484.slice/crio-864f2a9ba3bd2702db022818497a26e1dadf3e869ce8b3765b0c062d38c5541c WatchSource:0}: Error finding container 864f2a9ba3bd2702db022818497a26e1dadf3e869ce8b3765b0c062d38c5541c: Status 404 returned error can't find the container with id 864f2a9ba3bd2702db022818497a26e1dadf3e869ce8b3765b0c062d38c5541c Jul 22 03:14:49.049385 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:49.049319 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fhnsh"] Jul 22 03:14:49.051616 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:49.051596 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fhnsh" Jul 22 03:14:49.061348 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:49.061327 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fhnsh"] Jul 22 03:14:49.107174 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:49.107146 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6q4pw\" (UniqueName: \"kubernetes.io/projected/80d312a3-2e8b-47ee-9a27-bab66ba1a670-kube-api-access-6q4pw\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fhnsh\" (UID: \"80d312a3-2e8b-47ee-9a27-bab66ba1a670\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fhnsh" Jul 22 03:14:49.107303 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:49.107189 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/80d312a3-2e8b-47ee-9a27-bab66ba1a670-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fhnsh\" (UID: \"80d312a3-2e8b-47ee-9a27-bab66ba1a670\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fhnsh" Jul 22 03:14:49.107303 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:49.107242 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/80d312a3-2e8b-47ee-9a27-bab66ba1a670-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fhnsh\" (UID: \"80d312a3-2e8b-47ee-9a27-bab66ba1a670\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fhnsh" Jul 22 03:14:49.207753 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:49.207725 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6q4pw\" (UniqueName: \"kubernetes.io/projected/80d312a3-2e8b-47ee-9a27-bab66ba1a670-kube-api-access-6q4pw\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fhnsh\" (UID: \"80d312a3-2e8b-47ee-9a27-bab66ba1a670\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fhnsh" Jul 22 03:14:49.207891 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:49.207765 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/80d312a3-2e8b-47ee-9a27-bab66ba1a670-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fhnsh\" (UID: \"80d312a3-2e8b-47ee-9a27-bab66ba1a670\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fhnsh" Jul 22 03:14:49.207936 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:49.207885 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/80d312a3-2e8b-47ee-9a27-bab66ba1a670-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fhnsh\" (UID: \"80d312a3-2e8b-47ee-9a27-bab66ba1a670\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fhnsh" Jul 22 03:14:49.208092 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:49.208077 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/80d312a3-2e8b-47ee-9a27-bab66ba1a670-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fhnsh\" (UID: \"80d312a3-2e8b-47ee-9a27-bab66ba1a670\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fhnsh" Jul 22 03:14:49.208192 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:49.208177 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/80d312a3-2e8b-47ee-9a27-bab66ba1a670-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fhnsh\" (UID: \"80d312a3-2e8b-47ee-9a27-bab66ba1a670\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fhnsh" Jul 22 03:14:49.216117 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:49.216096 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6q4pw\" (UniqueName: \"kubernetes.io/projected/80d312a3-2e8b-47ee-9a27-bab66ba1a670-kube-api-access-6q4pw\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fhnsh\" (UID: \"80d312a3-2e8b-47ee-9a27-bab66ba1a670\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fhnsh" Jul 22 03:14:49.360840 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:49.360808 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fhnsh" Jul 22 03:14:49.498535 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:49.498505 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fhnsh"] Jul 22 03:14:49.501435 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:14:49.501407 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80d312a3_2e8b_47ee_9a27_bab66ba1a670.slice/crio-049e4ba712d65a00f5c2d86cde6d8f9beaa87d72b20cfe6616a9b2aab3cb170a WatchSource:0}: Error finding container 049e4ba712d65a00f5c2d86cde6d8f9beaa87d72b20cfe6616a9b2aab3cb170a: Status 404 returned error can't find the container with id 049e4ba712d65a00f5c2d86cde6d8f9beaa87d72b20cfe6616a9b2aab3cb170a Jul 22 03:14:49.664586 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:49.664556 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lgx6j"] Jul 22 03:14:49.666840 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:49.666825 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lgx6j" Jul 22 03:14:49.678919 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:49.678897 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lgx6j"] Jul 22 03:14:49.710915 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:49.710890 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eee6f75e-6170-48c2-ac4f-8e340679dde2-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lgx6j\" (UID: \"eee6f75e-6170-48c2-ac4f-8e340679dde2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lgx6j" Jul 22 03:14:49.711064 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:49.710983 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52pt5\" (UniqueName: \"kubernetes.io/projected/eee6f75e-6170-48c2-ac4f-8e340679dde2-kube-api-access-52pt5\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lgx6j\" (UID: \"eee6f75e-6170-48c2-ac4f-8e340679dde2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lgx6j" Jul 22 03:14:49.711064 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:49.711057 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eee6f75e-6170-48c2-ac4f-8e340679dde2-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lgx6j\" (UID: \"eee6f75e-6170-48c2-ac4f-8e340679dde2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lgx6j" Jul 22 03:14:49.812312 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:49.812285 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eee6f75e-6170-48c2-ac4f-8e340679dde2-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lgx6j\" (UID: \"eee6f75e-6170-48c2-ac4f-8e340679dde2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lgx6j" Jul 22 03:14:49.812456 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:49.812338 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eee6f75e-6170-48c2-ac4f-8e340679dde2-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lgx6j\" (UID: \"eee6f75e-6170-48c2-ac4f-8e340679dde2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lgx6j" Jul 22 03:14:49.812456 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:49.812376 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52pt5\" (UniqueName: \"kubernetes.io/projected/eee6f75e-6170-48c2-ac4f-8e340679dde2-kube-api-access-52pt5\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lgx6j\" (UID: \"eee6f75e-6170-48c2-ac4f-8e340679dde2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lgx6j" Jul 22 03:14:49.812652 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:49.812633 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eee6f75e-6170-48c2-ac4f-8e340679dde2-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lgx6j\" (UID: \"eee6f75e-6170-48c2-ac4f-8e340679dde2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lgx6j" Jul 22 03:14:49.812721 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:49.812701 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eee6f75e-6170-48c2-ac4f-8e340679dde2-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lgx6j\" (UID: \"eee6f75e-6170-48c2-ac4f-8e340679dde2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lgx6j" Jul 22 03:14:49.821537 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:49.821510 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52pt5\" (UniqueName: \"kubernetes.io/projected/eee6f75e-6170-48c2-ac4f-8e340679dde2-kube-api-access-52pt5\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lgx6j\" (UID: \"eee6f75e-6170-48c2-ac4f-8e340679dde2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lgx6j" Jul 22 03:14:49.879240 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:49.879152 2556 generic.go:356] "Generic (PLEG): container finished" podID="80d312a3-2e8b-47ee-9a27-bab66ba1a670" containerID="4557c3309c4e45cac04b20b1f59b8457d79535e7a8f0a07dbc22b5b8cf068f29" exitCode=0 Jul 22 03:14:49.879393 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:49.879239 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fhnsh" event={"ID":"80d312a3-2e8b-47ee-9a27-bab66ba1a670","Type":"ContainerDied","Data":"4557c3309c4e45cac04b20b1f59b8457d79535e7a8f0a07dbc22b5b8cf068f29"} Jul 22 03:14:49.879393 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:49.879317 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fhnsh" event={"ID":"80d312a3-2e8b-47ee-9a27-bab66ba1a670","Type":"ContainerStarted","Data":"049e4ba712d65a00f5c2d86cde6d8f9beaa87d72b20cfe6616a9b2aab3cb170a"} Jul 22 03:14:49.880872 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:49.880851 2556 generic.go:356] "Generic (PLEG): container finished" podID="4d7095a2-1171-4232-8838-b98a0848a484" containerID="bef363e522b409a02981c98adf88b10dae204b2b6b587983ab74a9961e254726" exitCode=0 Jul 22 03:14:49.881006 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:49.880877 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rwqdw" event={"ID":"4d7095a2-1171-4232-8838-b98a0848a484","Type":"ContainerDied","Data":"bef363e522b409a02981c98adf88b10dae204b2b6b587983ab74a9961e254726"} Jul 22 03:14:49.881006 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:49.880903 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rwqdw" event={"ID":"4d7095a2-1171-4232-8838-b98a0848a484","Type":"ContainerStarted","Data":"864f2a9ba3bd2702db022818497a26e1dadf3e869ce8b3765b0c062d38c5541c"} Jul 22 03:14:49.975551 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:49.975517 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lgx6j" Jul 22 03:14:50.105363 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:50.105340 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lgx6j"] Jul 22 03:14:50.107551 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:14:50.107528 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeee6f75e_6170_48c2_ac4f_8e340679dde2.slice/crio-b4d2c4d098e9b6be1e7a27ca5a2f1741a896af90d9d639b24997766ad66c40d4 WatchSource:0}: Error finding container b4d2c4d098e9b6be1e7a27ca5a2f1741a896af90d9d639b24997766ad66c40d4: Status 404 returned error can't find the container with id b4d2c4d098e9b6be1e7a27ca5a2f1741a896af90d9d639b24997766ad66c40d4 Jul 22 03:14:50.256076 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:50.256048 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q6hgs"] Jul 22 03:14:50.258374 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:50.258358 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q6hgs" Jul 22 03:14:50.268433 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:50.268411 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q6hgs"] Jul 22 03:14:50.315429 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:50.315397 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/49081396-f915-404a-a95a-1625bba5456b-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q6hgs\" (UID: \"49081396-f915-404a-a95a-1625bba5456b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q6hgs" Jul 22 03:14:50.315586 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:50.315445 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtqgr\" (UniqueName: \"kubernetes.io/projected/49081396-f915-404a-a95a-1625bba5456b-kube-api-access-mtqgr\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q6hgs\" (UID: \"49081396-f915-404a-a95a-1625bba5456b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q6hgs" Jul 22 03:14:50.315586 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:50.315565 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/49081396-f915-404a-a95a-1625bba5456b-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q6hgs\" (UID: \"49081396-f915-404a-a95a-1625bba5456b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q6hgs" Jul 22 03:14:50.416654 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:50.416580 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/49081396-f915-404a-a95a-1625bba5456b-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q6hgs\" (UID: \"49081396-f915-404a-a95a-1625bba5456b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q6hgs" Jul 22 03:14:50.416654 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:50.416637 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/49081396-f915-404a-a95a-1625bba5456b-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q6hgs\" (UID: \"49081396-f915-404a-a95a-1625bba5456b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q6hgs" Jul 22 03:14:50.416845 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:50.416746 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtqgr\" (UniqueName: \"kubernetes.io/projected/49081396-f915-404a-a95a-1625bba5456b-kube-api-access-mtqgr\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q6hgs\" (UID: \"49081396-f915-404a-a95a-1625bba5456b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q6hgs" Jul 22 03:14:50.416941 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:50.416925 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/49081396-f915-404a-a95a-1625bba5456b-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q6hgs\" (UID: \"49081396-f915-404a-a95a-1625bba5456b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q6hgs" Jul 22 03:14:50.417038 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:50.416943 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/49081396-f915-404a-a95a-1625bba5456b-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q6hgs\" (UID: \"49081396-f915-404a-a95a-1625bba5456b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q6hgs" Jul 22 03:14:50.425943 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:50.425918 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtqgr\" (UniqueName: \"kubernetes.io/projected/49081396-f915-404a-a95a-1625bba5456b-kube-api-access-mtqgr\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q6hgs\" (UID: \"49081396-f915-404a-a95a-1625bba5456b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q6hgs" Jul 22 03:14:50.568319 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:50.568290 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q6hgs" Jul 22 03:14:50.851843 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:50.851817 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q6hgs"] Jul 22 03:14:50.887163 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:50.887126 2556 generic.go:356] "Generic (PLEG): container finished" podID="eee6f75e-6170-48c2-ac4f-8e340679dde2" containerID="eb9014277491b68f11bcbaabbc105d5f58f6c589fef436b4470a463f39dd912b" exitCode=0 Jul 22 03:14:50.887283 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:50.887213 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lgx6j" event={"ID":"eee6f75e-6170-48c2-ac4f-8e340679dde2","Type":"ContainerDied","Data":"eb9014277491b68f11bcbaabbc105d5f58f6c589fef436b4470a463f39dd912b"} Jul 22 03:14:50.887283 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:50.887267 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lgx6j" event={"ID":"eee6f75e-6170-48c2-ac4f-8e340679dde2","Type":"ContainerStarted","Data":"b4d2c4d098e9b6be1e7a27ca5a2f1741a896af90d9d639b24997766ad66c40d4"} Jul 22 03:14:50.889147 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:50.889125 2556 generic.go:356] "Generic (PLEG): container finished" podID="80d312a3-2e8b-47ee-9a27-bab66ba1a670" containerID="9cb2f30c15d654e1e6c2517c38d360542f18d459eaceaaf97f2e5d564a895de3" exitCode=0 Jul 22 03:14:50.889268 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:50.889203 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fhnsh" event={"ID":"80d312a3-2e8b-47ee-9a27-bab66ba1a670","Type":"ContainerDied","Data":"9cb2f30c15d654e1e6c2517c38d360542f18d459eaceaaf97f2e5d564a895de3"} Jul 22 03:14:50.891195 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:50.891173 2556 generic.go:356] "Generic (PLEG): container finished" podID="4d7095a2-1171-4232-8838-b98a0848a484" containerID="656ebd0e278bdbee2c8195abf0cd525adc478a3374872559668d00199ac80803" exitCode=0 Jul 22 03:14:50.891310 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:50.891205 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rwqdw" event={"ID":"4d7095a2-1171-4232-8838-b98a0848a484","Type":"ContainerDied","Data":"656ebd0e278bdbee2c8195abf0cd525adc478a3374872559668d00199ac80803"} Jul 22 03:14:50.914856 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:14:50.914831 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49081396_f915_404a_a95a_1625bba5456b.slice/crio-9110ad47e9e57341fdf11f23a17859f691de3c3ca268bf78d60a0db37e2c6231 WatchSource:0}: Error finding container 9110ad47e9e57341fdf11f23a17859f691de3c3ca268bf78d60a0db37e2c6231: Status 404 returned error can't find the container with id 9110ad47e9e57341fdf11f23a17859f691de3c3ca268bf78d60a0db37e2c6231 Jul 22 03:14:51.898123 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:51.898096 2556 generic.go:356] "Generic (PLEG): container finished" podID="80d312a3-2e8b-47ee-9a27-bab66ba1a670" containerID="21bbe09f72bf9a95bcb53c88cb3160d001d1e758da247ca30a2b506e25904488" exitCode=0 Jul 22 03:14:51.898529 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:51.898177 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fhnsh" event={"ID":"80d312a3-2e8b-47ee-9a27-bab66ba1a670","Type":"ContainerDied","Data":"21bbe09f72bf9a95bcb53c88cb3160d001d1e758da247ca30a2b506e25904488"} Jul 22 03:14:51.899572 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:51.899547 2556 generic.go:356] "Generic (PLEG): container finished" podID="49081396-f915-404a-a95a-1625bba5456b" containerID="5618b4a87a57f5f266d7ab992f5717b85c57c9eee52a3388f17c32650f01c493" exitCode=0 Jul 22 03:14:51.899701 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:51.899635 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q6hgs" event={"ID":"49081396-f915-404a-a95a-1625bba5456b","Type":"ContainerDied","Data":"5618b4a87a57f5f266d7ab992f5717b85c57c9eee52a3388f17c32650f01c493"} Jul 22 03:14:51.899701 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:51.899678 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q6hgs" event={"ID":"49081396-f915-404a-a95a-1625bba5456b","Type":"ContainerStarted","Data":"9110ad47e9e57341fdf11f23a17859f691de3c3ca268bf78d60a0db37e2c6231"} Jul 22 03:14:51.901750 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:51.901728 2556 generic.go:356] "Generic (PLEG): container finished" podID="4d7095a2-1171-4232-8838-b98a0848a484" containerID="b49d34add9f07bbd62f63dc9fb68264575a9a45cef1a06f943ebed923492b9e1" exitCode=0 Jul 22 03:14:51.901861 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:51.901787 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rwqdw" event={"ID":"4d7095a2-1171-4232-8838-b98a0848a484","Type":"ContainerDied","Data":"b49d34add9f07bbd62f63dc9fb68264575a9a45cef1a06f943ebed923492b9e1"} Jul 22 03:14:51.909171 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:51.909147 2556 generic.go:356] "Generic (PLEG): container finished" podID="eee6f75e-6170-48c2-ac4f-8e340679dde2" containerID="c50ae0d1018014fade33327ac480d12829a353573b095a0e7666b5a849be2136" exitCode=0 Jul 22 03:14:51.909274 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:51.909199 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lgx6j" event={"ID":"eee6f75e-6170-48c2-ac4f-8e340679dde2","Type":"ContainerDied","Data":"c50ae0d1018014fade33327ac480d12829a353573b095a0e7666b5a849be2136"} Jul 22 03:14:52.915605 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:52.915573 2556 generic.go:356] "Generic (PLEG): container finished" podID="49081396-f915-404a-a95a-1625bba5456b" containerID="6c0a74e4403fd22b2bebd49cedaa780b03cfd250852e22adb174177a2d8c093a" exitCode=0 Jul 22 03:14:52.916037 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:52.915657 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q6hgs" event={"ID":"49081396-f915-404a-a95a-1625bba5456b","Type":"ContainerDied","Data":"6c0a74e4403fd22b2bebd49cedaa780b03cfd250852e22adb174177a2d8c093a"} Jul 22 03:14:52.917729 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:52.917706 2556 generic.go:356] "Generic (PLEG): container finished" podID="eee6f75e-6170-48c2-ac4f-8e340679dde2" containerID="d2475799f70853b44dfe0093ab0794846e6dc592435eec398cb9b9e6087438ab" exitCode=0 Jul 22 03:14:52.917801 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:52.917778 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lgx6j" event={"ID":"eee6f75e-6170-48c2-ac4f-8e340679dde2","Type":"ContainerDied","Data":"d2475799f70853b44dfe0093ab0794846e6dc592435eec398cb9b9e6087438ab"} Jul 22 03:14:53.081407 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:53.081381 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fhnsh" Jul 22 03:14:53.095599 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:53.095577 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rwqdw" Jul 22 03:14:53.150913 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:53.150891 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d7095a2-1171-4232-8838-b98a0848a484-bundle\") pod \"4d7095a2-1171-4232-8838-b98a0848a484\" (UID: \"4d7095a2-1171-4232-8838-b98a0848a484\") " Jul 22 03:14:53.151030 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:53.150919 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qcdr7\" (UniqueName: \"kubernetes.io/projected/4d7095a2-1171-4232-8838-b98a0848a484-kube-api-access-qcdr7\") pod \"4d7095a2-1171-4232-8838-b98a0848a484\" (UID: \"4d7095a2-1171-4232-8838-b98a0848a484\") " Jul 22 03:14:53.151030 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:53.150947 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/80d312a3-2e8b-47ee-9a27-bab66ba1a670-bundle\") pod \"80d312a3-2e8b-47ee-9a27-bab66ba1a670\" (UID: \"80d312a3-2e8b-47ee-9a27-bab66ba1a670\") " Jul 22 03:14:53.151030 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:53.151019 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6q4pw\" (UniqueName: \"kubernetes.io/projected/80d312a3-2e8b-47ee-9a27-bab66ba1a670-kube-api-access-6q4pw\") pod \"80d312a3-2e8b-47ee-9a27-bab66ba1a670\" (UID: \"80d312a3-2e8b-47ee-9a27-bab66ba1a670\") " Jul 22 03:14:53.151175 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:53.151063 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/80d312a3-2e8b-47ee-9a27-bab66ba1a670-util\") pod \"80d312a3-2e8b-47ee-9a27-bab66ba1a670\" (UID: \"80d312a3-2e8b-47ee-9a27-bab66ba1a670\") " Jul 22 03:14:53.151175 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:53.151169 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d7095a2-1171-4232-8838-b98a0848a484-util\") pod \"4d7095a2-1171-4232-8838-b98a0848a484\" (UID: \"4d7095a2-1171-4232-8838-b98a0848a484\") " Jul 22 03:14:53.151517 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:53.151493 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d7095a2-1171-4232-8838-b98a0848a484-bundle" (OuterVolumeSpecName: "bundle") pod "4d7095a2-1171-4232-8838-b98a0848a484" (UID: "4d7095a2-1171-4232-8838-b98a0848a484"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:14:53.151918 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:53.151893 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80d312a3-2e8b-47ee-9a27-bab66ba1a670-bundle" (OuterVolumeSpecName: "bundle") pod "80d312a3-2e8b-47ee-9a27-bab66ba1a670" (UID: "80d312a3-2e8b-47ee-9a27-bab66ba1a670"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:14:53.159853 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:53.159820 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80d312a3-2e8b-47ee-9a27-bab66ba1a670-util" (OuterVolumeSpecName: "util") pod "80d312a3-2e8b-47ee-9a27-bab66ba1a670" (UID: "80d312a3-2e8b-47ee-9a27-bab66ba1a670"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:14:53.159963 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:53.159913 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d7095a2-1171-4232-8838-b98a0848a484-kube-api-access-qcdr7" (OuterVolumeSpecName: "kube-api-access-qcdr7") pod "4d7095a2-1171-4232-8838-b98a0848a484" (UID: "4d7095a2-1171-4232-8838-b98a0848a484"). InnerVolumeSpecName "kube-api-access-qcdr7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:14:53.160831 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:53.160807 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d7095a2-1171-4232-8838-b98a0848a484-util" (OuterVolumeSpecName: "util") pod "4d7095a2-1171-4232-8838-b98a0848a484" (UID: "4d7095a2-1171-4232-8838-b98a0848a484"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:14:53.165254 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:53.165233 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80d312a3-2e8b-47ee-9a27-bab66ba1a670-kube-api-access-6q4pw" (OuterVolumeSpecName: "kube-api-access-6q4pw") pod "80d312a3-2e8b-47ee-9a27-bab66ba1a670" (UID: "80d312a3-2e8b-47ee-9a27-bab66ba1a670"). InnerVolumeSpecName "kube-api-access-6q4pw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:14:53.252446 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:53.252367 2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d7095a2-1171-4232-8838-b98a0848a484-util\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:14:53.252446 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:53.252395 2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d7095a2-1171-4232-8838-b98a0848a484-bundle\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:14:53.252446 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:53.252404 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qcdr7\" (UniqueName: \"kubernetes.io/projected/4d7095a2-1171-4232-8838-b98a0848a484-kube-api-access-qcdr7\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:14:53.252446 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:53.252414 2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/80d312a3-2e8b-47ee-9a27-bab66ba1a670-bundle\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:14:53.252446 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:53.252423 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6q4pw\" (UniqueName: \"kubernetes.io/projected/80d312a3-2e8b-47ee-9a27-bab66ba1a670-kube-api-access-6q4pw\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:14:53.252446 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:53.252431 2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/80d312a3-2e8b-47ee-9a27-bab66ba1a670-util\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:14:53.457523 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:53.457485 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:14:53.924470 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:53.924394 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fhnsh" Jul 22 03:14:53.924470 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:53.924415 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fhnsh" event={"ID":"80d312a3-2e8b-47ee-9a27-bab66ba1a670","Type":"ContainerDied","Data":"049e4ba712d65a00f5c2d86cde6d8f9beaa87d72b20cfe6616a9b2aab3cb170a"} Jul 22 03:14:53.924470 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:53.924454 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="049e4ba712d65a00f5c2d86cde6d8f9beaa87d72b20cfe6616a9b2aab3cb170a" Jul 22 03:14:53.926408 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:53.926380 2556 generic.go:356] "Generic (PLEG): container finished" podID="49081396-f915-404a-a95a-1625bba5456b" containerID="a9ed5915345ac62693bd4e169e9742ea252af6d240839902db5749673afea5f5" exitCode=0 Jul 22 03:14:53.926528 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:53.926481 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q6hgs" event={"ID":"49081396-f915-404a-a95a-1625bba5456b","Type":"ContainerDied","Data":"a9ed5915345ac62693bd4e169e9742ea252af6d240839902db5749673afea5f5"} Jul 22 03:14:53.928379 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:53.928364 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rwqdw" Jul 22 03:14:53.928454 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:53.928400 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rwqdw" event={"ID":"4d7095a2-1171-4232-8838-b98a0848a484","Type":"ContainerDied","Data":"864f2a9ba3bd2702db022818497a26e1dadf3e869ce8b3765b0c062d38c5541c"} Jul 22 03:14:53.928454 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:53.928437 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="864f2a9ba3bd2702db022818497a26e1dadf3e869ce8b3765b0c062d38c5541c" Jul 22 03:14:54.060830 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:54.060808 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lgx6j" Jul 22 03:14:54.159571 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:54.159540 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eee6f75e-6170-48c2-ac4f-8e340679dde2-bundle\") pod \"eee6f75e-6170-48c2-ac4f-8e340679dde2\" (UID: \"eee6f75e-6170-48c2-ac4f-8e340679dde2\") " Jul 22 03:14:54.159710 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:54.159583 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eee6f75e-6170-48c2-ac4f-8e340679dde2-util\") pod \"eee6f75e-6170-48c2-ac4f-8e340679dde2\" (UID: \"eee6f75e-6170-48c2-ac4f-8e340679dde2\") " Jul 22 03:14:54.159710 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:54.159657 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-52pt5\" (UniqueName: \"kubernetes.io/projected/eee6f75e-6170-48c2-ac4f-8e340679dde2-kube-api-access-52pt5\") pod \"eee6f75e-6170-48c2-ac4f-8e340679dde2\" (UID: \"eee6f75e-6170-48c2-ac4f-8e340679dde2\") " Jul 22 03:14:54.160180 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:54.160150 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eee6f75e-6170-48c2-ac4f-8e340679dde2-bundle" (OuterVolumeSpecName: "bundle") pod "eee6f75e-6170-48c2-ac4f-8e340679dde2" (UID: "eee6f75e-6170-48c2-ac4f-8e340679dde2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:14:54.161809 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:54.161786 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eee6f75e-6170-48c2-ac4f-8e340679dde2-kube-api-access-52pt5" (OuterVolumeSpecName: "kube-api-access-52pt5") pod "eee6f75e-6170-48c2-ac4f-8e340679dde2" (UID: "eee6f75e-6170-48c2-ac4f-8e340679dde2"). InnerVolumeSpecName "kube-api-access-52pt5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:14:54.165327 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:54.165303 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eee6f75e-6170-48c2-ac4f-8e340679dde2-util" (OuterVolumeSpecName: "util") pod "eee6f75e-6170-48c2-ac4f-8e340679dde2" (UID: "eee6f75e-6170-48c2-ac4f-8e340679dde2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:14:54.261248 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:54.261173 2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eee6f75e-6170-48c2-ac4f-8e340679dde2-util\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:14:54.261248 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:54.261199 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52pt5\" (UniqueName: \"kubernetes.io/projected/eee6f75e-6170-48c2-ac4f-8e340679dde2-kube-api-access-52pt5\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:14:54.261248 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:54.261210 2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eee6f75e-6170-48c2-ac4f-8e340679dde2-bundle\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:14:54.935608 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:54.935569 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lgx6j" event={"ID":"eee6f75e-6170-48c2-ac4f-8e340679dde2","Type":"ContainerDied","Data":"b4d2c4d098e9b6be1e7a27ca5a2f1741a896af90d9d639b24997766ad66c40d4"} Jul 22 03:14:54.935608 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:54.935595 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lgx6j" Jul 22 03:14:54.936049 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:54.935625 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b4d2c4d098e9b6be1e7a27ca5a2f1741a896af90d9d639b24997766ad66c40d4" Jul 22 03:14:55.058761 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:55.058736 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q6hgs" Jul 22 03:14:55.168172 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:55.168139 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/49081396-f915-404a-a95a-1625bba5456b-util\") pod \"49081396-f915-404a-a95a-1625bba5456b\" (UID: \"49081396-f915-404a-a95a-1625bba5456b\") " Jul 22 03:14:55.168172 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:55.168177 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/49081396-f915-404a-a95a-1625bba5456b-bundle\") pod \"49081396-f915-404a-a95a-1625bba5456b\" (UID: \"49081396-f915-404a-a95a-1625bba5456b\") " Jul 22 03:14:55.168367 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:55.168243 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mtqgr\" (UniqueName: \"kubernetes.io/projected/49081396-f915-404a-a95a-1625bba5456b-kube-api-access-mtqgr\") pod \"49081396-f915-404a-a95a-1625bba5456b\" (UID: \"49081396-f915-404a-a95a-1625bba5456b\") " Jul 22 03:14:55.168934 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:55.168898 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49081396-f915-404a-a95a-1625bba5456b-bundle" (OuterVolumeSpecName: "bundle") pod "49081396-f915-404a-a95a-1625bba5456b" (UID: "49081396-f915-404a-a95a-1625bba5456b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:14:55.170309 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:55.170279 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49081396-f915-404a-a95a-1625bba5456b-kube-api-access-mtqgr" (OuterVolumeSpecName: "kube-api-access-mtqgr") pod "49081396-f915-404a-a95a-1625bba5456b" (UID: "49081396-f915-404a-a95a-1625bba5456b"). InnerVolumeSpecName "kube-api-access-mtqgr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:14:55.173717 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:55.173683 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49081396-f915-404a-a95a-1625bba5456b-util" (OuterVolumeSpecName: "util") pod "49081396-f915-404a-a95a-1625bba5456b" (UID: "49081396-f915-404a-a95a-1625bba5456b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:14:55.269867 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:55.269792 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mtqgr\" (UniqueName: \"kubernetes.io/projected/49081396-f915-404a-a95a-1625bba5456b-kube-api-access-mtqgr\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:14:55.269867 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:55.269823 2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/49081396-f915-404a-a95a-1625bba5456b-util\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:14:55.269867 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:55.269835 2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/49081396-f915-404a-a95a-1625bba5456b-bundle\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:14:55.942682 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:55.942649 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q6hgs" Jul 22 03:14:55.942682 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:55.942658 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q6hgs" event={"ID":"49081396-f915-404a-a95a-1625bba5456b","Type":"ContainerDied","Data":"9110ad47e9e57341fdf11f23a17859f691de3c3ca268bf78d60a0db37e2c6231"} Jul 22 03:14:55.943240 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:14:55.942724 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9110ad47e9e57341fdf11f23a17859f691de3c3ca268bf78d60a0db37e2c6231" Jul 22 03:15:02.657463 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:02.657368 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:05.090230 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.090197 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pw672"] Jul 22 03:15:05.090599 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.090506 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80d312a3-2e8b-47ee-9a27-bab66ba1a670" containerName="util" Jul 22 03:15:05.090599 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.090517 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="80d312a3-2e8b-47ee-9a27-bab66ba1a670" containerName="util" Jul 22 03:15:05.090599 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.090532 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eee6f75e-6170-48c2-ac4f-8e340679dde2" containerName="extract" Jul 22 03:15:05.090599 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.090537 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="eee6f75e-6170-48c2-ac4f-8e340679dde2" containerName="extract" Jul 22 03:15:05.090599 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.090545 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d7095a2-1171-4232-8838-b98a0848a484" containerName="extract" Jul 22 03:15:05.090599 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.090550 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d7095a2-1171-4232-8838-b98a0848a484" containerName="extract" Jul 22 03:15:05.090599 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.090556 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49081396-f915-404a-a95a-1625bba5456b" containerName="extract" Jul 22 03:15:05.090599 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.090562 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="49081396-f915-404a-a95a-1625bba5456b" containerName="extract" Jul 22 03:15:05.090599 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.090574 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d7095a2-1171-4232-8838-b98a0848a484" containerName="util" Jul 22 03:15:05.090599 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.090579 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d7095a2-1171-4232-8838-b98a0848a484" containerName="util" Jul 22 03:15:05.090599 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.090585 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80d312a3-2e8b-47ee-9a27-bab66ba1a670" containerName="extract" Jul 22 03:15:05.090599 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.090589 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="80d312a3-2e8b-47ee-9a27-bab66ba1a670" containerName="extract" Jul 22 03:15:05.090599 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.090595 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49081396-f915-404a-a95a-1625bba5456b" containerName="pull" Jul 22 03:15:05.090599 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.090599 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="49081396-f915-404a-a95a-1625bba5456b" containerName="pull" Jul 22 03:15:05.090599 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.090607 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49081396-f915-404a-a95a-1625bba5456b" containerName="util" Jul 22 03:15:05.091314 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.090612 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="49081396-f915-404a-a95a-1625bba5456b" containerName="util" Jul 22 03:15:05.091314 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.090618 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eee6f75e-6170-48c2-ac4f-8e340679dde2" containerName="pull" Jul 22 03:15:05.091314 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.090622 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="eee6f75e-6170-48c2-ac4f-8e340679dde2" containerName="pull" Jul 22 03:15:05.091314 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.090629 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80d312a3-2e8b-47ee-9a27-bab66ba1a670" containerName="pull" Jul 22 03:15:05.091314 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.090634 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="80d312a3-2e8b-47ee-9a27-bab66ba1a670" containerName="pull" Jul 22 03:15:05.091314 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.090640 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d7095a2-1171-4232-8838-b98a0848a484" containerName="pull" Jul 22 03:15:05.091314 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.090646 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d7095a2-1171-4232-8838-b98a0848a484" containerName="pull" Jul 22 03:15:05.091314 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.090652 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eee6f75e-6170-48c2-ac4f-8e340679dde2" containerName="util" Jul 22 03:15:05.091314 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.090657 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="eee6f75e-6170-48c2-ac4f-8e340679dde2" containerName="util" Jul 22 03:15:05.091314 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.090705 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="49081396-f915-404a-a95a-1625bba5456b" containerName="extract" Jul 22 03:15:05.091314 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.090715 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d7095a2-1171-4232-8838-b98a0848a484" containerName="extract" Jul 22 03:15:05.091314 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.090720 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="80d312a3-2e8b-47ee-9a27-bab66ba1a670" containerName="extract" Jul 22 03:15:05.091314 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.090726 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="eee6f75e-6170-48c2-ac4f-8e340679dde2" containerName="extract" Jul 22 03:15:05.093713 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.093695 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pw672" Jul 22 03:15:05.096989 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.096968 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-sn7vz\"" Jul 22 03:15:05.115222 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.115194 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pw672"] Jul 22 03:15:05.255515 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.255474 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncj4v\" (UniqueName: \"kubernetes.io/projected/d4d065b2-4196-49c2-aa57-04630336da07-kube-api-access-ncj4v\") pod \"limitador-operator-controller-manager-859d7b78bd-pw672\" (UID: \"d4d065b2-4196-49c2-aa57-04630336da07\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pw672" Jul 22 03:15:05.357069 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.356972 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncj4v\" (UniqueName: \"kubernetes.io/projected/d4d065b2-4196-49c2-aa57-04630336da07-kube-api-access-ncj4v\") pod \"limitador-operator-controller-manager-859d7b78bd-pw672\" (UID: \"d4d065b2-4196-49c2-aa57-04630336da07\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pw672" Jul 22 03:15:05.372223 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.372194 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncj4v\" (UniqueName: \"kubernetes.io/projected/d4d065b2-4196-49c2-aa57-04630336da07-kube-api-access-ncj4v\") pod \"limitador-operator-controller-manager-859d7b78bd-pw672\" (UID: \"d4d065b2-4196-49c2-aa57-04630336da07\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pw672" Jul 22 03:15:05.403097 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.403055 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pw672" Jul 22 03:15:05.579015 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.578991 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pw672"] Jul 22 03:15:05.581860 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:15:05.581835 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4d065b2_4196_49c2_aa57_04630336da07.slice/crio-6a9ffc3e2afb789edff3a82c1b5cb92befee5b352d6f1797c761684bb134814e WatchSource:0}: Error finding container 6a9ffc3e2afb789edff3a82c1b5cb92befee5b352d6f1797c761684bb134814e: Status 404 returned error can't find the container with id 6a9ffc3e2afb789edff3a82c1b5cb92befee5b352d6f1797c761684bb134814e Jul 22 03:15:05.795610 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.795518 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:05.992234 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:05.992201 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pw672" event={"ID":"d4d065b2-4196-49c2-aa57-04630336da07","Type":"ContainerStarted","Data":"6a9ffc3e2afb789edff3a82c1b5cb92befee5b352d6f1797c761684bb134814e"} Jul 22 03:15:06.676449 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:06.676406 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:08.005357 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:08.005314 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pw672" event={"ID":"d4d065b2-4196-49c2-aa57-04630336da07","Type":"ContainerStarted","Data":"525877dfb28bcb4ad68ff8efc0fd58b08ac8beb23b4ef897467d46c51dee1d2d"} Jul 22 03:15:08.005803 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:08.005520 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pw672" Jul 22 03:15:08.067056 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:08.066997 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pw672" podStartSLOduration=1.553163451 podStartE2EDuration="3.06698332s" totalImagesPullingTime="1.513819869s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:15:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:15:08.065100329 +0000 UTC m=+448.514083578" watchObservedRunningTime="2026-07-22 03:15:08.06698332 +0000 UTC m=+448.515966566" Jul 22 03:15:09.353348 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:09.353316 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-g58pn"] Jul 22 03:15:09.359560 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:09.359540 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-g58pn" Jul 22 03:15:09.362236 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:09.362215 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-p6hkr\"" Jul 22 03:15:09.367407 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:09.367374 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-g58pn"] Jul 22 03:15:09.497711 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:09.497670 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmgcn\" (UniqueName: \"kubernetes.io/projected/c7d619e0-6915-49f1-b700-515c176c680a-kube-api-access-pmgcn\") pod \"authorino-operator-75cc8d5cb7-g58pn\" (UID: \"c7d619e0-6915-49f1-b700-515c176c680a\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-g58pn" Jul 22 03:15:09.598854 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:09.598818 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmgcn\" (UniqueName: \"kubernetes.io/projected/c7d619e0-6915-49f1-b700-515c176c680a-kube-api-access-pmgcn\") pod \"authorino-operator-75cc8d5cb7-g58pn\" (UID: \"c7d619e0-6915-49f1-b700-515c176c680a\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-g58pn" Jul 22 03:15:09.614342 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:09.614275 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmgcn\" (UniqueName: \"kubernetes.io/projected/c7d619e0-6915-49f1-b700-515c176c680a-kube-api-access-pmgcn\") pod \"authorino-operator-75cc8d5cb7-g58pn\" (UID: \"c7d619e0-6915-49f1-b700-515c176c680a\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-g58pn" Jul 22 03:15:09.670055 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:09.670018 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-g58pn" Jul 22 03:15:09.801172 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:09.801148 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-g58pn"] Jul 22 03:15:09.803129 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:15:09.803102 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7d619e0_6915_49f1_b700_515c176c680a.slice/crio-3557a3f85c1b7ee4e30816df45d833aa350b5f8a4590584c0e1dd6846b7e40fe WatchSource:0}: Error finding container 3557a3f85c1b7ee4e30816df45d833aa350b5f8a4590584c0e1dd6846b7e40fe: Status 404 returned error can't find the container with id 3557a3f85c1b7ee4e30816df45d833aa350b5f8a4590584c0e1dd6846b7e40fe Jul 22 03:15:09.988605 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:09.988520 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:10.016223 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:10.016181 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-g58pn" event={"ID":"c7d619e0-6915-49f1-b700-515c176c680a","Type":"ContainerStarted","Data":"3557a3f85c1b7ee4e30816df45d833aa350b5f8a4590584c0e1dd6846b7e40fe"} Jul 22 03:15:10.099237 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:10.099206 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:10.369799 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:10.369770 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:11.989161 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:11.989126 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:12.574748 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:12.574715 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-jkmck"] Jul 22 03:15:12.576856 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:12.576839 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jkmck" Jul 22 03:15:12.580534 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:12.580517 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-2vz7h\"" Jul 22 03:15:12.580755 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:12.580741 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 22 03:15:12.596492 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:12.596465 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-jkmck"] Jul 22 03:15:12.629681 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:12.629644 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtn7x\" (UniqueName: \"kubernetes.io/projected/df73b24e-a59c-4074-a0ff-afe832f72d32-kube-api-access-xtn7x\") pod \"dns-operator-controller-manager-864dd78c4c-jkmck\" (UID: \"df73b24e-a59c-4074-a0ff-afe832f72d32\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jkmck" Jul 22 03:15:12.734589 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:12.734541 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtn7x\" (UniqueName: \"kubernetes.io/projected/df73b24e-a59c-4074-a0ff-afe832f72d32-kube-api-access-xtn7x\") pod \"dns-operator-controller-manager-864dd78c4c-jkmck\" (UID: \"df73b24e-a59c-4074-a0ff-afe832f72d32\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jkmck" Jul 22 03:15:12.754560 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:12.754524 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtn7x\" (UniqueName: \"kubernetes.io/projected/df73b24e-a59c-4074-a0ff-afe832f72d32-kube-api-access-xtn7x\") pod \"dns-operator-controller-manager-864dd78c4c-jkmck\" (UID: \"df73b24e-a59c-4074-a0ff-afe832f72d32\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jkmck" Jul 22 03:15:12.887799 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:12.887706 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jkmck" Jul 22 03:15:13.034245 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:13.034210 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-g58pn" event={"ID":"c7d619e0-6915-49f1-b700-515c176c680a","Type":"ContainerStarted","Data":"fc49ddbb829cb7dc6beb2c01b8e22d148fc57c09db27b83bb97f698e52910efe"} Jul 22 03:15:13.034607 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:13.034328 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-g58pn" Jul 22 03:15:13.052111 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:13.052090 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-jkmck"] Jul 22 03:15:13.053798 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:15:13.053775 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf73b24e_a59c_4074_a0ff_afe832f72d32.slice/crio-f0622b136ed3de9804253bf4b430a974721a4daee59f348f09839dd905b02442 WatchSource:0}: Error finding container f0622b136ed3de9804253bf4b430a974721a4daee59f348f09839dd905b02442: Status 404 returned error can't find the container with id f0622b136ed3de9804253bf4b430a974721a4daee59f348f09839dd905b02442 Jul 22 03:15:13.059293 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:13.059212 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-g58pn" podStartSLOduration=0.924014621 podStartE2EDuration="4.05919382s" totalImagesPullingTime="3.135179199s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:15:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:15:13.056793507 +0000 UTC m=+453.505776756" watchObservedRunningTime="2026-07-22 03:15:13.05919382 +0000 UTC m=+453.508177071" Jul 22 03:15:13.148138 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:13.148049 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:13.248029 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:13.247983 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:14.042107 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:14.042070 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jkmck" event={"ID":"df73b24e-a59c-4074-a0ff-afe832f72d32","Type":"ContainerStarted","Data":"f0622b136ed3de9804253bf4b430a974721a4daee59f348f09839dd905b02442"} Jul 22 03:15:15.002786 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:15.002722 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:16.055379 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:16.055345 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jkmck" event={"ID":"df73b24e-a59c-4074-a0ff-afe832f72d32","Type":"ContainerStarted","Data":"b47b30e8cbe0ec23b2aab069624918de2caecf55b66e2894f614c63e49d515d5"} Jul 22 03:15:16.055770 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:16.055581 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jkmck" Jul 22 03:15:16.075969 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:16.075903 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jkmck" podStartSLOduration=2.017188835 podStartE2EDuration="4.075888632s" totalImagesPullingTime="2.058699797s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:15:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:15:16.074339554 +0000 UTC m=+456.523322794" watchObservedRunningTime="2026-07-22 03:15:16.075888632 +0000 UTC m=+456.524871883" Jul 22 03:15:17.185338 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:17.185289 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxc8b"] Jul 22 03:15:17.187373 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:17.187356 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxc8b" Jul 22 03:15:17.190305 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:17.190285 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-zr6ln\"" Jul 22 03:15:17.203257 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:17.203233 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxc8b"] Jul 22 03:15:17.289068 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:17.289035 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/61813f2a-1f53-42e5-82a6-0ccdfffe0335-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vxc8b\" (UID: \"61813f2a-1f53-42e5-82a6-0ccdfffe0335\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxc8b" Jul 22 03:15:17.289247 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:17.289085 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9w67k\" (UniqueName: \"kubernetes.io/projected/61813f2a-1f53-42e5-82a6-0ccdfffe0335-kube-api-access-9w67k\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vxc8b\" (UID: \"61813f2a-1f53-42e5-82a6-0ccdfffe0335\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxc8b" Jul 22 03:15:17.390354 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:17.390315 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/61813f2a-1f53-42e5-82a6-0ccdfffe0335-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vxc8b\" (UID: \"61813f2a-1f53-42e5-82a6-0ccdfffe0335\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxc8b" Jul 22 03:15:17.390518 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:17.390379 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w67k\" (UniqueName: \"kubernetes.io/projected/61813f2a-1f53-42e5-82a6-0ccdfffe0335-kube-api-access-9w67k\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vxc8b\" (UID: \"61813f2a-1f53-42e5-82a6-0ccdfffe0335\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxc8b" Jul 22 03:15:17.390778 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:17.390760 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/61813f2a-1f53-42e5-82a6-0ccdfffe0335-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vxc8b\" (UID: \"61813f2a-1f53-42e5-82a6-0ccdfffe0335\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxc8b" Jul 22 03:15:17.403927 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:17.403898 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9w67k\" (UniqueName: \"kubernetes.io/projected/61813f2a-1f53-42e5-82a6-0ccdfffe0335-kube-api-access-9w67k\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vxc8b\" (UID: \"61813f2a-1f53-42e5-82a6-0ccdfffe0335\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxc8b" Jul 22 03:15:17.497672 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:17.497571 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxc8b" Jul 22 03:15:17.630622 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:17.630595 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxc8b"] Jul 22 03:15:17.632678 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:15:17.632649 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61813f2a_1f53_42e5_82a6_0ccdfffe0335.slice/crio-1abc5b1b7e8e8fc4cc7b95a86d49ffb16c3dbdaaa6e82cdd87a0bdccc758e0dd WatchSource:0}: Error finding container 1abc5b1b7e8e8fc4cc7b95a86d49ffb16c3dbdaaa6e82cdd87a0bdccc758e0dd: Status 404 returned error can't find the container with id 1abc5b1b7e8e8fc4cc7b95a86d49ffb16c3dbdaaa6e82cdd87a0bdccc758e0dd Jul 22 03:15:18.067107 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:18.067064 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxc8b" event={"ID":"61813f2a-1f53-42e5-82a6-0ccdfffe0335","Type":"ContainerStarted","Data":"1abc5b1b7e8e8fc4cc7b95a86d49ffb16c3dbdaaa6e82cdd87a0bdccc758e0dd"} Jul 22 03:15:18.259508 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:18.259465 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:18.374017 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:18.373981 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:19.013003 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:19.012943 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pw672" Jul 22 03:15:23.536731 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:23.536699 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:23.641315 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:23.641278 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:23.940151 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:23.940110 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:24.042161 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:24.042128 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:24.044182 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:24.044159 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-g58pn" Jul 22 03:15:24.103679 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:24.103634 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxc8b" event={"ID":"61813f2a-1f53-42e5-82a6-0ccdfffe0335","Type":"ContainerStarted","Data":"52f670a4448cb26f05c21c2b249965ddbf63d0e363da54baf5ecf5b23495493b"} Jul 22 03:15:24.103679 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:24.103710 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxc8b" Jul 22 03:15:24.131664 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:24.131615 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxc8b" podStartSLOduration=1.291037 podStartE2EDuration="7.131598802s" totalImagesPullingTime="5.840561802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:15:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:15:24.130202325 +0000 UTC m=+464.579185574" watchObservedRunningTime="2026-07-22 03:15:24.131598802 +0000 UTC m=+464.580582050" Jul 22 03:15:25.676007 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:25.675973 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-767c8f5bf5-w88vb"] Jul 22 03:15:25.679379 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:25.679355 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-767c8f5bf5-w88vb" Jul 22 03:15:25.698226 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:25.698195 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-767c8f5bf5-w88vb"] Jul 22 03:15:25.770408 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:25.770374 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519-service-ca\") pod \"console-767c8f5bf5-w88vb\" (UID: \"85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519\") " pod="openshift-console/console-767c8f5bf5-w88vb" Jul 22 03:15:25.770572 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:25.770449 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519-console-oauth-config\") pod \"console-767c8f5bf5-w88vb\" (UID: \"85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519\") " pod="openshift-console/console-767c8f5bf5-w88vb" Jul 22 03:15:25.770572 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:25.770483 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519-console-serving-cert\") pod \"console-767c8f5bf5-w88vb\" (UID: \"85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519\") " pod="openshift-console/console-767c8f5bf5-w88vb" Jul 22 03:15:25.770572 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:25.770499 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519-trusted-ca-bundle\") pod \"console-767c8f5bf5-w88vb\" (UID: \"85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519\") " pod="openshift-console/console-767c8f5bf5-w88vb" Jul 22 03:15:25.770572 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:25.770523 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519-oauth-serving-cert\") pod \"console-767c8f5bf5-w88vb\" (UID: \"85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519\") " pod="openshift-console/console-767c8f5bf5-w88vb" Jul 22 03:15:25.770572 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:25.770540 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519-console-config\") pod \"console-767c8f5bf5-w88vb\" (UID: \"85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519\") " pod="openshift-console/console-767c8f5bf5-w88vb" Jul 22 03:15:25.770572 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:25.770565 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wndqz\" (UniqueName: \"kubernetes.io/projected/85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519-kube-api-access-wndqz\") pod \"console-767c8f5bf5-w88vb\" (UID: \"85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519\") " pod="openshift-console/console-767c8f5bf5-w88vb" Jul 22 03:15:25.819055 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:25.818997 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:25.871277 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:25.871246 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wndqz\" (UniqueName: \"kubernetes.io/projected/85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519-kube-api-access-wndqz\") pod \"console-767c8f5bf5-w88vb\" (UID: \"85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519\") " pod="openshift-console/console-767c8f5bf5-w88vb" Jul 22 03:15:25.871440 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:25.871312 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519-service-ca\") pod \"console-767c8f5bf5-w88vb\" (UID: \"85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519\") " pod="openshift-console/console-767c8f5bf5-w88vb" Jul 22 03:15:25.871440 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:25.871369 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519-console-oauth-config\") pod \"console-767c8f5bf5-w88vb\" (UID: \"85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519\") " pod="openshift-console/console-767c8f5bf5-w88vb" Jul 22 03:15:25.871440 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:25.871402 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519-console-serving-cert\") pod \"console-767c8f5bf5-w88vb\" (UID: \"85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519\") " pod="openshift-console/console-767c8f5bf5-w88vb" Jul 22 03:15:25.871440 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:25.871416 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519-trusted-ca-bundle\") pod \"console-767c8f5bf5-w88vb\" (UID: \"85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519\") " pod="openshift-console/console-767c8f5bf5-w88vb" Jul 22 03:15:25.871440 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:25.871432 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519-oauth-serving-cert\") pod \"console-767c8f5bf5-w88vb\" (UID: \"85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519\") " pod="openshift-console/console-767c8f5bf5-w88vb" Jul 22 03:15:25.871682 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:25.871455 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519-console-config\") pod \"console-767c8f5bf5-w88vb\" (UID: \"85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519\") " pod="openshift-console/console-767c8f5bf5-w88vb" Jul 22 03:15:25.872103 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:25.872067 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519-service-ca\") pod \"console-767c8f5bf5-w88vb\" (UID: \"85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519\") " pod="openshift-console/console-767c8f5bf5-w88vb" Jul 22 03:15:25.872243 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:25.872138 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519-console-config\") pod \"console-767c8f5bf5-w88vb\" (UID: \"85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519\") " pod="openshift-console/console-767c8f5bf5-w88vb" Jul 22 03:15:25.872306 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:25.872283 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519-oauth-serving-cert\") pod \"console-767c8f5bf5-w88vb\" (UID: \"85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519\") " pod="openshift-console/console-767c8f5bf5-w88vb" Jul 22 03:15:25.872345 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:25.872318 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519-trusted-ca-bundle\") pod \"console-767c8f5bf5-w88vb\" (UID: \"85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519\") " pod="openshift-console/console-767c8f5bf5-w88vb" Jul 22 03:15:25.873880 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:25.873858 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519-console-oauth-config\") pod \"console-767c8f5bf5-w88vb\" (UID: \"85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519\") " pod="openshift-console/console-767c8f5bf5-w88vb" Jul 22 03:15:25.874001 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:25.873983 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519-console-serving-cert\") pod \"console-767c8f5bf5-w88vb\" (UID: \"85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519\") " pod="openshift-console/console-767c8f5bf5-w88vb" Jul 22 03:15:25.891616 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:25.891588 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wndqz\" (UniqueName: \"kubernetes.io/projected/85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519-kube-api-access-wndqz\") pod \"console-767c8f5bf5-w88vb\" (UID: \"85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519\") " pod="openshift-console/console-767c8f5bf5-w88vb" Jul 22 03:15:25.970102 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:25.970011 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:25.989097 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:25.989075 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-767c8f5bf5-w88vb" Jul 22 03:15:26.066141 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:26.066089 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:26.126266 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:26.126235 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-767c8f5bf5-w88vb"] Jul 22 03:15:26.127617 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:15:26.127592 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85100ccf_bd8f_4ec5_bd5f_ce0cf7dbd519.slice/crio-ac49eb04c6e0c3b852c1b906572b24a8e21c1c73130799348d51355e36f7035c WatchSource:0}: Error finding container ac49eb04c6e0c3b852c1b906572b24a8e21c1c73130799348d51355e36f7035c: Status 404 returned error can't find the container with id ac49eb04c6e0c3b852c1b906572b24a8e21c1c73130799348d51355e36f7035c Jul 22 03:15:26.516179 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:26.516144 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:26.599295 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:26.599253 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:26.706816 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:26.706779 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:27.063499 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:27.063468 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jkmck" Jul 22 03:15:27.121348 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:27.121314 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-767c8f5bf5-w88vb" event={"ID":"85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519","Type":"ContainerStarted","Data":"05bfce87e5ee3ddc83a09117d8bd98355d1fa997ffa5f34275667e91be4fdade"} Jul 22 03:15:27.121540 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:27.121371 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-767c8f5bf5-w88vb" event={"ID":"85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519","Type":"ContainerStarted","Data":"ac49eb04c6e0c3b852c1b906572b24a8e21c1c73130799348d51355e36f7035c"} Jul 22 03:15:27.141601 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:27.141547 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-767c8f5bf5-w88vb" podStartSLOduration=2.141530479 podStartE2EDuration="2.141530479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:15:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:15:27.139391537 +0000 UTC m=+467.588374787" watchObservedRunningTime="2026-07-22 03:15:27.141530479 +0000 UTC m=+467.590513729" Jul 22 03:15:27.808426 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:27.808394 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:27.964807 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:27.964767 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:31.181977 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:31.181920 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:31.287366 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:31.287325 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:35.110567 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:35.110531 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxc8b" Jul 22 03:15:35.989795 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:35.989743 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-767c8f5bf5-w88vb" Jul 22 03:15:35.990017 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:35.989814 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-767c8f5bf5-w88vb" Jul 22 03:15:35.994373 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:35.994349 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-767c8f5bf5-w88vb" Jul 22 03:15:36.173214 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:36.173186 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-767c8f5bf5-w88vb" Jul 22 03:15:36.329732 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:36.329696 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57fd7bf74d-kpksw"] Jul 22 03:15:36.649995 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:36.649891 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:36.959091 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:36.958991 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:37.123077 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.123034 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:37.231607 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.231520 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxc8b"] Jul 22 03:15:37.232078 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.231693 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxc8b" podUID="61813f2a-1f53-42e5-82a6-0ccdfffe0335" containerName="manager" containerID="cri-o://52f670a4448cb26f05c21c2b249965ddbf63d0e363da54baf5ecf5b23495493b" gracePeriod=2 Jul 22 03:15:37.238860 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.238830 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxc8b"] Jul 22 03:15:37.268741 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.268706 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pw672"] Jul 22 03:15:37.269100 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.269035 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pw672" podUID="d4d065b2-4196-49c2-aa57-04630336da07" containerName="manager" containerID="cri-o://525877dfb28bcb4ad68ff8efc0fd58b08ac8beb23b4ef897467d46c51dee1d2d" gracePeriod=2 Jul 22 03:15:37.274471 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.274443 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pw672"] Jul 22 03:15:37.274600 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:15:37.274470 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vxc8b\" is forbidden: User \"system:node:ip-10-0-133-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-13.ec2.internal' and this object" podUID="61813f2a-1f53-42e5-82a6-0ccdfffe0335" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxc8b" Jul 22 03:15:37.288198 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:15:37.288166 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-pw672\" is forbidden: User \"system:node:ip-10-0-133-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-13.ec2.internal' and this object" podUID="d4d065b2-4196-49c2-aa57-04630336da07" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pw672" Jul 22 03:15:37.322177 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.322135 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhljs"] Jul 22 03:15:37.322619 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.322598 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4d065b2-4196-49c2-aa57-04630336da07" containerName="manager" Jul 22 03:15:37.322695 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.322622 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4d065b2-4196-49c2-aa57-04630336da07" containerName="manager" Jul 22 03:15:37.322695 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.322637 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61813f2a-1f53-42e5-82a6-0ccdfffe0335" containerName="manager" Jul 22 03:15:37.322695 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.322642 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="61813f2a-1f53-42e5-82a6-0ccdfffe0335" containerName="manager" Jul 22 03:15:37.322852 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.322722 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4d065b2-4196-49c2-aa57-04630336da07" containerName="manager" Jul 22 03:15:37.322852 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.322734 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="61813f2a-1f53-42e5-82a6-0ccdfffe0335" containerName="manager" Jul 22 03:15:37.324778 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.324758 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-flnml"] Jul 22 03:15:37.324916 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.324888 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhljs" Jul 22 03:15:37.327037 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.327012 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-flnml" Jul 22 03:15:37.328021 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:15:37.327999 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-pw672\" is forbidden: User \"system:node:ip-10-0-133-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-13.ec2.internal' and this object" podUID="d4d065b2-4196-49c2-aa57-04630336da07" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pw672" Jul 22 03:15:37.351900 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.351871 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhljs"] Jul 22 03:15:37.377730 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.377707 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-flnml"] Jul 22 03:15:37.387985 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:15:37.387920 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vxc8b\" is forbidden: User \"system:node:ip-10-0-133-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-13.ec2.internal' and this object" podUID="61813f2a-1f53-42e5-82a6-0ccdfffe0335" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxc8b" Jul 22 03:15:37.439006 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:15:37.438972 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vxc8b\" is forbidden: User \"system:node:ip-10-0-133-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-13.ec2.internal' and this object" podUID="61813f2a-1f53-42e5-82a6-0ccdfffe0335" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxc8b" Jul 22 03:15:37.442648 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:15:37.442593 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-pw672\" is forbidden: User \"system:node:ip-10-0-133-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-13.ec2.internal' and this object" podUID="d4d065b2-4196-49c2-aa57-04630336da07" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pw672" Jul 22 03:15:37.472364 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.472328 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9tgg\" (UniqueName: \"kubernetes.io/projected/abbcc799-0450-425a-be2d-f8f2324e6a23-kube-api-access-b9tgg\") pod \"limitador-operator-controller-manager-859d7b78bd-flnml\" (UID: \"abbcc799-0450-425a-be2d-f8f2324e6a23\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-flnml" Jul 22 03:15:37.472364 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.472367 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33e8789a-aa76-4266-bf71-838add29dadb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xhljs\" (UID: \"33e8789a-aa76-4266-bf71-838add29dadb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhljs" Jul 22 03:15:37.472622 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.472391 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rcdj\" (UniqueName: \"kubernetes.io/projected/33e8789a-aa76-4266-bf71-838add29dadb-kube-api-access-9rcdj\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xhljs\" (UID: \"33e8789a-aa76-4266-bf71-838add29dadb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhljs" Jul 22 03:15:37.513341 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.513316 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxc8b" Jul 22 03:15:37.516516 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.516496 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pw672" Jul 22 03:15:37.518699 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:15:37.518678 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-pw672\" is forbidden: User \"system:node:ip-10-0-133-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-13.ec2.internal' and this object" podUID="d4d065b2-4196-49c2-aa57-04630336da07" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pw672" Jul 22 03:15:37.521512 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:15:37.521490 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vxc8b\" is forbidden: User \"system:node:ip-10-0-133-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-13.ec2.internal' and this object" podUID="61813f2a-1f53-42e5-82a6-0ccdfffe0335" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxc8b" Jul 22 03:15:37.524229 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:15:37.524208 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vxc8b\" is forbidden: User \"system:node:ip-10-0-133-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-13.ec2.internal' and this object" podUID="61813f2a-1f53-42e5-82a6-0ccdfffe0335" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxc8b" Jul 22 03:15:37.528183 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:15:37.528165 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-pw672\" is forbidden: User \"system:node:ip-10-0-133-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-13.ec2.internal' and this object" podUID="d4d065b2-4196-49c2-aa57-04630336da07" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pw672" Jul 22 03:15:37.573645 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.573602 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9tgg\" (UniqueName: \"kubernetes.io/projected/abbcc799-0450-425a-be2d-f8f2324e6a23-kube-api-access-b9tgg\") pod \"limitador-operator-controller-manager-859d7b78bd-flnml\" (UID: \"abbcc799-0450-425a-be2d-f8f2324e6a23\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-flnml" Jul 22 03:15:37.573809 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.573658 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33e8789a-aa76-4266-bf71-838add29dadb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xhljs\" (UID: \"33e8789a-aa76-4266-bf71-838add29dadb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhljs" Jul 22 03:15:37.573809 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.573689 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rcdj\" (UniqueName: \"kubernetes.io/projected/33e8789a-aa76-4266-bf71-838add29dadb-kube-api-access-9rcdj\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xhljs\" (UID: \"33e8789a-aa76-4266-bf71-838add29dadb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhljs" Jul 22 03:15:37.574056 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.574036 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33e8789a-aa76-4266-bf71-838add29dadb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xhljs\" (UID: \"33e8789a-aa76-4266-bf71-838add29dadb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhljs" Jul 22 03:15:37.588120 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.588099 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9tgg\" (UniqueName: \"kubernetes.io/projected/abbcc799-0450-425a-be2d-f8f2324e6a23-kube-api-access-b9tgg\") pod \"limitador-operator-controller-manager-859d7b78bd-flnml\" (UID: \"abbcc799-0450-425a-be2d-f8f2324e6a23\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-flnml" Jul 22 03:15:37.591698 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.591677 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rcdj\" (UniqueName: \"kubernetes.io/projected/33e8789a-aa76-4266-bf71-838add29dadb-kube-api-access-9rcdj\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xhljs\" (UID: \"33e8789a-aa76-4266-bf71-838add29dadb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhljs" Jul 22 03:15:37.674465 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.674432 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ncj4v\" (UniqueName: \"kubernetes.io/projected/d4d065b2-4196-49c2-aa57-04630336da07-kube-api-access-ncj4v\") pod \"d4d065b2-4196-49c2-aa57-04630336da07\" (UID: \"d4d065b2-4196-49c2-aa57-04630336da07\") " Jul 22 03:15:37.674640 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.674574 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9w67k\" (UniqueName: \"kubernetes.io/projected/61813f2a-1f53-42e5-82a6-0ccdfffe0335-kube-api-access-9w67k\") pod \"61813f2a-1f53-42e5-82a6-0ccdfffe0335\" (UID: \"61813f2a-1f53-42e5-82a6-0ccdfffe0335\") " Jul 22 03:15:37.674640 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.674622 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/61813f2a-1f53-42e5-82a6-0ccdfffe0335-extensions-socket-volume\") pod \"61813f2a-1f53-42e5-82a6-0ccdfffe0335\" (UID: \"61813f2a-1f53-42e5-82a6-0ccdfffe0335\") " Jul 22 03:15:37.681054 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.675604 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61813f2a-1f53-42e5-82a6-0ccdfffe0335-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "61813f2a-1f53-42e5-82a6-0ccdfffe0335" (UID: "61813f2a-1f53-42e5-82a6-0ccdfffe0335"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:15:37.681054 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.677561 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61813f2a-1f53-42e5-82a6-0ccdfffe0335-kube-api-access-9w67k" (OuterVolumeSpecName: "kube-api-access-9w67k") pod "61813f2a-1f53-42e5-82a6-0ccdfffe0335" (UID: "61813f2a-1f53-42e5-82a6-0ccdfffe0335"). InnerVolumeSpecName "kube-api-access-9w67k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:15:37.681054 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.677615 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4d065b2-4196-49c2-aa57-04630336da07-kube-api-access-ncj4v" (OuterVolumeSpecName: "kube-api-access-ncj4v") pod "d4d065b2-4196-49c2-aa57-04630336da07" (UID: "d4d065b2-4196-49c2-aa57-04630336da07"). InnerVolumeSpecName "kube-api-access-ncj4v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:15:37.696475 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.696447 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhljs" Jul 22 03:15:37.703549 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.703528 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-flnml" Jul 22 03:15:37.776488 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.776431 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9w67k\" (UniqueName: \"kubernetes.io/projected/61813f2a-1f53-42e5-82a6-0ccdfffe0335-kube-api-access-9w67k\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:15:37.776488 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.776456 2556 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/61813f2a-1f53-42e5-82a6-0ccdfffe0335-extensions-socket-volume\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:15:37.776488 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.776467 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ncj4v\" (UniqueName: \"kubernetes.io/projected/d4d065b2-4196-49c2-aa57-04630336da07-kube-api-access-ncj4v\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:15:37.863909 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.863879 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhljs"] Jul 22 03:15:37.865853 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:15:37.865822 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33e8789a_aa76_4266_bf71_838add29dadb.slice/crio-20dad10d5dee5066ccb8bde7c4e3abe3bef5b9cd1cceb566ba2e30623dfaf20c WatchSource:0}: Error finding container 20dad10d5dee5066ccb8bde7c4e3abe3bef5b9cd1cceb566ba2e30623dfaf20c: Status 404 returned error can't find the container with id 20dad10d5dee5066ccb8bde7c4e3abe3bef5b9cd1cceb566ba2e30623dfaf20c Jul 22 03:15:37.897403 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:37.897385 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-flnml"] Jul 22 03:15:37.904004 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:15:37.903978 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabbcc799_0450_425a_be2d_f8f2324e6a23.slice/crio-1ba34f1a6bd59dc5bd78354e216b94e871c72bf1daaadec75411dc16684d6578 WatchSource:0}: Error finding container 1ba34f1a6bd59dc5bd78354e216b94e871c72bf1daaadec75411dc16684d6578: Status 404 returned error can't find the container with id 1ba34f1a6bd59dc5bd78354e216b94e871c72bf1daaadec75411dc16684d6578 Jul 22 03:15:38.024937 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:38.024859 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61813f2a-1f53-42e5-82a6-0ccdfffe0335" path="/var/lib/kubelet/pods/61813f2a-1f53-42e5-82a6-0ccdfffe0335/volumes" Jul 22 03:15:38.025228 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:38.025214 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4d065b2-4196-49c2-aa57-04630336da07" path="/var/lib/kubelet/pods/d4d065b2-4196-49c2-aa57-04630336da07/volumes" Jul 22 03:15:38.182384 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:38.182335 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-flnml" event={"ID":"abbcc799-0450-425a-be2d-f8f2324e6a23","Type":"ContainerStarted","Data":"5e1fd3650413acead8ac8ce1ef03989c2c0eb9b49540e3d5a38610182eab151c"} Jul 22 03:15:38.182583 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:38.182410 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-flnml" Jul 22 03:15:38.182583 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:38.182434 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-flnml" event={"ID":"abbcc799-0450-425a-be2d-f8f2324e6a23","Type":"ContainerStarted","Data":"1ba34f1a6bd59dc5bd78354e216b94e871c72bf1daaadec75411dc16684d6578"} Jul 22 03:15:38.183905 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:38.183865 2556 generic.go:356] "Generic (PLEG): container finished" podID="d4d065b2-4196-49c2-aa57-04630336da07" containerID="525877dfb28bcb4ad68ff8efc0fd58b08ac8beb23b4ef897467d46c51dee1d2d" exitCode=0 Jul 22 03:15:38.184050 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:38.183926 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pw672" Jul 22 03:15:38.184050 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:38.183995 2556 scope.go:117] "RemoveContainer" containerID="525877dfb28bcb4ad68ff8efc0fd58b08ac8beb23b4ef897467d46c51dee1d2d" Jul 22 03:15:38.186999 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:38.186931 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhljs" event={"ID":"33e8789a-aa76-4266-bf71-838add29dadb","Type":"ContainerStarted","Data":"308ef2bc9e02416f9271d4e641c2a2c5fe251fa12fdfa0397fd624d9eec9d1c8"} Jul 22 03:15:38.187116 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:38.187014 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhljs" event={"ID":"33e8789a-aa76-4266-bf71-838add29dadb","Type":"ContainerStarted","Data":"20dad10d5dee5066ccb8bde7c4e3abe3bef5b9cd1cceb566ba2e30623dfaf20c"} Jul 22 03:15:38.188454 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:38.188429 2556 generic.go:356] "Generic (PLEG): container finished" podID="61813f2a-1f53-42e5-82a6-0ccdfffe0335" containerID="52f670a4448cb26f05c21c2b249965ddbf63d0e363da54baf5ecf5b23495493b" exitCode=0 Jul 22 03:15:38.188550 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:38.188479 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxc8b" Jul 22 03:15:38.193682 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:38.193662 2556 scope.go:117] "RemoveContainer" containerID="525877dfb28bcb4ad68ff8efc0fd58b08ac8beb23b4ef897467d46c51dee1d2d" Jul 22 03:15:38.193992 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:15:38.193938 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"525877dfb28bcb4ad68ff8efc0fd58b08ac8beb23b4ef897467d46c51dee1d2d\": container with ID starting with 525877dfb28bcb4ad68ff8efc0fd58b08ac8beb23b4ef897467d46c51dee1d2d not found: ID does not exist" containerID="525877dfb28bcb4ad68ff8efc0fd58b08ac8beb23b4ef897467d46c51dee1d2d" Jul 22 03:15:38.193992 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:38.193978 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"525877dfb28bcb4ad68ff8efc0fd58b08ac8beb23b4ef897467d46c51dee1d2d"} err="failed to get container status \"525877dfb28bcb4ad68ff8efc0fd58b08ac8beb23b4ef897467d46c51dee1d2d\": rpc error: code = NotFound desc = could not find container \"525877dfb28bcb4ad68ff8efc0fd58b08ac8beb23b4ef897467d46c51dee1d2d\": container with ID starting with 525877dfb28bcb4ad68ff8efc0fd58b08ac8beb23b4ef897467d46c51dee1d2d not found: ID does not exist" Jul 22 03:15:38.194116 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:38.193995 2556 scope.go:117] "RemoveContainer" containerID="52f670a4448cb26f05c21c2b249965ddbf63d0e363da54baf5ecf5b23495493b" Jul 22 03:15:38.201783 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:38.201768 2556 scope.go:117] "RemoveContainer" containerID="52f670a4448cb26f05c21c2b249965ddbf63d0e363da54baf5ecf5b23495493b" Jul 22 03:15:38.202110 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:15:38.202091 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52f670a4448cb26f05c21c2b249965ddbf63d0e363da54baf5ecf5b23495493b\": container with ID starting with 52f670a4448cb26f05c21c2b249965ddbf63d0e363da54baf5ecf5b23495493b not found: ID does not exist" containerID="52f670a4448cb26f05c21c2b249965ddbf63d0e363da54baf5ecf5b23495493b" Jul 22 03:15:38.202170 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:38.202116 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52f670a4448cb26f05c21c2b249965ddbf63d0e363da54baf5ecf5b23495493b"} err="failed to get container status \"52f670a4448cb26f05c21c2b249965ddbf63d0e363da54baf5ecf5b23495493b\": rpc error: code = NotFound desc = could not find container \"52f670a4448cb26f05c21c2b249965ddbf63d0e363da54baf5ecf5b23495493b\": container with ID starting with 52f670a4448cb26f05c21c2b249965ddbf63d0e363da54baf5ecf5b23495493b not found: ID does not exist" Jul 22 03:15:38.220499 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:38.220457 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-flnml" podStartSLOduration=1.220443167 podStartE2EDuration="1.220443167s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:15:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:15:38.21969704 +0000 UTC m=+478.668680291" watchObservedRunningTime="2026-07-22 03:15:38.220443167 +0000 UTC m=+478.669426416" Jul 22 03:15:38.267427 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:38.267376 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhljs" podStartSLOduration=1.267358791 podStartE2EDuration="1.267358791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:15:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:15:38.265561255 +0000 UTC m=+478.714544505" watchObservedRunningTime="2026-07-22 03:15:38.267358791 +0000 UTC m=+478.716342041" Jul 22 03:15:38.328402 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:38.328369 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:38.523099 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:38.523061 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:38.933017 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:38.932981 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxmvv"] Jul 22 03:15:38.935816 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:38.935798 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxmvv" Jul 22 03:15:38.956915 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:38.956888 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxmvv"] Jul 22 03:15:39.086731 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:39.086692 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwllt\" (UniqueName: \"kubernetes.io/projected/2ade98d2-3be5-4f2f-9605-b15e334be488-kube-api-access-qwllt\") pod \"kuadrant-operator-controller-manager-86878c769f-bxmvv\" (UID: \"2ade98d2-3be5-4f2f-9605-b15e334be488\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxmvv" Jul 22 03:15:39.086931 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:39.086754 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ade98d2-3be5-4f2f-9605-b15e334be488-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bxmvv\" (UID: \"2ade98d2-3be5-4f2f-9605-b15e334be488\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxmvv" Jul 22 03:15:39.187903 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:39.187803 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwllt\" (UniqueName: \"kubernetes.io/projected/2ade98d2-3be5-4f2f-9605-b15e334be488-kube-api-access-qwllt\") pod \"kuadrant-operator-controller-manager-86878c769f-bxmvv\" (UID: \"2ade98d2-3be5-4f2f-9605-b15e334be488\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxmvv" Jul 22 03:15:39.187903 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:39.187869 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ade98d2-3be5-4f2f-9605-b15e334be488-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bxmvv\" (UID: \"2ade98d2-3be5-4f2f-9605-b15e334be488\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxmvv" Jul 22 03:15:39.188251 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:39.188230 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ade98d2-3be5-4f2f-9605-b15e334be488-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bxmvv\" (UID: \"2ade98d2-3be5-4f2f-9605-b15e334be488\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxmvv" Jul 22 03:15:39.196016 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:39.195993 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhljs" Jul 22 03:15:39.237544 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:39.237509 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwllt\" (UniqueName: \"kubernetes.io/projected/2ade98d2-3be5-4f2f-9605-b15e334be488-kube-api-access-qwllt\") pod \"kuadrant-operator-controller-manager-86878c769f-bxmvv\" (UID: \"2ade98d2-3be5-4f2f-9605-b15e334be488\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxmvv" Jul 22 03:15:39.245273 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:39.245241 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxmvv" Jul 22 03:15:39.439358 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:39.439331 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxmvv"] Jul 22 03:15:39.442167 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:15:39.442142 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ade98d2_3be5_4f2f_9605_b15e334be488.slice/crio-2a5502b64f34fbbaac8ca2a0bfdc4adbe41214352b7fe6a6dc1167e0433d8ef4 WatchSource:0}: Error finding container 2a5502b64f34fbbaac8ca2a0bfdc4adbe41214352b7fe6a6dc1167e0433d8ef4: Status 404 returned error can't find the container with id 2a5502b64f34fbbaac8ca2a0bfdc4adbe41214352b7fe6a6dc1167e0433d8ef4 Jul 22 03:15:40.202492 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:40.202447 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxmvv" event={"ID":"2ade98d2-3be5-4f2f-9605-b15e334be488","Type":"ContainerStarted","Data":"7185bc600f9b051ac1dae2804435cce4c52634fea08e07b71ef4b798a5494f7b"} Jul 22 03:15:40.202699 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:40.202583 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxmvv" Jul 22 03:15:40.202699 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:40.202612 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxmvv" event={"ID":"2ade98d2-3be5-4f2f-9605-b15e334be488","Type":"ContainerStarted","Data":"2a5502b64f34fbbaac8ca2a0bfdc4adbe41214352b7fe6a6dc1167e0433d8ef4"} Jul 22 03:15:40.232884 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:40.232823 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxmvv" podStartSLOduration=2.232807562 podStartE2EDuration="2.232807562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:15:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:15:40.229463586 +0000 UTC m=+480.678446835" watchObservedRunningTime="2026-07-22 03:15:40.232807562 +0000 UTC m=+480.681790811" Jul 22 03:15:41.779292 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:41.779256 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:41.877010 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:41.876947 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:41.976564 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:41.976530 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:45.517116 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:45.517065 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:45.621266 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:45.621222 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:49.198455 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:49.198427 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-flnml" Jul 22 03:15:49.334607 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:49.334578 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:49.441506 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:49.441466 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:50.131970 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:50.131915 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:50.205117 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:50.205091 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhljs" Jul 22 03:15:50.236645 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:50.236599 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:15:51.210694 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:51.210663 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxmvv" Jul 22 03:15:51.265801 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:51.265764 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhljs"] Jul 22 03:15:51.265988 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:51.265934 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhljs" podUID="33e8789a-aa76-4266-bf71-838add29dadb" containerName="manager" containerID="cri-o://308ef2bc9e02416f9271d4e641c2a2c5fe251fa12fdfa0397fd624d9eec9d1c8" gracePeriod=10 Jul 22 03:15:51.509992 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:51.509969 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhljs" Jul 22 03:15:51.701046 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:51.701016 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9rcdj\" (UniqueName: \"kubernetes.io/projected/33e8789a-aa76-4266-bf71-838add29dadb-kube-api-access-9rcdj\") pod \"33e8789a-aa76-4266-bf71-838add29dadb\" (UID: \"33e8789a-aa76-4266-bf71-838add29dadb\") " Jul 22 03:15:51.701225 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:51.701096 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33e8789a-aa76-4266-bf71-838add29dadb-extensions-socket-volume\") pod \"33e8789a-aa76-4266-bf71-838add29dadb\" (UID: \"33e8789a-aa76-4266-bf71-838add29dadb\") " Jul 22 03:15:51.701454 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:51.701432 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33e8789a-aa76-4266-bf71-838add29dadb-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "33e8789a-aa76-4266-bf71-838add29dadb" (UID: "33e8789a-aa76-4266-bf71-838add29dadb"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:15:51.703144 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:51.703117 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33e8789a-aa76-4266-bf71-838add29dadb-kube-api-access-9rcdj" (OuterVolumeSpecName: "kube-api-access-9rcdj") pod "33e8789a-aa76-4266-bf71-838add29dadb" (UID: "33e8789a-aa76-4266-bf71-838add29dadb"). InnerVolumeSpecName "kube-api-access-9rcdj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:15:51.802709 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:51.802632 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rcdj\" (UniqueName: \"kubernetes.io/projected/33e8789a-aa76-4266-bf71-838add29dadb-kube-api-access-9rcdj\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:15:51.802709 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:51.802661 2556 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33e8789a-aa76-4266-bf71-838add29dadb-extensions-socket-volume\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:15:52.265455 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:52.265418 2556 generic.go:356] "Generic (PLEG): container finished" podID="33e8789a-aa76-4266-bf71-838add29dadb" containerID="308ef2bc9e02416f9271d4e641c2a2c5fe251fa12fdfa0397fd624d9eec9d1c8" exitCode=0 Jul 22 03:15:52.265455 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:52.265456 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhljs" Jul 22 03:15:52.265937 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:52.265503 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhljs" event={"ID":"33e8789a-aa76-4266-bf71-838add29dadb","Type":"ContainerDied","Data":"308ef2bc9e02416f9271d4e641c2a2c5fe251fa12fdfa0397fd624d9eec9d1c8"} Jul 22 03:15:52.265937 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:52.265547 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhljs" event={"ID":"33e8789a-aa76-4266-bf71-838add29dadb","Type":"ContainerDied","Data":"20dad10d5dee5066ccb8bde7c4e3abe3bef5b9cd1cceb566ba2e30623dfaf20c"} Jul 22 03:15:52.265937 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:52.265570 2556 scope.go:117] "RemoveContainer" containerID="308ef2bc9e02416f9271d4e641c2a2c5fe251fa12fdfa0397fd624d9eec9d1c8" Jul 22 03:15:52.274257 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:52.274241 2556 scope.go:117] "RemoveContainer" containerID="308ef2bc9e02416f9271d4e641c2a2c5fe251fa12fdfa0397fd624d9eec9d1c8" Jul 22 03:15:52.274575 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:15:52.274554 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"308ef2bc9e02416f9271d4e641c2a2c5fe251fa12fdfa0397fd624d9eec9d1c8\": container with ID starting with 308ef2bc9e02416f9271d4e641c2a2c5fe251fa12fdfa0397fd624d9eec9d1c8 not found: ID does not exist" containerID="308ef2bc9e02416f9271d4e641c2a2c5fe251fa12fdfa0397fd624d9eec9d1c8" Jul 22 03:15:52.274630 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:52.274583 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"308ef2bc9e02416f9271d4e641c2a2c5fe251fa12fdfa0397fd624d9eec9d1c8"} err="failed to get container status \"308ef2bc9e02416f9271d4e641c2a2c5fe251fa12fdfa0397fd624d9eec9d1c8\": rpc error: code = NotFound desc = could not find container \"308ef2bc9e02416f9271d4e641c2a2c5fe251fa12fdfa0397fd624d9eec9d1c8\": container with ID starting with 308ef2bc9e02416f9271d4e641c2a2c5fe251fa12fdfa0397fd624d9eec9d1c8 not found: ID does not exist" Jul 22 03:15:52.285464 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:52.285443 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhljs"] Jul 22 03:15:52.292523 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:52.292501 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhljs"] Jul 22 03:15:54.024211 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:15:54.024179 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33e8789a-aa76-4266-bf71-838add29dadb" path="/var/lib/kubelet/pods/33e8789a-aa76-4266-bf71-838add29dadb/volumes" Jul 22 03:16:01.350093 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:01.350031 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-57fd7bf74d-kpksw" podUID="d2b3082a-2729-43ca-ba14-8ca41d1c83c1" containerName="console" containerID="cri-o://0a0ccdde824b44ff873d7bd84aac660813ec9532caee83ef1e7e03d29fbbad27" gracePeriod=15 Jul 22 03:16:01.605947 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:01.605890 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57fd7bf74d-kpksw" Jul 22 03:16:01.693795 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:01.693762 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-console-oauth-config\") pod \"d2b3082a-2729-43ca-ba14-8ca41d1c83c1\" (UID: \"d2b3082a-2729-43ca-ba14-8ca41d1c83c1\") " Jul 22 03:16:01.694001 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:01.693822 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-oauth-serving-cert\") pod \"d2b3082a-2729-43ca-ba14-8ca41d1c83c1\" (UID: \"d2b3082a-2729-43ca-ba14-8ca41d1c83c1\") " Jul 22 03:16:01.694001 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:01.693874 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-55rzv\" (UniqueName: \"kubernetes.io/projected/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-kube-api-access-55rzv\") pod \"d2b3082a-2729-43ca-ba14-8ca41d1c83c1\" (UID: \"d2b3082a-2729-43ca-ba14-8ca41d1c83c1\") " Jul 22 03:16:01.694001 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:01.693902 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-console-config\") pod \"d2b3082a-2729-43ca-ba14-8ca41d1c83c1\" (UID: \"d2b3082a-2729-43ca-ba14-8ca41d1c83c1\") " Jul 22 03:16:01.694001 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:01.693933 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-service-ca\") pod \"d2b3082a-2729-43ca-ba14-8ca41d1c83c1\" (UID: \"d2b3082a-2729-43ca-ba14-8ca41d1c83c1\") " Jul 22 03:16:01.694001 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:01.693974 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-console-serving-cert\") pod \"d2b3082a-2729-43ca-ba14-8ca41d1c83c1\" (UID: \"d2b3082a-2729-43ca-ba14-8ca41d1c83c1\") " Jul 22 03:16:01.694280 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:01.694012 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-trusted-ca-bundle\") pod \"d2b3082a-2729-43ca-ba14-8ca41d1c83c1\" (UID: \"d2b3082a-2729-43ca-ba14-8ca41d1c83c1\") " Jul 22 03:16:01.694350 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:01.694312 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d2b3082a-2729-43ca-ba14-8ca41d1c83c1" (UID: "d2b3082a-2729-43ca-ba14-8ca41d1c83c1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:16:01.694350 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:01.694320 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-console-config" (OuterVolumeSpecName: "console-config") pod "d2b3082a-2729-43ca-ba14-8ca41d1c83c1" (UID: "d2b3082a-2729-43ca-ba14-8ca41d1c83c1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:16:01.694732 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:01.694397 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-service-ca" (OuterVolumeSpecName: "service-ca") pod "d2b3082a-2729-43ca-ba14-8ca41d1c83c1" (UID: "d2b3082a-2729-43ca-ba14-8ca41d1c83c1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:16:01.694732 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:01.694582 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d2b3082a-2729-43ca-ba14-8ca41d1c83c1" (UID: "d2b3082a-2729-43ca-ba14-8ca41d1c83c1"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:16:01.696090 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:01.696066 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-kube-api-access-55rzv" (OuterVolumeSpecName: "kube-api-access-55rzv") pod "d2b3082a-2729-43ca-ba14-8ca41d1c83c1" (UID: "d2b3082a-2729-43ca-ba14-8ca41d1c83c1"). InnerVolumeSpecName "kube-api-access-55rzv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:16:01.696194 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:01.696104 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d2b3082a-2729-43ca-ba14-8ca41d1c83c1" (UID: "d2b3082a-2729-43ca-ba14-8ca41d1c83c1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:16:01.696194 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:01.696134 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d2b3082a-2729-43ca-ba14-8ca41d1c83c1" (UID: "d2b3082a-2729-43ca-ba14-8ca41d1c83c1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:16:01.795451 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:01.795417 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55rzv\" (UniqueName: \"kubernetes.io/projected/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-kube-api-access-55rzv\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:16:01.795451 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:01.795446 2556 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-console-config\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:16:01.795451 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:01.795459 2556 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-service-ca\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:16:01.795686 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:01.795470 2556 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-console-serving-cert\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:16:01.795686 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:01.795483 2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-trusted-ca-bundle\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:16:01.795686 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:01.795494 2556 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-console-oauth-config\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:16:01.795686 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:01.795504 2556 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d2b3082a-2729-43ca-ba14-8ca41d1c83c1-oauth-serving-cert\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:16:02.324210 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:02.324174 2556 generic.go:356] "Generic (PLEG): container finished" podID="d2b3082a-2729-43ca-ba14-8ca41d1c83c1" containerID="0a0ccdde824b44ff873d7bd84aac660813ec9532caee83ef1e7e03d29fbbad27" exitCode=0 Jul 22 03:16:02.324372 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:02.324222 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57fd7bf74d-kpksw" Jul 22 03:16:02.324372 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:02.324253 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57fd7bf74d-kpksw" event={"ID":"d2b3082a-2729-43ca-ba14-8ca41d1c83c1","Type":"ContainerDied","Data":"0a0ccdde824b44ff873d7bd84aac660813ec9532caee83ef1e7e03d29fbbad27"} Jul 22 03:16:02.324372 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:02.324302 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57fd7bf74d-kpksw" event={"ID":"d2b3082a-2729-43ca-ba14-8ca41d1c83c1","Type":"ContainerDied","Data":"d3fe2e596a214ae5d070580094578d5f5779156f6c1c1e2c7555dfd639b6d1ed"} Jul 22 03:16:02.324372 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:02.324321 2556 scope.go:117] "RemoveContainer" containerID="0a0ccdde824b44ff873d7bd84aac660813ec9532caee83ef1e7e03d29fbbad27" Jul 22 03:16:02.332908 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:02.332883 2556 scope.go:117] "RemoveContainer" containerID="0a0ccdde824b44ff873d7bd84aac660813ec9532caee83ef1e7e03d29fbbad27" Jul 22 03:16:02.333191 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:16:02.333173 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a0ccdde824b44ff873d7bd84aac660813ec9532caee83ef1e7e03d29fbbad27\": container with ID starting with 0a0ccdde824b44ff873d7bd84aac660813ec9532caee83ef1e7e03d29fbbad27 not found: ID does not exist" containerID="0a0ccdde824b44ff873d7bd84aac660813ec9532caee83ef1e7e03d29fbbad27" Jul 22 03:16:02.333271 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:02.333199 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a0ccdde824b44ff873d7bd84aac660813ec9532caee83ef1e7e03d29fbbad27"} err="failed to get container status \"0a0ccdde824b44ff873d7bd84aac660813ec9532caee83ef1e7e03d29fbbad27\": rpc error: code = NotFound desc = could not find container \"0a0ccdde824b44ff873d7bd84aac660813ec9532caee83ef1e7e03d29fbbad27\": container with ID starting with 0a0ccdde824b44ff873d7bd84aac660813ec9532caee83ef1e7e03d29fbbad27 not found: ID does not exist" Jul 22 03:16:02.343496 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:02.343476 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57fd7bf74d-kpksw"] Jul 22 03:16:02.347545 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:02.347521 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-57fd7bf74d-kpksw"] Jul 22 03:16:04.024085 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:04.024053 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2b3082a-2729-43ca-ba14-8ca41d1c83c1" path="/var/lib/kubelet/pods/d2b3082a-2729-43ca-ba14-8ca41d1c83c1/volumes" Jul 22 03:16:07.558319 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.558285 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn"] Jul 22 03:16:07.558798 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.558783 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2b3082a-2729-43ca-ba14-8ca41d1c83c1" containerName="console" Jul 22 03:16:07.558870 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.558801 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2b3082a-2729-43ca-ba14-8ca41d1c83c1" containerName="console" Jul 22 03:16:07.558870 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.558819 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33e8789a-aa76-4266-bf71-838add29dadb" containerName="manager" Jul 22 03:16:07.558870 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.558828 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="33e8789a-aa76-4266-bf71-838add29dadb" containerName="manager" Jul 22 03:16:07.558993 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.558919 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2b3082a-2729-43ca-ba14-8ca41d1c83c1" containerName="console" Jul 22 03:16:07.558993 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.558934 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="33e8789a-aa76-4266-bf71-838add29dadb" containerName="manager" Jul 22 03:16:07.563666 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.563645 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" Jul 22 03:16:07.566313 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.566294 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-z6j4n\"" Jul 22 03:16:07.572763 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.572743 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn"] Jul 22 03:16:07.582032 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.582000 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:16:07.641947 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.641909 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/221f388a-ed5d-4c97-a320-1339d242c437-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-4c9nn\" (UID: \"221f388a-ed5d-4c97-a320-1339d242c437\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" Jul 22 03:16:07.641947 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.641959 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/221f388a-ed5d-4c97-a320-1339d242c437-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-4c9nn\" (UID: \"221f388a-ed5d-4c97-a320-1339d242c437\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" Jul 22 03:16:07.642187 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.641994 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/221f388a-ed5d-4c97-a320-1339d242c437-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-4c9nn\" (UID: \"221f388a-ed5d-4c97-a320-1339d242c437\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" Jul 22 03:16:07.642187 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.642016 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/221f388a-ed5d-4c97-a320-1339d242c437-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-4c9nn\" (UID: \"221f388a-ed5d-4c97-a320-1339d242c437\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" Jul 22 03:16:07.642187 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.642058 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ht7rn\" (UniqueName: \"kubernetes.io/projected/221f388a-ed5d-4c97-a320-1339d242c437-kube-api-access-ht7rn\") pod \"maas-default-gateway-openshift-default-56654dcf7-4c9nn\" (UID: \"221f388a-ed5d-4c97-a320-1339d242c437\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" Jul 22 03:16:07.642187 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.642099 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/221f388a-ed5d-4c97-a320-1339d242c437-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-4c9nn\" (UID: \"221f388a-ed5d-4c97-a320-1339d242c437\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" Jul 22 03:16:07.642187 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.642128 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/221f388a-ed5d-4c97-a320-1339d242c437-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-4c9nn\" (UID: \"221f388a-ed5d-4c97-a320-1339d242c437\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" Jul 22 03:16:07.642187 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.642154 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/221f388a-ed5d-4c97-a320-1339d242c437-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-4c9nn\" (UID: \"221f388a-ed5d-4c97-a320-1339d242c437\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" Jul 22 03:16:07.642187 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.642171 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/221f388a-ed5d-4c97-a320-1339d242c437-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-4c9nn\" (UID: \"221f388a-ed5d-4c97-a320-1339d242c437\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" Jul 22 03:16:07.728333 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.728296 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:16:07.743396 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.743366 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/221f388a-ed5d-4c97-a320-1339d242c437-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-4c9nn\" (UID: \"221f388a-ed5d-4c97-a320-1339d242c437\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" Jul 22 03:16:07.743554 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.743401 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/221f388a-ed5d-4c97-a320-1339d242c437-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-4c9nn\" (UID: \"221f388a-ed5d-4c97-a320-1339d242c437\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" Jul 22 03:16:07.743554 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.743425 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/221f388a-ed5d-4c97-a320-1339d242c437-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-4c9nn\" (UID: \"221f388a-ed5d-4c97-a320-1339d242c437\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" Jul 22 03:16:07.743554 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.743459 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ht7rn\" (UniqueName: \"kubernetes.io/projected/221f388a-ed5d-4c97-a320-1339d242c437-kube-api-access-ht7rn\") pod \"maas-default-gateway-openshift-default-56654dcf7-4c9nn\" (UID: \"221f388a-ed5d-4c97-a320-1339d242c437\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" Jul 22 03:16:07.743554 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.743477 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/221f388a-ed5d-4c97-a320-1339d242c437-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-4c9nn\" (UID: \"221f388a-ed5d-4c97-a320-1339d242c437\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" Jul 22 03:16:07.743554 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.743498 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/221f388a-ed5d-4c97-a320-1339d242c437-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-4c9nn\" (UID: \"221f388a-ed5d-4c97-a320-1339d242c437\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" Jul 22 03:16:07.743824 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.743684 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/221f388a-ed5d-4c97-a320-1339d242c437-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-4c9nn\" (UID: \"221f388a-ed5d-4c97-a320-1339d242c437\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" Jul 22 03:16:07.743824 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.743744 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/221f388a-ed5d-4c97-a320-1339d242c437-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-4c9nn\" (UID: \"221f388a-ed5d-4c97-a320-1339d242c437\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" Jul 22 03:16:07.743824 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.743814 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/221f388a-ed5d-4c97-a320-1339d242c437-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-4c9nn\" (UID: \"221f388a-ed5d-4c97-a320-1339d242c437\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" Jul 22 03:16:07.743986 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.743862 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/221f388a-ed5d-4c97-a320-1339d242c437-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-4c9nn\" (UID: \"221f388a-ed5d-4c97-a320-1339d242c437\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" Jul 22 03:16:07.743986 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.743904 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/221f388a-ed5d-4c97-a320-1339d242c437-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-4c9nn\" (UID: \"221f388a-ed5d-4c97-a320-1339d242c437\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" Jul 22 03:16:07.744107 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.744086 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/221f388a-ed5d-4c97-a320-1339d242c437-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-4c9nn\" (UID: \"221f388a-ed5d-4c97-a320-1339d242c437\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" Jul 22 03:16:07.744184 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.744168 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/221f388a-ed5d-4c97-a320-1339d242c437-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-4c9nn\" (UID: \"221f388a-ed5d-4c97-a320-1339d242c437\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" Jul 22 03:16:07.744377 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.744357 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/221f388a-ed5d-4c97-a320-1339d242c437-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-4c9nn\" (UID: \"221f388a-ed5d-4c97-a320-1339d242c437\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" Jul 22 03:16:07.745812 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.745789 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/221f388a-ed5d-4c97-a320-1339d242c437-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-4c9nn\" (UID: \"221f388a-ed5d-4c97-a320-1339d242c437\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" Jul 22 03:16:07.746086 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.746067 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/221f388a-ed5d-4c97-a320-1339d242c437-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-4c9nn\" (UID: \"221f388a-ed5d-4c97-a320-1339d242c437\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" Jul 22 03:16:07.756036 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.756008 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ht7rn\" (UniqueName: \"kubernetes.io/projected/221f388a-ed5d-4c97-a320-1339d242c437-kube-api-access-ht7rn\") pod \"maas-default-gateway-openshift-default-56654dcf7-4c9nn\" (UID: \"221f388a-ed5d-4c97-a320-1339d242c437\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" Jul 22 03:16:07.758989 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.758967 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/221f388a-ed5d-4c97-a320-1339d242c437-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-4c9nn\" (UID: \"221f388a-ed5d-4c97-a320-1339d242c437\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" Jul 22 03:16:07.874782 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:07.874744 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" Jul 22 03:16:08.026444 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:08.026420 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn"] Jul 22 03:16:08.027704 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:16:08.027682 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod221f388a_ed5d_4c97_a320_1339d242c437.slice/crio-3b8f6fa17be169fcb5c5d79a90d392ddf5ce6d2657730843f4aebbedba680352 WatchSource:0}: Error finding container 3b8f6fa17be169fcb5c5d79a90d392ddf5ce6d2657730843f4aebbedba680352: Status 404 returned error can't find the container with id 3b8f6fa17be169fcb5c5d79a90d392ddf5ce6d2657730843f4aebbedba680352 Jul 22 03:16:08.029881 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:08.029854 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:16:08.029977 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:08.029916 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:16:08.029977 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:08.029944 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:16:08.357764 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:08.357729 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" event={"ID":"221f388a-ed5d-4c97-a320-1339d242c437","Type":"ContainerStarted","Data":"be10699decf7e2ab517ba52646aebf0c133bf77d7d753e8e57a1792916de4851"} Jul 22 03:16:08.357944 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:08.357781 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" event={"ID":"221f388a-ed5d-4c97-a320-1339d242c437","Type":"ContainerStarted","Data":"3b8f6fa17be169fcb5c5d79a90d392ddf5ce6d2657730843f4aebbedba680352"} Jul 22 03:16:08.381636 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:08.381586 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" podStartSLOduration=1.381570747 podStartE2EDuration="1.381570747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:16:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:16:08.379246853 +0000 UTC m=+508.828230119" watchObservedRunningTime="2026-07-22 03:16:08.381570747 +0000 UTC m=+508.830553997" Jul 22 03:16:08.875091 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:08.875055 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" Jul 22 03:16:09.462138 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:09.462093 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:16:09.565654 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:09.565617 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:16:09.629698 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:09.629662 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:16:09.844723 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:09.844690 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:16:09.848236 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:09.848215 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-5m587" Jul 22 03:16:09.850805 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:09.850786 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-2pb9b\"" Jul 22 03:16:09.850904 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:09.850792 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 22 03:16:09.857434 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:09.857409 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:16:09.880289 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:09.880261 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" Jul 22 03:16:09.888907 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:09.888878 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:16:09.963551 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:09.963517 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4e4f6830-a3fd-4bd8-bd05-b5827da572f3-config-file\") pod \"limitador-limitador-54d6d988d5-5m587\" (UID: \"4e4f6830-a3fd-4bd8-bd05-b5827da572f3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5m587" Jul 22 03:16:09.963741 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:09.963584 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xmbw\" (UniqueName: \"kubernetes.io/projected/4e4f6830-a3fd-4bd8-bd05-b5827da572f3-kube-api-access-5xmbw\") pod \"limitador-limitador-54d6d988d5-5m587\" (UID: \"4e4f6830-a3fd-4bd8-bd05-b5827da572f3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5m587" Jul 22 03:16:10.064084 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:10.064050 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4e4f6830-a3fd-4bd8-bd05-b5827da572f3-config-file\") pod \"limitador-limitador-54d6d988d5-5m587\" (UID: \"4e4f6830-a3fd-4bd8-bd05-b5827da572f3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5m587" Jul 22 03:16:10.064256 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:10.064110 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xmbw\" (UniqueName: \"kubernetes.io/projected/4e4f6830-a3fd-4bd8-bd05-b5827da572f3-kube-api-access-5xmbw\") pod \"limitador-limitador-54d6d988d5-5m587\" (UID: \"4e4f6830-a3fd-4bd8-bd05-b5827da572f3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5m587" Jul 22 03:16:10.064781 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:10.064759 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4e4f6830-a3fd-4bd8-bd05-b5827da572f3-config-file\") pod \"limitador-limitador-54d6d988d5-5m587\" (UID: \"4e4f6830-a3fd-4bd8-bd05-b5827da572f3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5m587" Jul 22 03:16:10.072521 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:10.072495 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xmbw\" (UniqueName: \"kubernetes.io/projected/4e4f6830-a3fd-4bd8-bd05-b5827da572f3-kube-api-access-5xmbw\") pod \"limitador-limitador-54d6d988d5-5m587\" (UID: \"4e4f6830-a3fd-4bd8-bd05-b5827da572f3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5m587" Jul 22 03:16:10.160119 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:10.160039 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-5m587" Jul 22 03:16:10.294065 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:10.294030 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:16:10.300182 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:10.300163 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:16:10.301817 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:16:10.301792 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e4f6830_a3fd_4bd8_bd05_b5827da572f3.slice/crio-53ceb0817319fc086f567c4b1401cd26749d9cd03b8abaa0625cb71e0f85f243 WatchSource:0}: Error finding container 53ceb0817319fc086f567c4b1401cd26749d9cd03b8abaa0625cb71e0f85f243: Status 404 returned error can't find the container with id 53ceb0817319fc086f567c4b1401cd26749d9cd03b8abaa0625cb71e0f85f243 Jul 22 03:16:10.368862 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:10.368827 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-5m587" event={"ID":"4e4f6830-a3fd-4bd8-bd05-b5827da572f3","Type":"ContainerStarted","Data":"53ceb0817319fc086f567c4b1401cd26749d9cd03b8abaa0625cb71e0f85f243"} Jul 22 03:16:10.374853 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:10.369843 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" Jul 22 03:16:10.374853 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:10.371049 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" Jul 22 03:16:10.402267 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:10.402231 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:16:13.387141 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:13.387108 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-5m587" event={"ID":"4e4f6830-a3fd-4bd8-bd05-b5827da572f3","Type":"ContainerStarted","Data":"dc962c581ecaa841571044be8a8c2f254c1bfa813a972758b6284944b4b20569"} Jul 22 03:16:13.387541 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:13.387178 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-5m587" Jul 22 03:16:13.405480 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:13.405428 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-5m587" podStartSLOduration=1.802218699 podStartE2EDuration="4.405413161s" totalImagesPullingTime="2.603194462s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:16:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:16:13.404236132 +0000 UTC m=+513.853219380" watchObservedRunningTime="2026-07-22 03:16:13.405413161 +0000 UTC m=+513.854396410" Jul 22 03:16:24.393546 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:24.393509 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-5m587" Jul 22 03:16:25.484915 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:25.484878 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:16:25.658205 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:25.658166 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:16:25.952464 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:25.952433 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:16:37.618041 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:37.617941 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:16:38.287866 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:38.287822 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:16:44.341470 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:44.341430 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:16:44.443514 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:44.443474 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:16:44.558648 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:44.558600 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:16:44.646259 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:44.646157 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:16:47.774040 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:47.774001 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:16:47.856484 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:47.856446 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:16:58.623774 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:16:58.623740 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:17:01.091480 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:01.091439 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:17:13.876351 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:13.876319 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:17:13.933056 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:13.933028 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:17:20.354264 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:20.354227 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:17:20.501225 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:20.501196 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:17:20.882010 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:20.881975 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:17:21.322001 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:21.321968 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:17:21.425857 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:21.425826 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:17:21.522659 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:21.522629 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:17:21.542654 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:21.542619 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:17:29.778317 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:29.778283 2556 generic.go:356] "Generic (PLEG): container finished" podID="221f388a-ed5d-4c97-a320-1339d242c437" containerID="be10699decf7e2ab517ba52646aebf0c133bf77d7d753e8e57a1792916de4851" exitCode=0 Jul 22 03:17:29.778317 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:29.778322 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" event={"ID":"221f388a-ed5d-4c97-a320-1339d242c437","Type":"ContainerDied","Data":"be10699decf7e2ab517ba52646aebf0c133bf77d7d753e8e57a1792916de4851"} Jul 22 03:17:29.778786 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:29.778694 2556 scope.go:117] "RemoveContainer" containerID="be10699decf7e2ab517ba52646aebf0c133bf77d7d753e8e57a1792916de4851" Jul 22 03:17:29.779652 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:29.779622 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:17:29.779703 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:29.779691 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:17:29.779746 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:29.779722 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:17:30.790235 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:30.790176 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" event={"ID":"221f388a-ed5d-4c97-a320-1339d242c437","Type":"ContainerStarted","Data":"b6b371c41c145f046ff8abd312481321a9d5d83adbc19ff3e6e793db2eaeefc8"} Jul 22 03:17:30.875701 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:30.875656 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" Jul 22 03:17:30.876001 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:30.875973 2556 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-4c9nn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 22 03:17:30.876094 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:30.876036 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" podUID="221f388a-ed5d-4c97-a320-1339d242c437" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 22 03:17:31.875442 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:31.875393 2556 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-4c9nn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 22 03:17:31.875942 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:31.875466 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" podUID="221f388a-ed5d-4c97-a320-1339d242c437" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 22 03:17:32.520650 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:32.520611 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:17:32.875766 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:32.875731 2556 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-4c9nn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 22 03:17:32.876355 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:32.876325 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" podUID="221f388a-ed5d-4c97-a320-1339d242c437" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 22 03:17:33.875844 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:33.875423 2556 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-4c9nn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 22 03:17:33.875844 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:33.875481 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" podUID="221f388a-ed5d-4c97-a320-1339d242c437" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 22 03:17:33.961931 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:33.961885 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:17:34.878885 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:34.878858 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" Jul 22 03:17:34.879308 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:34.879269 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" Jul 22 03:17:34.879783 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:34.879768 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4c9nn" Jul 22 03:17:38.165220 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:38.165188 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:17:38.726786 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:38.726749 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:17:40.038517 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:40.038487 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscf8_a01dfba4-f590-412a-8438-1646c51bb1a1/ovn-acl-logging/0.log" Jul 22 03:17:40.039946 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:40.039920 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscf8_a01dfba4-f590-412a-8438-1646c51bb1a1/ovn-acl-logging/0.log" Jul 22 03:17:43.329940 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:43.329908 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:17:49.729865 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:49.729829 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-rx4vr"] Jul 22 03:17:49.732328 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:49.732307 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rx4vr" Jul 22 03:17:49.736005 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:49.735985 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 22 03:17:49.736134 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:49.736047 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 22 03:17:49.736134 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:49.736083 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 22 03:17:49.736260 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:49.736133 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-9dh47\"" Jul 22 03:17:49.741377 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:49.741359 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-rx4vr"] Jul 22 03:17:49.755188 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:49.755160 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:17:49.852561 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:49.852524 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f17974a8-c3f1-42ed-bc11-eec56e94281d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rx4vr\" (UID: \"f17974a8-c3f1-42ed-bc11-eec56e94281d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rx4vr" Jul 22 03:17:49.852561 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:49.852562 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f17974a8-c3f1-42ed-bc11-eec56e94281d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rx4vr\" (UID: \"f17974a8-c3f1-42ed-bc11-eec56e94281d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rx4vr" Jul 22 03:17:49.852780 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:49.852630 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f17974a8-c3f1-42ed-bc11-eec56e94281d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rx4vr\" (UID: \"f17974a8-c3f1-42ed-bc11-eec56e94281d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rx4vr" Jul 22 03:17:49.852780 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:49.852731 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f17974a8-c3f1-42ed-bc11-eec56e94281d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rx4vr\" (UID: \"f17974a8-c3f1-42ed-bc11-eec56e94281d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rx4vr" Jul 22 03:17:49.852780 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:49.852760 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lgzk\" (UniqueName: \"kubernetes.io/projected/f17974a8-c3f1-42ed-bc11-eec56e94281d-kube-api-access-8lgzk\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rx4vr\" (UID: \"f17974a8-c3f1-42ed-bc11-eec56e94281d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rx4vr" Jul 22 03:17:49.852892 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:49.852780 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f17974a8-c3f1-42ed-bc11-eec56e94281d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rx4vr\" (UID: \"f17974a8-c3f1-42ed-bc11-eec56e94281d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rx4vr" Jul 22 03:17:49.953779 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:49.953744 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f17974a8-c3f1-42ed-bc11-eec56e94281d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rx4vr\" (UID: \"f17974a8-c3f1-42ed-bc11-eec56e94281d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rx4vr" Jul 22 03:17:49.954002 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:49.953808 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f17974a8-c3f1-42ed-bc11-eec56e94281d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rx4vr\" (UID: \"f17974a8-c3f1-42ed-bc11-eec56e94281d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rx4vr" Jul 22 03:17:49.954002 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:49.953836 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lgzk\" (UniqueName: \"kubernetes.io/projected/f17974a8-c3f1-42ed-bc11-eec56e94281d-kube-api-access-8lgzk\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rx4vr\" (UID: \"f17974a8-c3f1-42ed-bc11-eec56e94281d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rx4vr" Jul 22 03:17:49.954002 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:49.953861 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f17974a8-c3f1-42ed-bc11-eec56e94281d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rx4vr\" (UID: \"f17974a8-c3f1-42ed-bc11-eec56e94281d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rx4vr" Jul 22 03:17:49.954002 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:49.953922 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f17974a8-c3f1-42ed-bc11-eec56e94281d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rx4vr\" (UID: \"f17974a8-c3f1-42ed-bc11-eec56e94281d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rx4vr" Jul 22 03:17:49.954250 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:49.953947 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f17974a8-c3f1-42ed-bc11-eec56e94281d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rx4vr\" (UID: \"f17974a8-c3f1-42ed-bc11-eec56e94281d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rx4vr" Jul 22 03:17:49.954250 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:49.954199 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f17974a8-c3f1-42ed-bc11-eec56e94281d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rx4vr\" (UID: \"f17974a8-c3f1-42ed-bc11-eec56e94281d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rx4vr" Jul 22 03:17:49.954357 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:49.954281 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f17974a8-c3f1-42ed-bc11-eec56e94281d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rx4vr\" (UID: \"f17974a8-c3f1-42ed-bc11-eec56e94281d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rx4vr" Jul 22 03:17:49.954357 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:49.954338 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f17974a8-c3f1-42ed-bc11-eec56e94281d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rx4vr\" (UID: \"f17974a8-c3f1-42ed-bc11-eec56e94281d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rx4vr" Jul 22 03:17:49.956213 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:49.956190 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f17974a8-c3f1-42ed-bc11-eec56e94281d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rx4vr\" (UID: \"f17974a8-c3f1-42ed-bc11-eec56e94281d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rx4vr" Jul 22 03:17:49.956547 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:49.956524 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f17974a8-c3f1-42ed-bc11-eec56e94281d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rx4vr\" (UID: \"f17974a8-c3f1-42ed-bc11-eec56e94281d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rx4vr" Jul 22 03:17:49.961658 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:49.961638 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lgzk\" (UniqueName: \"kubernetes.io/projected/f17974a8-c3f1-42ed-bc11-eec56e94281d-kube-api-access-8lgzk\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rx4vr\" (UID: \"f17974a8-c3f1-42ed-bc11-eec56e94281d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rx4vr" Jul 22 03:17:50.043937 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:50.043850 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rx4vr" Jul 22 03:17:50.183469 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:50.183435 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-rx4vr"] Jul 22 03:17:50.183589 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:17:50.183540 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf17974a8_c3f1_42ed_bc11_eec56e94281d.slice/crio-e3c3ff6af6cf3ef8bc03a5869f495493b51dab3c67672a325bf2ba6f37cca1c3 WatchSource:0}: Error finding container e3c3ff6af6cf3ef8bc03a5869f495493b51dab3c67672a325bf2ba6f37cca1c3: Status 404 returned error can't find the container with id e3c3ff6af6cf3ef8bc03a5869f495493b51dab3c67672a325bf2ba6f37cca1c3 Jul 22 03:17:50.228877 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:50.228849 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:17:50.896988 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:50.896926 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rx4vr" event={"ID":"f17974a8-c3f1-42ed-bc11-eec56e94281d","Type":"ContainerStarted","Data":"e3c3ff6af6cf3ef8bc03a5869f495493b51dab3c67672a325bf2ba6f37cca1c3"} Jul 22 03:17:51.909581 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:51.909535 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rx4vr" event={"ID":"f17974a8-c3f1-42ed-bc11-eec56e94281d","Type":"ContainerStarted","Data":"724db74568392a67dc6d229973ea610b0fa9b75693a655e473396df6d5b68755"} Jul 22 03:17:51.930215 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:17:51.930161 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rx4vr" podStartSLOduration=1.946636164 podStartE2EDuration="2.930146678s" totalImagesPullingTime="983.510514ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:17:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:17:51.928129761 +0000 UTC m=+612.377113011" watchObservedRunningTime="2026-07-22 03:17:51.930146678 +0000 UTC m=+612.379129926" Jul 22 03:18:00.044109 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:00.044023 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rx4vr" Jul 22 03:18:00.044109 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:00.044089 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rx4vr" Jul 22 03:18:00.056702 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:00.056681 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rx4vr" Jul 22 03:18:00.968432 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:00.968404 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rx4vr" Jul 22 03:18:11.584141 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:11.584100 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:18:12.634492 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:12.634451 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:18:20.653508 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:20.653470 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:18:22.031210 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:22.031173 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:18:33.422679 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:33.422625 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-vzhkz"] Jul 22 03:18:33.427816 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:33.427785 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vzhkz" Jul 22 03:18:33.431139 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:33.431105 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 22 03:18:33.437051 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:33.437028 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-vzhkz"] Jul 22 03:18:33.446343 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:33.446312 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/131ea682-fde2-41ab-9593-490f403e63b8-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vzhkz\" (UID: \"131ea682-fde2-41ab-9593-490f403e63b8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vzhkz" Jul 22 03:18:33.446538 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:33.446516 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c77bd\" (UniqueName: \"kubernetes.io/projected/131ea682-fde2-41ab-9593-490f403e63b8-kube-api-access-c77bd\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vzhkz\" (UID: \"131ea682-fde2-41ab-9593-490f403e63b8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vzhkz" Jul 22 03:18:33.446881 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:33.446861 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/131ea682-fde2-41ab-9593-490f403e63b8-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vzhkz\" (UID: \"131ea682-fde2-41ab-9593-490f403e63b8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vzhkz" Jul 22 03:18:33.447048 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:33.447029 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/131ea682-fde2-41ab-9593-490f403e63b8-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vzhkz\" (UID: \"131ea682-fde2-41ab-9593-490f403e63b8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vzhkz" Jul 22 03:18:33.447180 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:33.447164 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/131ea682-fde2-41ab-9593-490f403e63b8-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vzhkz\" (UID: \"131ea682-fde2-41ab-9593-490f403e63b8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vzhkz" Jul 22 03:18:33.447349 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:33.447319 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/131ea682-fde2-41ab-9593-490f403e63b8-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vzhkz\" (UID: \"131ea682-fde2-41ab-9593-490f403e63b8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vzhkz" Jul 22 03:18:33.455914 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:33.455885 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:18:33.548388 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:33.548334 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/131ea682-fde2-41ab-9593-490f403e63b8-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vzhkz\" (UID: \"131ea682-fde2-41ab-9593-490f403e63b8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vzhkz" Jul 22 03:18:33.548587 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:33.548412 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/131ea682-fde2-41ab-9593-490f403e63b8-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vzhkz\" (UID: \"131ea682-fde2-41ab-9593-490f403e63b8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vzhkz" Jul 22 03:18:33.548587 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:33.548441 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/131ea682-fde2-41ab-9593-490f403e63b8-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vzhkz\" (UID: \"131ea682-fde2-41ab-9593-490f403e63b8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vzhkz" Jul 22 03:18:33.548587 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:33.548504 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/131ea682-fde2-41ab-9593-490f403e63b8-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vzhkz\" (UID: \"131ea682-fde2-41ab-9593-490f403e63b8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vzhkz" Jul 22 03:18:33.548587 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:33.548560 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/131ea682-fde2-41ab-9593-490f403e63b8-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vzhkz\" (UID: \"131ea682-fde2-41ab-9593-490f403e63b8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vzhkz" Jul 22 03:18:33.548808 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:33.548591 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c77bd\" (UniqueName: \"kubernetes.io/projected/131ea682-fde2-41ab-9593-490f403e63b8-kube-api-access-c77bd\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vzhkz\" (UID: \"131ea682-fde2-41ab-9593-490f403e63b8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vzhkz" Jul 22 03:18:33.549083 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:33.549054 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/131ea682-fde2-41ab-9593-490f403e63b8-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vzhkz\" (UID: \"131ea682-fde2-41ab-9593-490f403e63b8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vzhkz" Jul 22 03:18:33.549387 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:33.549306 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/131ea682-fde2-41ab-9593-490f403e63b8-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vzhkz\" (UID: \"131ea682-fde2-41ab-9593-490f403e63b8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vzhkz" Jul 22 03:18:33.549495 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:33.549441 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/131ea682-fde2-41ab-9593-490f403e63b8-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vzhkz\" (UID: \"131ea682-fde2-41ab-9593-490f403e63b8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vzhkz" Jul 22 03:18:33.551043 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:33.551015 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/131ea682-fde2-41ab-9593-490f403e63b8-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vzhkz\" (UID: \"131ea682-fde2-41ab-9593-490f403e63b8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vzhkz" Jul 22 03:18:33.551534 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:33.551513 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/131ea682-fde2-41ab-9593-490f403e63b8-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vzhkz\" (UID: \"131ea682-fde2-41ab-9593-490f403e63b8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vzhkz" Jul 22 03:18:33.558409 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:33.558382 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c77bd\" (UniqueName: \"kubernetes.io/projected/131ea682-fde2-41ab-9593-490f403e63b8-kube-api-access-c77bd\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vzhkz\" (UID: \"131ea682-fde2-41ab-9593-490f403e63b8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vzhkz" Jul 22 03:18:33.740840 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:33.740743 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vzhkz" Jul 22 03:18:33.872942 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:33.872914 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-vzhkz"] Jul 22 03:18:33.875159 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:18:33.875130 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod131ea682_fde2_41ab_9593_490f403e63b8.slice/crio-38157ee499bc96a5c0f16fd528ba10fd9b860937f3cbce28a73e6f3687b47f83 WatchSource:0}: Error finding container 38157ee499bc96a5c0f16fd528ba10fd9b860937f3cbce28a73e6f3687b47f83: Status 404 returned error can't find the container with id 38157ee499bc96a5c0f16fd528ba10fd9b860937f3cbce28a73e6f3687b47f83 Jul 22 03:18:33.876874 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:33.876855 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 03:18:34.135649 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:34.135616 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vzhkz" event={"ID":"131ea682-fde2-41ab-9593-490f403e63b8","Type":"ContainerStarted","Data":"5c4b0e1c0b74398083e0e6d54b3bb00cd12a531222a709ad03bb7b00aa66c14b"} Jul 22 03:18:34.135837 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:34.135813 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vzhkz" event={"ID":"131ea682-fde2-41ab-9593-490f403e63b8","Type":"ContainerStarted","Data":"38157ee499bc96a5c0f16fd528ba10fd9b860937f3cbce28a73e6f3687b47f83"} Jul 22 03:18:34.154436 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:34.154387 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vzhkz" podStartSLOduration=0.997526424 podStartE2EDuration="1.154370056s" totalImagesPullingTime="156.843632ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:18:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:18:34.151622655 +0000 UTC m=+654.600605898" watchObservedRunningTime="2026-07-22 03:18:34.154370056 +0000 UTC m=+654.603353306" Jul 22 03:18:36.723459 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:36.723422 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:18:43.741646 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:43.741593 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vzhkz" Jul 22 03:18:43.742080 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:43.741703 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vzhkz" Jul 22 03:18:43.754704 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:43.754674 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vzhkz" Jul 22 03:18:44.207835 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:44.207806 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vzhkz" Jul 22 03:18:52.160822 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:52.160781 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:18:53.833813 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:18:53.833777 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:19:58.742903 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:19:58.742843 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:19:58.812926 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:19:58.812891 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:19:59.357569 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:19:59.357514 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:19:59.426224 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:19:59.426171 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:19:59.943644 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:19:59.943602 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:20:00.033040 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:20:00.032975 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:20:28.235508 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:20:28.235474 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:20:37.744438 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:20:37.744399 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:20:48.337194 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:20:48.337158 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:21:39.395939 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:21:39.395904 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:22:04.136666 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:22:04.136619 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:22:05.671422 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:22:05.671371 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:22:25.812489 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:22:25.812449 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:22:25.843209 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:22:25.843176 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:22:36.356944 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:22:36.356849 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:22:40.079919 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:22:40.079892 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscf8_a01dfba4-f590-412a-8438-1646c51bb1a1/ovn-acl-logging/0.log" Jul 22 03:22:40.080357 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:22:40.079893 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscf8_a01dfba4-f590-412a-8438-1646c51bb1a1/ovn-acl-logging/0.log" Jul 22 03:22:44.835348 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:22:44.835313 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:22:46.161933 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:22:46.161895 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:22:51.694864 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:22:51.694824 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:22:51.811316 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:22:51.811268 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:22:51.949278 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:22:51.949192 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:22:52.130516 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:22:52.130482 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:22:57.845003 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:22:57.844947 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:22:57.944941 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:22:57.944876 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:23:15.034709 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:23:15.034675 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:23:24.334033 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:23:24.333994 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:23:32.853639 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:23:32.853601 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:23:36.864285 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:23:36.864254 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:23:42.134037 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:23:42.133996 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:23:52.842810 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:23:52.842771 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:24:02.434274 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:24:02.434189 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:24:13.036025 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:24:13.035991 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:24:22.940414 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:24:22.940378 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:24:45.323704 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:24:45.323668 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:24:57.029219 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:24:57.029185 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:25:14.710323 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:25:14.710278 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:25:20.140792 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:25:20.140756 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:25:30.727689 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:25:30.727611 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:25:46.047799 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:25:46.047754 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:25:55.638375 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:25:55.638337 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:26:05.042491 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:26:05.042457 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:26:14.738383 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:26:14.738342 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:26:23.133090 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:26:23.133056 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:26:50.352815 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:26:50.352769 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxmvv"] Jul 22 03:26:50.353421 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:26:50.353035 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxmvv" podUID="2ade98d2-3be5-4f2f-9605-b15e334be488" containerName="manager" containerID="cri-o://7185bc600f9b051ac1dae2804435cce4c52634fea08e07b71ef4b798a5494f7b" gracePeriod=10 Jul 22 03:26:53.825977 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:26:53.825919 2556 generic.go:356] "Generic (PLEG): container finished" podID="2ade98d2-3be5-4f2f-9605-b15e334be488" containerID="7185bc600f9b051ac1dae2804435cce4c52634fea08e07b71ef4b798a5494f7b" exitCode=0 Jul 22 03:26:53.826311 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:26:53.825981 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxmvv" event={"ID":"2ade98d2-3be5-4f2f-9605-b15e334be488","Type":"ContainerDied","Data":"7185bc600f9b051ac1dae2804435cce4c52634fea08e07b71ef4b798a5494f7b"} Jul 22 03:26:53.905624 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:26:53.905594 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxmvv" Jul 22 03:26:54.009992 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:26:54.009937 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ade98d2-3be5-4f2f-9605-b15e334be488-extensions-socket-volume\") pod \"2ade98d2-3be5-4f2f-9605-b15e334be488\" (UID: \"2ade98d2-3be5-4f2f-9605-b15e334be488\") " Jul 22 03:26:54.010176 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:26:54.010013 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qwllt\" (UniqueName: \"kubernetes.io/projected/2ade98d2-3be5-4f2f-9605-b15e334be488-kube-api-access-qwllt\") pod \"2ade98d2-3be5-4f2f-9605-b15e334be488\" (UID: \"2ade98d2-3be5-4f2f-9605-b15e334be488\") " Jul 22 03:26:54.010298 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:26:54.010274 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ade98d2-3be5-4f2f-9605-b15e334be488-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2ade98d2-3be5-4f2f-9605-b15e334be488" (UID: "2ade98d2-3be5-4f2f-9605-b15e334be488"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:26:54.012084 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:26:54.012058 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ade98d2-3be5-4f2f-9605-b15e334be488-kube-api-access-qwllt" (OuterVolumeSpecName: "kube-api-access-qwllt") pod "2ade98d2-3be5-4f2f-9605-b15e334be488" (UID: "2ade98d2-3be5-4f2f-9605-b15e334be488"). InnerVolumeSpecName "kube-api-access-qwllt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:26:54.111178 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:26:54.111148 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwllt\" (UniqueName: \"kubernetes.io/projected/2ade98d2-3be5-4f2f-9605-b15e334be488-kube-api-access-qwllt\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:26:54.111178 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:26:54.111175 2556 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ade98d2-3be5-4f2f-9605-b15e334be488-extensions-socket-volume\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:26:54.832714 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:26:54.832677 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxmvv" Jul 22 03:26:54.832714 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:26:54.832712 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxmvv" event={"ID":"2ade98d2-3be5-4f2f-9605-b15e334be488","Type":"ContainerDied","Data":"2a5502b64f34fbbaac8ca2a0bfdc4adbe41214352b7fe6a6dc1167e0433d8ef4"} Jul 22 03:26:54.833199 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:26:54.832772 2556 scope.go:117] "RemoveContainer" containerID="7185bc600f9b051ac1dae2804435cce4c52634fea08e07b71ef4b798a5494f7b" Jul 22 03:26:54.851786 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:26:54.851758 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxmvv"] Jul 22 03:26:54.855885 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:26:54.855863 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxmvv"] Jul 22 03:26:56.024029 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:26:56.023990 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ade98d2-3be5-4f2f-9605-b15e334be488" path="/var/lib/kubelet/pods/2ade98d2-3be5-4f2f-9605-b15e334be488/volumes" Jul 22 03:27:40.117547 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:27:40.117517 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscf8_a01dfba4-f590-412a-8438-1646c51bb1a1/ovn-acl-logging/0.log" Jul 22 03:27:40.119902 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:27:40.118658 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscf8_a01dfba4-f590-412a-8438-1646c51bb1a1/ovn-acl-logging/0.log" Jul 22 03:27:51.541573 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:27:51.541538 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zvqzj"] Jul 22 03:27:51.542033 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:27:51.541920 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ade98d2-3be5-4f2f-9605-b15e334be488" containerName="manager" Jul 22 03:27:51.542033 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:27:51.541931 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ade98d2-3be5-4f2f-9605-b15e334be488" containerName="manager" Jul 22 03:27:51.542033 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:27:51.542015 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ade98d2-3be5-4f2f-9605-b15e334be488" containerName="manager" Jul 22 03:27:51.546255 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:27:51.546237 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zvqzj" Jul 22 03:27:51.551944 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:27:51.551920 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-zr6ln\"" Jul 22 03:27:51.564964 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:27:51.564925 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zvqzj"] Jul 22 03:27:51.625232 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:27:51.625200 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/50ce2458-b52d-4a77-bdab-da22cc84dafd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zvqzj\" (UID: \"50ce2458-b52d-4a77-bdab-da22cc84dafd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zvqzj" Jul 22 03:27:51.625396 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:27:51.625276 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmn9l\" (UniqueName: \"kubernetes.io/projected/50ce2458-b52d-4a77-bdab-da22cc84dafd-kube-api-access-kmn9l\") pod \"kuadrant-operator-controller-manager-86878c769f-zvqzj\" (UID: \"50ce2458-b52d-4a77-bdab-da22cc84dafd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zvqzj" Jul 22 03:27:51.726271 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:27:51.726235 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmn9l\" (UniqueName: \"kubernetes.io/projected/50ce2458-b52d-4a77-bdab-da22cc84dafd-kube-api-access-kmn9l\") pod \"kuadrant-operator-controller-manager-86878c769f-zvqzj\" (UID: \"50ce2458-b52d-4a77-bdab-da22cc84dafd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zvqzj" Jul 22 03:27:51.726431 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:27:51.726299 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/50ce2458-b52d-4a77-bdab-da22cc84dafd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zvqzj\" (UID: \"50ce2458-b52d-4a77-bdab-da22cc84dafd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zvqzj" Jul 22 03:27:51.726624 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:27:51.726599 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/50ce2458-b52d-4a77-bdab-da22cc84dafd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zvqzj\" (UID: \"50ce2458-b52d-4a77-bdab-da22cc84dafd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zvqzj" Jul 22 03:27:51.742609 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:27:51.742573 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmn9l\" (UniqueName: \"kubernetes.io/projected/50ce2458-b52d-4a77-bdab-da22cc84dafd-kube-api-access-kmn9l\") pod \"kuadrant-operator-controller-manager-86878c769f-zvqzj\" (UID: \"50ce2458-b52d-4a77-bdab-da22cc84dafd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zvqzj" Jul 22 03:27:51.855592 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:27:51.855563 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zvqzj" Jul 22 03:27:52.025336 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:27:52.025314 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zvqzj"] Jul 22 03:27:52.027152 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:27:52.027124 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50ce2458_b52d_4a77_bdab_da22cc84dafd.slice/crio-bc25da954f2c6e29f90ed3bb5d1f90e35b99193b0ecf771d7e45ab2a5074d3e7 WatchSource:0}: Error finding container bc25da954f2c6e29f90ed3bb5d1f90e35b99193b0ecf771d7e45ab2a5074d3e7: Status 404 returned error can't find the container with id bc25da954f2c6e29f90ed3bb5d1f90e35b99193b0ecf771d7e45ab2a5074d3e7 Jul 22 03:27:52.139471 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:27:52.139392 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zvqzj" event={"ID":"50ce2458-b52d-4a77-bdab-da22cc84dafd","Type":"ContainerStarted","Data":"34dc9d7c7da42551c3d353afcfc438fa21aaafdc8487f780a7f066cf77de4c30"} Jul 22 03:27:52.139471 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:27:52.139449 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zvqzj" event={"ID":"50ce2458-b52d-4a77-bdab-da22cc84dafd","Type":"ContainerStarted","Data":"bc25da954f2c6e29f90ed3bb5d1f90e35b99193b0ecf771d7e45ab2a5074d3e7"} Jul 22 03:27:52.139638 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:27:52.139478 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zvqzj" Jul 22 03:27:52.188118 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:27:52.188052 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zvqzj" podStartSLOduration=1.18803255 podStartE2EDuration="1.18803255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:27:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:27:52.182767748 +0000 UTC m=+1212.631751017" watchObservedRunningTime="2026-07-22 03:27:52.18803255 +0000 UTC m=+1212.637015803" Jul 22 03:28:03.149777 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:28:03.149742 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zvqzj" Jul 22 03:28:15.342529 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:28:15.342495 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:28:26.550229 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:28:26.550190 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:28:37.436442 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:28:37.436357 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:28:46.642947 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:28:46.642915 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:28:57.831017 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:28:57.830978 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:29:01.423464 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:29:01.423429 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:29:06.239883 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:29:06.239848 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:29:15.936058 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:29:15.936020 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:29:27.835836 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:29:27.835799 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:29:35.429692 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:29:35.429654 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:29:45.932350 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:29:45.932319 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:29:54.410447 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:29:54.410395 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:29:54.511387 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:29:54.511348 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:30:21.330203 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:21.330169 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:30:30.737547 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:30.737508 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:30:47.448426 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:47.448392 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr"] Jul 22 03:30:47.451836 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:47.451815 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr" Jul 22 03:30:47.455457 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:47.455437 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-e04161e8-kserve-self-signed-certs\"" Jul 22 03:30:47.471782 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:47.471756 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr"] Jul 22 03:30:47.505081 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:47.505050 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:30:47.602040 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:47.602008 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93ef631d-85c6-45e5-8b03-f2205cb81995-home\") pod \"e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr\" (UID: \"93ef631d-85c6-45e5-8b03-f2205cb81995\") " pod="llm/e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr" Jul 22 03:30:47.602263 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:47.602082 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93ef631d-85c6-45e5-8b03-f2205cb81995-model-cache\") pod \"e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr\" (UID: \"93ef631d-85c6-45e5-8b03-f2205cb81995\") " pod="llm/e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr" Jul 22 03:30:47.602263 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:47.602154 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93ef631d-85c6-45e5-8b03-f2205cb81995-tls-certs\") pod \"e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr\" (UID: \"93ef631d-85c6-45e5-8b03-f2205cb81995\") " pod="llm/e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr" Jul 22 03:30:47.602263 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:47.602197 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93ef631d-85c6-45e5-8b03-f2205cb81995-dshm\") pod \"e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr\" (UID: \"93ef631d-85c6-45e5-8b03-f2205cb81995\") " pod="llm/e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr" Jul 22 03:30:47.602263 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:47.602223 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93ef631d-85c6-45e5-8b03-f2205cb81995-tmp-dir\") pod \"e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr\" (UID: \"93ef631d-85c6-45e5-8b03-f2205cb81995\") " pod="llm/e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr" Jul 22 03:30:47.602263 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:47.602250 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2f6t\" (UniqueName: \"kubernetes.io/projected/93ef631d-85c6-45e5-8b03-f2205cb81995-kube-api-access-p2f6t\") pod \"e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr\" (UID: \"93ef631d-85c6-45e5-8b03-f2205cb81995\") " pod="llm/e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr" Jul 22 03:30:47.703579 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:47.703484 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93ef631d-85c6-45e5-8b03-f2205cb81995-dshm\") pod \"e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr\" (UID: \"93ef631d-85c6-45e5-8b03-f2205cb81995\") " pod="llm/e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr" Jul 22 03:30:47.703579 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:47.703529 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93ef631d-85c6-45e5-8b03-f2205cb81995-tmp-dir\") pod \"e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr\" (UID: \"93ef631d-85c6-45e5-8b03-f2205cb81995\") " pod="llm/e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr" Jul 22 03:30:47.703579 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:47.703565 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2f6t\" (UniqueName: \"kubernetes.io/projected/93ef631d-85c6-45e5-8b03-f2205cb81995-kube-api-access-p2f6t\") pod \"e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr\" (UID: \"93ef631d-85c6-45e5-8b03-f2205cb81995\") " pod="llm/e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr" Jul 22 03:30:47.703849 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:47.703620 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93ef631d-85c6-45e5-8b03-f2205cb81995-home\") pod \"e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr\" (UID: \"93ef631d-85c6-45e5-8b03-f2205cb81995\") " pod="llm/e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr" Jul 22 03:30:47.703849 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:47.703701 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93ef631d-85c6-45e5-8b03-f2205cb81995-model-cache\") pod \"e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr\" (UID: \"93ef631d-85c6-45e5-8b03-f2205cb81995\") " pod="llm/e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr" Jul 22 03:30:47.703849 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:47.703788 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93ef631d-85c6-45e5-8b03-f2205cb81995-tls-certs\") pod \"e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr\" (UID: \"93ef631d-85c6-45e5-8b03-f2205cb81995\") " pod="llm/e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr" Jul 22 03:30:47.704039 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:47.704023 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93ef631d-85c6-45e5-8b03-f2205cb81995-tmp-dir\") pod \"e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr\" (UID: \"93ef631d-85c6-45e5-8b03-f2205cb81995\") " pod="llm/e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr" Jul 22 03:30:47.704158 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:47.704131 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93ef631d-85c6-45e5-8b03-f2205cb81995-model-cache\") pod \"e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr\" (UID: \"93ef631d-85c6-45e5-8b03-f2205cb81995\") " pod="llm/e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr" Jul 22 03:30:47.704236 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:47.704162 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93ef631d-85c6-45e5-8b03-f2205cb81995-home\") pod \"e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr\" (UID: \"93ef631d-85c6-45e5-8b03-f2205cb81995\") " pod="llm/e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr" Jul 22 03:30:47.706158 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:47.706127 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93ef631d-85c6-45e5-8b03-f2205cb81995-dshm\") pod \"e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr\" (UID: \"93ef631d-85c6-45e5-8b03-f2205cb81995\") " pod="llm/e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr" Jul 22 03:30:47.706312 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:47.706297 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93ef631d-85c6-45e5-8b03-f2205cb81995-tls-certs\") pod \"e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr\" (UID: \"93ef631d-85c6-45e5-8b03-f2205cb81995\") " pod="llm/e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr" Jul 22 03:30:47.713117 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:47.713080 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2f6t\" (UniqueName: \"kubernetes.io/projected/93ef631d-85c6-45e5-8b03-f2205cb81995-kube-api-access-p2f6t\") pod \"e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr\" (UID: \"93ef631d-85c6-45e5-8b03-f2205cb81995\") " pod="llm/e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr" Jul 22 03:30:47.761666 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:47.761630 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr" Jul 22 03:30:47.902136 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:47.902108 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr"] Jul 22 03:30:47.904018 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:30:47.903991 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod93ef631d_85c6_45e5_8b03_f2205cb81995.slice/crio-b9785b64c120e7906b53004335b230e234239c767da5f20aeb557c1a183495e9 WatchSource:0}: Error finding container b9785b64c120e7906b53004335b230e234239c767da5f20aeb557c1a183495e9: Status 404 returned error can't find the container with id b9785b64c120e7906b53004335b230e234239c767da5f20aeb557c1a183495e9 Jul 22 03:30:48.103109 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:48.103060 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr" event={"ID":"93ef631d-85c6-45e5-8b03-f2205cb81995","Type":"ContainerStarted","Data":"53fb9efaee28cf919f50cb73389fcce0f28ad9823129a33a6eeb9727a5bcad2f"} Jul 22 03:30:48.103109 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:48.103110 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr" event={"ID":"93ef631d-85c6-45e5-8b03-f2205cb81995","Type":"ContainerStarted","Data":"b9785b64c120e7906b53004335b230e234239c767da5f20aeb557c1a183495e9"} Jul 22 03:30:48.122308 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:48.122256 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr" podStartSLOduration=1.122240398 podStartE2EDuration="1.122240398s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:30:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:30:48.119541331 +0000 UTC m=+1388.568524580" watchObservedRunningTime="2026-07-22 03:30:48.122240398 +0000 UTC m=+1388.571223647" Jul 22 03:30:50.338447 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:50.338405 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:30:52.231563 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:52.231532 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk"] Jul 22 03:30:52.235324 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:52.235300 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk" Jul 22 03:30:52.238003 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:52.237982 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-e04161e8-kserve-self-signed-certs\"" Jul 22 03:30:52.246703 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:52.246678 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3d0b4e4-f077-4b1f-aaa7-04f4737eb742-tmp-dir\") pod \"e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk\" (UID: \"a3d0b4e4-f077-4b1f-aaa7-04f4737eb742\") " pod="llm/e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk" Jul 22 03:30:52.246825 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:52.246725 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzj4p\" (UniqueName: \"kubernetes.io/projected/a3d0b4e4-f077-4b1f-aaa7-04f4737eb742-kube-api-access-vzj4p\") pod \"e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk\" (UID: \"a3d0b4e4-f077-4b1f-aaa7-04f4737eb742\") " pod="llm/e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk" Jul 22 03:30:52.246825 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:52.246749 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3d0b4e4-f077-4b1f-aaa7-04f4737eb742-model-cache\") pod \"e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk\" (UID: \"a3d0b4e4-f077-4b1f-aaa7-04f4737eb742\") " pod="llm/e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk" Jul 22 03:30:52.246908 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:52.246874 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3d0b4e4-f077-4b1f-aaa7-04f4737eb742-tls-certs\") pod \"e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk\" (UID: \"a3d0b4e4-f077-4b1f-aaa7-04f4737eb742\") " pod="llm/e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk" Jul 22 03:30:52.246996 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:52.246978 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3d0b4e4-f077-4b1f-aaa7-04f4737eb742-home\") pod \"e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk\" (UID: \"a3d0b4e4-f077-4b1f-aaa7-04f4737eb742\") " pod="llm/e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk" Jul 22 03:30:52.247042 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:52.247013 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3d0b4e4-f077-4b1f-aaa7-04f4737eb742-dshm\") pod \"e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk\" (UID: \"a3d0b4e4-f077-4b1f-aaa7-04f4737eb742\") " pod="llm/e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk" Jul 22 03:30:52.247994 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:52.247975 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk"] Jul 22 03:30:52.348092 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:52.348056 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3d0b4e4-f077-4b1f-aaa7-04f4737eb742-home\") pod \"e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk\" (UID: \"a3d0b4e4-f077-4b1f-aaa7-04f4737eb742\") " pod="llm/e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk" Jul 22 03:30:52.348092 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:52.348097 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3d0b4e4-f077-4b1f-aaa7-04f4737eb742-dshm\") pod \"e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk\" (UID: \"a3d0b4e4-f077-4b1f-aaa7-04f4737eb742\") " pod="llm/e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk" Jul 22 03:30:52.348365 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:52.348131 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3d0b4e4-f077-4b1f-aaa7-04f4737eb742-tmp-dir\") pod \"e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk\" (UID: \"a3d0b4e4-f077-4b1f-aaa7-04f4737eb742\") " pod="llm/e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk" Jul 22 03:30:52.348365 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:52.348176 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzj4p\" (UniqueName: \"kubernetes.io/projected/a3d0b4e4-f077-4b1f-aaa7-04f4737eb742-kube-api-access-vzj4p\") pod \"e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk\" (UID: \"a3d0b4e4-f077-4b1f-aaa7-04f4737eb742\") " pod="llm/e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk" Jul 22 03:30:52.348365 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:52.348202 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3d0b4e4-f077-4b1f-aaa7-04f4737eb742-model-cache\") pod \"e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk\" (UID: \"a3d0b4e4-f077-4b1f-aaa7-04f4737eb742\") " pod="llm/e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk" Jul 22 03:30:52.348365 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:52.348269 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3d0b4e4-f077-4b1f-aaa7-04f4737eb742-tls-certs\") pod \"e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk\" (UID: \"a3d0b4e4-f077-4b1f-aaa7-04f4737eb742\") " pod="llm/e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk" Jul 22 03:30:52.348588 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:52.348552 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3d0b4e4-f077-4b1f-aaa7-04f4737eb742-home\") pod \"e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk\" (UID: \"a3d0b4e4-f077-4b1f-aaa7-04f4737eb742\") " pod="llm/e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk" Jul 22 03:30:52.348634 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:52.348620 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3d0b4e4-f077-4b1f-aaa7-04f4737eb742-tmp-dir\") pod \"e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk\" (UID: \"a3d0b4e4-f077-4b1f-aaa7-04f4737eb742\") " pod="llm/e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk" Jul 22 03:30:52.348678 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:52.348653 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3d0b4e4-f077-4b1f-aaa7-04f4737eb742-model-cache\") pod \"e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk\" (UID: \"a3d0b4e4-f077-4b1f-aaa7-04f4737eb742\") " pod="llm/e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk" Jul 22 03:30:52.350592 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:52.350558 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3d0b4e4-f077-4b1f-aaa7-04f4737eb742-dshm\") pod \"e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk\" (UID: \"a3d0b4e4-f077-4b1f-aaa7-04f4737eb742\") " pod="llm/e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk" Jul 22 03:30:52.350817 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:52.350799 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3d0b4e4-f077-4b1f-aaa7-04f4737eb742-tls-certs\") pod \"e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk\" (UID: \"a3d0b4e4-f077-4b1f-aaa7-04f4737eb742\") " pod="llm/e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk" Jul 22 03:30:52.355307 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:52.355280 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:30:52.358355 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:52.358332 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzj4p\" (UniqueName: \"kubernetes.io/projected/a3d0b4e4-f077-4b1f-aaa7-04f4737eb742-kube-api-access-vzj4p\") pod \"e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk\" (UID: \"a3d0b4e4-f077-4b1f-aaa7-04f4737eb742\") " pod="llm/e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk" Jul 22 03:30:52.544638 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:52.544550 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk" Jul 22 03:30:52.706721 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:52.706684 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk"] Jul 22 03:30:52.722598 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:30:52.722574 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda3d0b4e4_f077_4b1f_aaa7_04f4737eb742.slice/crio-c1cb786adaa3c6970cd080cb1424d931af819e1a53299cf240a181b61e0014cb WatchSource:0}: Error finding container c1cb786adaa3c6970cd080cb1424d931af819e1a53299cf240a181b61e0014cb: Status 404 returned error can't find the container with id c1cb786adaa3c6970cd080cb1424d931af819e1a53299cf240a181b61e0014cb Jul 22 03:30:53.130793 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:53.130753 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk" event={"ID":"a3d0b4e4-f077-4b1f-aaa7-04f4737eb742","Type":"ContainerStarted","Data":"1ad92a20ba02055c6ccb09bfdb3a1f595a937747a755cd25307864f5a8503a5d"} Jul 22 03:30:53.130793 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:53.130801 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk" event={"ID":"a3d0b4e4-f077-4b1f-aaa7-04f4737eb742","Type":"ContainerStarted","Data":"c1cb786adaa3c6970cd080cb1424d931af819e1a53299cf240a181b61e0014cb"} Jul 22 03:30:53.149430 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:53.149380 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk" podStartSLOduration=1.14936585 podStartE2EDuration="1.14936585s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:30:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:30:53.147352851 +0000 UTC m=+1393.596336114" watchObservedRunningTime="2026-07-22 03:30:53.14936585 +0000 UTC m=+1393.598349139" Jul 22 03:30:57.762208 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:57.762167 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr" Jul 22 03:30:57.762691 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:57.762232 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr" Jul 22 03:30:57.775364 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:57.775335 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr" Jul 22 03:30:58.169298 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:30:58.169269 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr" Jul 22 03:31:02.545600 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:02.545556 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk" Jul 22 03:31:02.546029 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:02.545617 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk" Jul 22 03:31:02.558131 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:02.558108 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk" Jul 22 03:31:03.204404 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:03.204374 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk" Jul 22 03:31:06.034156 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:06.034118 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:31:30.900459 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:30.900371 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr"] Jul 22 03:31:30.922716 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:30.922658 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:31:32.526193 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:32.526150 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk"] Jul 22 03:31:32.551678 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:32.551634 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:31:35.034784 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:35.034750 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:31:45.922032 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:45.921984 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr" podUID="93ef631d-85c6-45e5-8b03-f2205cb81995" containerName="main" containerID="cri-o://53fb9efaee28cf919f50cb73389fcce0f28ad9823129a33a6eeb9727a5bcad2f" gracePeriod=45 Jul 22 03:31:46.174019 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:46.173932 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr" Jul 22 03:31:46.228709 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:46.228677 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93ef631d-85c6-45e5-8b03-f2205cb81995-model-cache\") pod \"93ef631d-85c6-45e5-8b03-f2205cb81995\" (UID: \"93ef631d-85c6-45e5-8b03-f2205cb81995\") " Jul 22 03:31:46.228709 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:46.228711 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2f6t\" (UniqueName: \"kubernetes.io/projected/93ef631d-85c6-45e5-8b03-f2205cb81995-kube-api-access-p2f6t\") pod \"93ef631d-85c6-45e5-8b03-f2205cb81995\" (UID: \"93ef631d-85c6-45e5-8b03-f2205cb81995\") " Jul 22 03:31:46.228988 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:46.228752 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93ef631d-85c6-45e5-8b03-f2205cb81995-home\") pod \"93ef631d-85c6-45e5-8b03-f2205cb81995\" (UID: \"93ef631d-85c6-45e5-8b03-f2205cb81995\") " Jul 22 03:31:46.228988 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:46.228806 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93ef631d-85c6-45e5-8b03-f2205cb81995-tls-certs\") pod \"93ef631d-85c6-45e5-8b03-f2205cb81995\" (UID: \"93ef631d-85c6-45e5-8b03-f2205cb81995\") " Jul 22 03:31:46.228988 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:46.228840 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93ef631d-85c6-45e5-8b03-f2205cb81995-dshm\") pod \"93ef631d-85c6-45e5-8b03-f2205cb81995\" (UID: \"93ef631d-85c6-45e5-8b03-f2205cb81995\") " Jul 22 03:31:46.228988 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:46.228862 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93ef631d-85c6-45e5-8b03-f2205cb81995-tmp-dir\") pod \"93ef631d-85c6-45e5-8b03-f2205cb81995\" (UID: \"93ef631d-85c6-45e5-8b03-f2205cb81995\") " Jul 22 03:31:46.228988 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:46.228918 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93ef631d-85c6-45e5-8b03-f2205cb81995-model-cache" (OuterVolumeSpecName: "model-cache") pod "93ef631d-85c6-45e5-8b03-f2205cb81995" (UID: "93ef631d-85c6-45e5-8b03-f2205cb81995"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:31:46.229230 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:46.229066 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93ef631d-85c6-45e5-8b03-f2205cb81995-home" (OuterVolumeSpecName: "home") pod "93ef631d-85c6-45e5-8b03-f2205cb81995" (UID: "93ef631d-85c6-45e5-8b03-f2205cb81995"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:31:46.229230 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:46.229220 2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93ef631d-85c6-45e5-8b03-f2205cb81995-model-cache\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:31:46.229308 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:46.229232 2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93ef631d-85c6-45e5-8b03-f2205cb81995-home\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:31:46.229391 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:46.229364 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93ef631d-85c6-45e5-8b03-f2205cb81995-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "93ef631d-85c6-45e5-8b03-f2205cb81995" (UID: "93ef631d-85c6-45e5-8b03-f2205cb81995"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:31:46.231042 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:46.231009 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93ef631d-85c6-45e5-8b03-f2205cb81995-dshm" (OuterVolumeSpecName: "dshm") pod "93ef631d-85c6-45e5-8b03-f2205cb81995" (UID: "93ef631d-85c6-45e5-8b03-f2205cb81995"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:31:46.231042 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:46.231026 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93ef631d-85c6-45e5-8b03-f2205cb81995-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "93ef631d-85c6-45e5-8b03-f2205cb81995" (UID: "93ef631d-85c6-45e5-8b03-f2205cb81995"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:31:46.231192 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:46.231055 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93ef631d-85c6-45e5-8b03-f2205cb81995-kube-api-access-p2f6t" (OuterVolumeSpecName: "kube-api-access-p2f6t") pod "93ef631d-85c6-45e5-8b03-f2205cb81995" (UID: "93ef631d-85c6-45e5-8b03-f2205cb81995"). InnerVolumeSpecName "kube-api-access-p2f6t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:31:46.330570 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:46.330535 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2f6t\" (UniqueName: \"kubernetes.io/projected/93ef631d-85c6-45e5-8b03-f2205cb81995-kube-api-access-p2f6t\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:31:46.330570 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:46.330567 2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93ef631d-85c6-45e5-8b03-f2205cb81995-tls-certs\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:31:46.330570 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:46.330577 2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93ef631d-85c6-45e5-8b03-f2205cb81995-dshm\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:31:46.330825 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:46.330586 2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93ef631d-85c6-45e5-8b03-f2205cb81995-tmp-dir\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:31:46.423488 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:46.423455 2556 generic.go:356] "Generic (PLEG): container finished" podID="93ef631d-85c6-45e5-8b03-f2205cb81995" containerID="53fb9efaee28cf919f50cb73389fcce0f28ad9823129a33a6eeb9727a5bcad2f" exitCode=0 Jul 22 03:31:46.423686 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:46.423504 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr" Jul 22 03:31:46.423686 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:46.423513 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr" event={"ID":"93ef631d-85c6-45e5-8b03-f2205cb81995","Type":"ContainerDied","Data":"53fb9efaee28cf919f50cb73389fcce0f28ad9823129a33a6eeb9727a5bcad2f"} Jul 22 03:31:46.423686 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:46.423559 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr" event={"ID":"93ef631d-85c6-45e5-8b03-f2205cb81995","Type":"ContainerDied","Data":"b9785b64c120e7906b53004335b230e234239c767da5f20aeb557c1a183495e9"} Jul 22 03:31:46.423686 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:46.423577 2556 scope.go:117] "RemoveContainer" containerID="53fb9efaee28cf919f50cb73389fcce0f28ad9823129a33a6eeb9727a5bcad2f" Jul 22 03:31:46.433202 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:46.433185 2556 scope.go:117] "RemoveContainer" containerID="53fb9efaee28cf919f50cb73389fcce0f28ad9823129a33a6eeb9727a5bcad2f" Jul 22 03:31:46.433468 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:31:46.433451 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53fb9efaee28cf919f50cb73389fcce0f28ad9823129a33a6eeb9727a5bcad2f\": container with ID starting with 53fb9efaee28cf919f50cb73389fcce0f28ad9823129a33a6eeb9727a5bcad2f not found: ID does not exist" containerID="53fb9efaee28cf919f50cb73389fcce0f28ad9823129a33a6eeb9727a5bcad2f" Jul 22 03:31:46.433515 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:46.433478 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53fb9efaee28cf919f50cb73389fcce0f28ad9823129a33a6eeb9727a5bcad2f"} err="failed to get container status \"53fb9efaee28cf919f50cb73389fcce0f28ad9823129a33a6eeb9727a5bcad2f\": rpc error: code = NotFound desc = could not find container \"53fb9efaee28cf919f50cb73389fcce0f28ad9823129a33a6eeb9727a5bcad2f\": container with ID starting with 53fb9efaee28cf919f50cb73389fcce0f28ad9823129a33a6eeb9727a5bcad2f not found: ID does not exist" Jul 22 03:31:46.446471 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:46.446446 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr"] Jul 22 03:31:46.450041 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:46.450017 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-e04161e8-kserve-7fdd4458d7-b5mdr"] Jul 22 03:31:47.548127 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:47.548068 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk" podUID="a3d0b4e4-f077-4b1f-aaa7-04f4737eb742" containerName="main" containerID="cri-o://1ad92a20ba02055c6ccb09bfdb3a1f595a937747a755cd25307864f5a8503a5d" gracePeriod=45 Jul 22 03:31:47.806347 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:47.806283 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk" Jul 22 03:31:47.845746 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:47.845710 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3d0b4e4-f077-4b1f-aaa7-04f4737eb742-dshm\") pod \"a3d0b4e4-f077-4b1f-aaa7-04f4737eb742\" (UID: \"a3d0b4e4-f077-4b1f-aaa7-04f4737eb742\") " Jul 22 03:31:47.845918 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:47.845781 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3d0b4e4-f077-4b1f-aaa7-04f4737eb742-model-cache\") pod \"a3d0b4e4-f077-4b1f-aaa7-04f4737eb742\" (UID: \"a3d0b4e4-f077-4b1f-aaa7-04f4737eb742\") " Jul 22 03:31:47.845918 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:47.845812 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3d0b4e4-f077-4b1f-aaa7-04f4737eb742-home\") pod \"a3d0b4e4-f077-4b1f-aaa7-04f4737eb742\" (UID: \"a3d0b4e4-f077-4b1f-aaa7-04f4737eb742\") " Jul 22 03:31:47.845918 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:47.845901 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3d0b4e4-f077-4b1f-aaa7-04f4737eb742-tmp-dir\") pod \"a3d0b4e4-f077-4b1f-aaa7-04f4737eb742\" (UID: \"a3d0b4e4-f077-4b1f-aaa7-04f4737eb742\") " Jul 22 03:31:47.846138 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:47.845993 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vzj4p\" (UniqueName: \"kubernetes.io/projected/a3d0b4e4-f077-4b1f-aaa7-04f4737eb742-kube-api-access-vzj4p\") pod \"a3d0b4e4-f077-4b1f-aaa7-04f4737eb742\" (UID: \"a3d0b4e4-f077-4b1f-aaa7-04f4737eb742\") " Jul 22 03:31:47.846138 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:47.846025 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3d0b4e4-f077-4b1f-aaa7-04f4737eb742-tls-certs\") pod \"a3d0b4e4-f077-4b1f-aaa7-04f4737eb742\" (UID: \"a3d0b4e4-f077-4b1f-aaa7-04f4737eb742\") " Jul 22 03:31:47.846138 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:47.846048 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3d0b4e4-f077-4b1f-aaa7-04f4737eb742-model-cache" (OuterVolumeSpecName: "model-cache") pod "a3d0b4e4-f077-4b1f-aaa7-04f4737eb742" (UID: "a3d0b4e4-f077-4b1f-aaa7-04f4737eb742"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:31:47.846292 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:47.846164 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3d0b4e4-f077-4b1f-aaa7-04f4737eb742-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a3d0b4e4-f077-4b1f-aaa7-04f4737eb742" (UID: "a3d0b4e4-f077-4b1f-aaa7-04f4737eb742"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:31:47.846292 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:47.846182 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3d0b4e4-f077-4b1f-aaa7-04f4737eb742-home" (OuterVolumeSpecName: "home") pod "a3d0b4e4-f077-4b1f-aaa7-04f4737eb742" (UID: "a3d0b4e4-f077-4b1f-aaa7-04f4737eb742"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:31:47.846393 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:47.846349 2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3d0b4e4-f077-4b1f-aaa7-04f4737eb742-tmp-dir\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:31:47.846393 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:47.846368 2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3d0b4e4-f077-4b1f-aaa7-04f4737eb742-model-cache\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:31:47.846393 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:47.846381 2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3d0b4e4-f077-4b1f-aaa7-04f4737eb742-home\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:31:47.848053 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:47.848027 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3d0b4e4-f077-4b1f-aaa7-04f4737eb742-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a3d0b4e4-f077-4b1f-aaa7-04f4737eb742" (UID: "a3d0b4e4-f077-4b1f-aaa7-04f4737eb742"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:31:47.848139 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:47.848117 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3d0b4e4-f077-4b1f-aaa7-04f4737eb742-kube-api-access-vzj4p" (OuterVolumeSpecName: "kube-api-access-vzj4p") pod "a3d0b4e4-f077-4b1f-aaa7-04f4737eb742" (UID: "a3d0b4e4-f077-4b1f-aaa7-04f4737eb742"). InnerVolumeSpecName "kube-api-access-vzj4p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:31:47.848321 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:47.848303 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3d0b4e4-f077-4b1f-aaa7-04f4737eb742-dshm" (OuterVolumeSpecName: "dshm") pod "a3d0b4e4-f077-4b1f-aaa7-04f4737eb742" (UID: "a3d0b4e4-f077-4b1f-aaa7-04f4737eb742"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:31:47.946849 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:47.946804 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzj4p\" (UniqueName: \"kubernetes.io/projected/a3d0b4e4-f077-4b1f-aaa7-04f4737eb742-kube-api-access-vzj4p\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:31:47.946849 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:47.946840 2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3d0b4e4-f077-4b1f-aaa7-04f4737eb742-tls-certs\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:31:47.946849 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:47.946854 2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3d0b4e4-f077-4b1f-aaa7-04f4737eb742-dshm\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:31:48.025100 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:48.025062 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93ef631d-85c6-45e5-8b03-f2205cb81995" path="/var/lib/kubelet/pods/93ef631d-85c6-45e5-8b03-f2205cb81995/volumes" Jul 22 03:31:48.437522 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:48.437488 2556 generic.go:356] "Generic (PLEG): container finished" podID="a3d0b4e4-f077-4b1f-aaa7-04f4737eb742" containerID="1ad92a20ba02055c6ccb09bfdb3a1f595a937747a755cd25307864f5a8503a5d" exitCode=0 Jul 22 03:31:48.437742 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:48.437540 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk" Jul 22 03:31:48.437742 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:48.437571 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk" event={"ID":"a3d0b4e4-f077-4b1f-aaa7-04f4737eb742","Type":"ContainerDied","Data":"1ad92a20ba02055c6ccb09bfdb3a1f595a937747a755cd25307864f5a8503a5d"} Jul 22 03:31:48.437742 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:48.437611 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk" event={"ID":"a3d0b4e4-f077-4b1f-aaa7-04f4737eb742","Type":"ContainerDied","Data":"c1cb786adaa3c6970cd080cb1424d931af819e1a53299cf240a181b61e0014cb"} Jul 22 03:31:48.437742 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:48.437629 2556 scope.go:117] "RemoveContainer" containerID="1ad92a20ba02055c6ccb09bfdb3a1f595a937747a755cd25307864f5a8503a5d" Jul 22 03:31:48.446497 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:48.446478 2556 scope.go:117] "RemoveContainer" containerID="1ad92a20ba02055c6ccb09bfdb3a1f595a937747a755cd25307864f5a8503a5d" Jul 22 03:31:48.446761 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:31:48.446744 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ad92a20ba02055c6ccb09bfdb3a1f595a937747a755cd25307864f5a8503a5d\": container with ID starting with 1ad92a20ba02055c6ccb09bfdb3a1f595a937747a755cd25307864f5a8503a5d not found: ID does not exist" containerID="1ad92a20ba02055c6ccb09bfdb3a1f595a937747a755cd25307864f5a8503a5d" Jul 22 03:31:48.446797 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:48.446772 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ad92a20ba02055c6ccb09bfdb3a1f595a937747a755cd25307864f5a8503a5d"} err="failed to get container status \"1ad92a20ba02055c6ccb09bfdb3a1f595a937747a755cd25307864f5a8503a5d\": rpc error: code = NotFound desc = could not find container \"1ad92a20ba02055c6ccb09bfdb3a1f595a937747a755cd25307864f5a8503a5d\": container with ID starting with 1ad92a20ba02055c6ccb09bfdb3a1f595a937747a755cd25307864f5a8503a5d not found: ID does not exist" Jul 22 03:31:48.454524 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:48.454502 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk"] Jul 22 03:31:48.462070 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:48.462040 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-e04161e8-kserve-5568dc8967-v9lfk"] Jul 22 03:31:49.132233 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:49.132195 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:31:50.024621 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:50.024588 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3d0b4e4-f077-4b1f-aaa7-04f4737eb742" path="/var/lib/kubelet/pods/a3d0b4e4-f077-4b1f-aaa7-04f4737eb742/volumes" Jul 22 03:31:56.732747 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:31:56.732714 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:32:12.936739 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:32:12.936699 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:32:22.435483 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:32:22.435442 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:32:31.226357 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:32:31.226314 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:32:40.162050 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:32:40.161914 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscf8_a01dfba4-f590-412a-8438-1646c51bb1a1/ovn-acl-logging/0.log" Jul 22 03:32:40.165340 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:32:40.164195 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscf8_a01dfba4-f590-412a-8438-1646c51bb1a1/ovn-acl-logging/0.log" Jul 22 03:32:43.137606 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:32:43.137572 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:32:52.139925 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:32:52.139892 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:33:01.931455 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:33:01.931375 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:33:23.233887 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:33:23.233849 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:33:32.444294 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:33:32.444256 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:33:38.000732 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:33:38.000700 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:33:44.675139 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:33:44.675093 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:33:44.939805 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:33:44.939725 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:33:54.857576 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:33:54.857544 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:34:01.536726 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:34:01.536671 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:34:17.446469 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:34:17.446435 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:34:28.947462 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:34:28.947428 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:34:40.435792 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:34:40.435714 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:34:43.275305 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:34:43.275263 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:34:44.532209 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:34:44.532176 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:34:50.705076 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:34:50.705042 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:34:50.805428 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:34:50.805390 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:34:55.843705 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:34:55.843665 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:34:58.950530 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:34:58.950500 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:34:59.022292 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:34:59.022263 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:35:05.263231 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:05.263192 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:35:05.360515 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:05.360484 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:35:08.148496 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:08.148460 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:35:08.250508 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:08.250470 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:35:08.616914 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:08.616878 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:35:08.951685 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:08.951601 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:35:14.475575 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:14.475534 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:35:14.577510 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:14.577460 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:35:14.715574 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:14.715535 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:35:14.995828 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:14.995792 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4"] Jul 22 03:35:14.996231 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:14.996214 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93ef631d-85c6-45e5-8b03-f2205cb81995" containerName="main" Jul 22 03:35:14.996231 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:14.996230 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="93ef631d-85c6-45e5-8b03-f2205cb81995" containerName="main" Jul 22 03:35:14.996392 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:14.996239 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3d0b4e4-f077-4b1f-aaa7-04f4737eb742" containerName="main" Jul 22 03:35:14.996392 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:14.996245 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3d0b4e4-f077-4b1f-aaa7-04f4737eb742" containerName="main" Jul 22 03:35:14.996392 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:14.996316 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="93ef631d-85c6-45e5-8b03-f2205cb81995" containerName="main" Jul 22 03:35:14.996392 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:14.996327 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3d0b4e4-f077-4b1f-aaa7-04f4737eb742" containerName="main" Jul 22 03:35:14.999395 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:14.999374 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:35:15.002639 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.002617 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-6abd7a28-openshift-default-dockercfg-4qcx4\"" Jul 22 03:35:15.024151 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.024127 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4"] Jul 22 03:35:15.025927 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.025894 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:35:15.125992 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.125966 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76c633c5-d34a-430f-a9c1-69a965a457a6-workload-certs\") pod \"e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4\" (UID: \"76c633c5-d34a-430f-a9c1-69a965a457a6\") " pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:35:15.126179 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.126001 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76c633c5-d34a-430f-a9c1-69a965a457a6-istio-data\") pod \"e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4\" (UID: \"76c633c5-d34a-430f-a9c1-69a965a457a6\") " pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:35:15.126179 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.126024 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76c633c5-d34a-430f-a9c1-69a965a457a6-istiod-ca-cert\") pod \"e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4\" (UID: \"76c633c5-d34a-430f-a9c1-69a965a457a6\") " pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:35:15.126179 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.126044 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgq75\" (UniqueName: \"kubernetes.io/projected/76c633c5-d34a-430f-a9c1-69a965a457a6-kube-api-access-fgq75\") pod \"e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4\" (UID: \"76c633c5-d34a-430f-a9c1-69a965a457a6\") " pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:35:15.126179 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.126074 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76c633c5-d34a-430f-a9c1-69a965a457a6-istio-podinfo\") pod \"e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4\" (UID: \"76c633c5-d34a-430f-a9c1-69a965a457a6\") " pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:35:15.126429 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.126178 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76c633c5-d34a-430f-a9c1-69a965a457a6-workload-socket\") pod \"e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4\" (UID: \"76c633c5-d34a-430f-a9c1-69a965a457a6\") " pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:35:15.126429 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.126211 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76c633c5-d34a-430f-a9c1-69a965a457a6-credential-socket\") pod \"e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4\" (UID: \"76c633c5-d34a-430f-a9c1-69a965a457a6\") " pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:35:15.126429 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.126284 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76c633c5-d34a-430f-a9c1-69a965a457a6-istio-token\") pod \"e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4\" (UID: \"76c633c5-d34a-430f-a9c1-69a965a457a6\") " pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:35:15.126429 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.126321 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76c633c5-d34a-430f-a9c1-69a965a457a6-istio-envoy\") pod \"e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4\" (UID: \"76c633c5-d34a-430f-a9c1-69a965a457a6\") " pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:35:15.227391 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.227352 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76c633c5-d34a-430f-a9c1-69a965a457a6-workload-socket\") pod \"e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4\" (UID: \"76c633c5-d34a-430f-a9c1-69a965a457a6\") " pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:35:15.227391 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.227392 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76c633c5-d34a-430f-a9c1-69a965a457a6-credential-socket\") pod \"e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4\" (UID: \"76c633c5-d34a-430f-a9c1-69a965a457a6\") " pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:35:15.227635 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.227420 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76c633c5-d34a-430f-a9c1-69a965a457a6-istio-token\") pod \"e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4\" (UID: \"76c633c5-d34a-430f-a9c1-69a965a457a6\") " pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:35:15.227635 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.227448 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76c633c5-d34a-430f-a9c1-69a965a457a6-istio-envoy\") pod \"e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4\" (UID: \"76c633c5-d34a-430f-a9c1-69a965a457a6\") " pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:35:15.227635 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.227519 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76c633c5-d34a-430f-a9c1-69a965a457a6-workload-certs\") pod \"e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4\" (UID: \"76c633c5-d34a-430f-a9c1-69a965a457a6\") " pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:35:15.227635 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.227540 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76c633c5-d34a-430f-a9c1-69a965a457a6-istio-data\") pod \"e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4\" (UID: \"76c633c5-d34a-430f-a9c1-69a965a457a6\") " pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:35:15.227635 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.227568 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76c633c5-d34a-430f-a9c1-69a965a457a6-istiod-ca-cert\") pod \"e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4\" (UID: \"76c633c5-d34a-430f-a9c1-69a965a457a6\") " pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:35:15.227635 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.227593 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgq75\" (UniqueName: \"kubernetes.io/projected/76c633c5-d34a-430f-a9c1-69a965a457a6-kube-api-access-fgq75\") pod \"e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4\" (UID: \"76c633c5-d34a-430f-a9c1-69a965a457a6\") " pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:35:15.227996 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.227637 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76c633c5-d34a-430f-a9c1-69a965a457a6-istio-podinfo\") pod \"e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4\" (UID: \"76c633c5-d34a-430f-a9c1-69a965a457a6\") " pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:35:15.227996 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.227818 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76c633c5-d34a-430f-a9c1-69a965a457a6-workload-socket\") pod \"e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4\" (UID: \"76c633c5-d34a-430f-a9c1-69a965a457a6\") " pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:35:15.228320 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.228277 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76c633c5-d34a-430f-a9c1-69a965a457a6-workload-certs\") pod \"e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4\" (UID: \"76c633c5-d34a-430f-a9c1-69a965a457a6\") " pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:35:15.228435 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.228338 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76c633c5-d34a-430f-a9c1-69a965a457a6-credential-socket\") pod \"e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4\" (UID: \"76c633c5-d34a-430f-a9c1-69a965a457a6\") " pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:35:15.228435 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.228339 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76c633c5-d34a-430f-a9c1-69a965a457a6-istiod-ca-cert\") pod \"e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4\" (UID: \"76c633c5-d34a-430f-a9c1-69a965a457a6\") " pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:35:15.228684 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.228661 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76c633c5-d34a-430f-a9c1-69a965a457a6-istio-data\") pod \"e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4\" (UID: \"76c633c5-d34a-430f-a9c1-69a965a457a6\") " pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:35:15.229889 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.229867 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76c633c5-d34a-430f-a9c1-69a965a457a6-istio-envoy\") pod \"e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4\" (UID: \"76c633c5-d34a-430f-a9c1-69a965a457a6\") " pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:35:15.230222 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.230202 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76c633c5-d34a-430f-a9c1-69a965a457a6-istio-podinfo\") pod \"e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4\" (UID: \"76c633c5-d34a-430f-a9c1-69a965a457a6\") " pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:35:15.236937 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.236915 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76c633c5-d34a-430f-a9c1-69a965a457a6-istio-token\") pod \"e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4\" (UID: \"76c633c5-d34a-430f-a9c1-69a965a457a6\") " pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:35:15.237174 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.237152 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgq75\" (UniqueName: \"kubernetes.io/projected/76c633c5-d34a-430f-a9c1-69a965a457a6-kube-api-access-fgq75\") pod \"e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4\" (UID: \"76c633c5-d34a-430f-a9c1-69a965a457a6\") " pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:35:15.309682 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.309568 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:35:15.455404 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.455381 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4"] Jul 22 03:35:15.457195 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:35:15.457166 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76c633c5_d34a_430f_a9c1_69a965a457a6.slice/crio-d767964be5c4612eba8d188478e0d7f8610b3e78596381ebb3a9fbb24e96d6d5 WatchSource:0}: Error finding container d767964be5c4612eba8d188478e0d7f8610b3e78596381ebb3a9fbb24e96d6d5: Status 404 returned error can't find the container with id d767964be5c4612eba8d188478e0d7f8610b3e78596381ebb3a9fbb24e96d6d5 Jul 22 03:35:15.459285 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.459243 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:35:15.459370 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.459324 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:35:15.459370 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.459353 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:35:15.478837 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.478814 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:35:15.545114 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.545073 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" event={"ID":"76c633c5-d34a-430f-a9c1-69a965a457a6","Type":"ContainerStarted","Data":"eed6fe548957c5683e77b39641a3c66d1da4f493fb4b2efdc5038daedc061df5"} Jul 22 03:35:15.545289 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.545133 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" event={"ID":"76c633c5-d34a-430f-a9c1-69a965a457a6","Type":"ContainerStarted","Data":"d767964be5c4612eba8d188478e0d7f8610b3e78596381ebb3a9fbb24e96d6d5"} Jul 22 03:35:15.574280 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:15.574224 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" podStartSLOduration=1.574202235 podStartE2EDuration="1.574202235s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:35:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:35:15.571486019 +0000 UTC m=+1656.020469265" watchObservedRunningTime="2026-07-22 03:35:15.574202235 +0000 UTC m=+1656.023185485" Jul 22 03:35:16.310572 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:16.310537 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:35:16.315611 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:16.315588 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:35:16.551015 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:16.550983 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:35:16.551927 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:16.551909 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:35:16.911094 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:16.911054 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:35:17.010024 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:17.009988 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:35:17.129018 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:17.128982 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:35:17.541523 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:17.541492 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:35:17.829140 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:17.829108 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:35:17.926478 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:17.926446 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:35:19.839780 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:19.839744 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:35:28.638263 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:28.638229 2556 generic.go:356] "Generic (PLEG): container finished" podID="76c633c5-d34a-430f-a9c1-69a965a457a6" containerID="eed6fe548957c5683e77b39641a3c66d1da4f493fb4b2efdc5038daedc061df5" exitCode=0 Jul 22 03:35:28.638707 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:28.638305 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" event={"ID":"76c633c5-d34a-430f-a9c1-69a965a457a6","Type":"ContainerDied","Data":"eed6fe548957c5683e77b39641a3c66d1da4f493fb4b2efdc5038daedc061df5"} Jul 22 03:35:28.638707 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:28.638705 2556 scope.go:117] "RemoveContainer" containerID="eed6fe548957c5683e77b39641a3c66d1da4f493fb4b2efdc5038daedc061df5" Jul 22 03:35:28.639736 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:28.639708 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:35:28.639796 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:28.639782 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:35:28.639832 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:28.639818 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:35:29.646141 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:29.646105 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" event={"ID":"76c633c5-d34a-430f-a9c1-69a965a457a6","Type":"ContainerStarted","Data":"2976f2123b6d257483d1b92ca1dcf7a84f489dc0afd611684a2fabea1d5f2d6e"} Jul 22 03:35:30.310693 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:30.310658 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:35:30.311878 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:30.311854 2556 patch_prober.go:28] interesting pod/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 22 03:35:30.312002 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:30.311910 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" podUID="76c633c5-d34a-430f-a9c1-69a965a457a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 22 03:35:31.310110 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:31.310066 2556 patch_prober.go:28] interesting pod/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 22 03:35:31.310504 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:31.310144 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" podUID="76c633c5-d34a-430f-a9c1-69a965a457a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 22 03:35:31.551649 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:31.551611 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:35:32.310210 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:32.310173 2556 patch_prober.go:28] interesting pod/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 22 03:35:32.310597 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:32.310243 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" podUID="76c633c5-d34a-430f-a9c1-69a965a457a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 22 03:35:33.310880 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:33.310840 2556 patch_prober.go:28] interesting pod/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 22 03:35:33.311305 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:33.310900 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" podUID="76c633c5-d34a-430f-a9c1-69a965a457a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 22 03:35:34.313992 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:34.313947 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:35:34.314927 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:34.314910 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:35:52.894868 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:52.894833 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:35:55.542962 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:55.542925 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:35:57.524159 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:57.524120 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:35:57.665658 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:57.665622 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4"] Jul 22 03:35:57.665902 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:57.665874 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" podUID="76c633c5-d34a-430f-a9c1-69a965a457a6" containerName="istio-proxy" containerID="cri-o://2976f2123b6d257483d1b92ca1dcf7a84f489dc0afd611684a2fabea1d5f2d6e" gracePeriod=30 Jul 22 03:35:57.686056 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:57.686021 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:35:57.910420 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:57.910387 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:35:58.168392 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:58.168303 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:35:58.262656 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:58.262626 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:35:58.383483 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:58.383445 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:35:58.864033 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:58.863999 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt"] Jul 22 03:35:58.867803 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:58.867786 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:35:58.872050 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:58.872026 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-462b7b96-gw-openshift-default-dockercfg-2tzqz\"" Jul 22 03:35:58.901205 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:58.901179 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt"] Jul 22 03:35:58.917305 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:58.917261 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:35:59.036634 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:59.036600 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87xkq\" (UniqueName: \"kubernetes.io/projected/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-kube-api-access-87xkq\") pod \"e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt\" (UID: \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\") " pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:35:59.036805 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:59.036649 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-istio-envoy\") pod \"e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt\" (UID: \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\") " pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:35:59.036805 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:59.036678 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-workload-socket\") pod \"e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt\" (UID: \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\") " pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:35:59.036805 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:59.036720 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-istiod-ca-cert\") pod \"e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt\" (UID: \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\") " pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:35:59.036805 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:59.036783 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-istio-token\") pod \"e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt\" (UID: \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\") " pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:35:59.036946 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:59.036824 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-istio-podinfo\") pod \"e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt\" (UID: \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\") " pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:35:59.036946 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:59.036844 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-istio-data\") pod \"e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt\" (UID: \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\") " pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:35:59.036946 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:59.036899 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-workload-certs\") pod \"e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt\" (UID: \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\") " pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:35:59.037062 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:59.036960 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-credential-socket\") pod \"e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt\" (UID: \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\") " pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:35:59.138297 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:59.138210 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-istio-token\") pod \"e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt\" (UID: \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\") " pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:35:59.138297 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:59.138258 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-istio-podinfo\") pod \"e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt\" (UID: \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\") " pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:35:59.138297 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:59.138276 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-istio-data\") pod \"e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt\" (UID: \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\") " pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:35:59.138582 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:59.138305 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-workload-certs\") pod \"e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt\" (UID: \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\") " pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:35:59.138582 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:59.138349 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-credential-socket\") pod \"e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt\" (UID: \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\") " pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:35:59.138582 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:59.138398 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87xkq\" (UniqueName: \"kubernetes.io/projected/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-kube-api-access-87xkq\") pod \"e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt\" (UID: \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\") " pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:35:59.138582 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:59.138448 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-istio-envoy\") pod \"e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt\" (UID: \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\") " pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:35:59.138582 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:59.138476 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-workload-socket\") pod \"e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt\" (UID: \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\") " pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:35:59.138582 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:59.138500 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-istiod-ca-cert\") pod \"e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt\" (UID: \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\") " pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:35:59.138888 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:59.138772 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-credential-socket\") pod \"e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt\" (UID: \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\") " pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:35:59.138976 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:59.138938 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-workload-certs\") pod \"e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt\" (UID: \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\") " pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:35:59.139051 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:59.138993 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-workload-socket\") pod \"e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt\" (UID: \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\") " pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:35:59.139164 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:59.139147 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-istio-data\") pod \"e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt\" (UID: \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\") " pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:35:59.139294 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:59.139272 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-istiod-ca-cert\") pod \"e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt\" (UID: \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\") " pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:35:59.140781 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:59.140762 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-istio-envoy\") pod \"e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt\" (UID: \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\") " pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:35:59.140913 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:59.140895 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-istio-podinfo\") pod \"e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt\" (UID: \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\") " pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:35:59.148075 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:59.148052 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-istio-token\") pod \"e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt\" (UID: \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\") " pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:35:59.148202 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:59.148183 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87xkq\" (UniqueName: \"kubernetes.io/projected/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-kube-api-access-87xkq\") pod \"e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt\" (UID: \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\") " pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:35:59.180758 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:59.180725 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:35:59.335696 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:59.335666 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt"] Jul 22 03:35:59.336271 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:35:59.336249 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22c8ae91_60df_42c3_8378_b6b6f2d0cd48.slice/crio-f1c5c24843c37345d7e1618d5ace96fd84ceb10a3b2d3d05a255230c298da7db WatchSource:0}: Error finding container f1c5c24843c37345d7e1618d5ace96fd84ceb10a3b2d3d05a255230c298da7db: Status 404 returned error can't find the container with id f1c5c24843c37345d7e1618d5ace96fd84ceb10a3b2d3d05a255230c298da7db Jul 22 03:35:59.338424 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:59.338390 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:35:59.338521 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:59.338460 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:35:59.338521 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:59.338491 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:35:59.340446 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:59.340425 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:35:59.810043 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:59.809943 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" event={"ID":"22c8ae91-60df-42c3-8378-b6b6f2d0cd48","Type":"ContainerStarted","Data":"0cab384381014de0c9e185f72b1b45a61f62a1682c4b7a0d65a25e52d6e9995d"} Jul 22 03:35:59.810043 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:59.810009 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" event={"ID":"22c8ae91-60df-42c3-8378-b6b6f2d0cd48","Type":"ContainerStarted","Data":"f1c5c24843c37345d7e1618d5ace96fd84ceb10a3b2d3d05a255230c298da7db"} Jul 22 03:35:59.833011 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:35:59.832930 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" podStartSLOduration=1.832913362 podStartE2EDuration="1.832913362s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:35:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:35:59.832430877 +0000 UTC m=+1700.281414126" watchObservedRunningTime="2026-07-22 03:35:59.832913362 +0000 UTC m=+1700.281896610" Jul 22 03:36:00.181784 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:00.181748 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:36:00.186609 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:00.186582 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:36:00.821417 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:00.821379 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:36:00.822254 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:00.822227 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:36:01.308939 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:01.308848 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:36:01.557654 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:01.557624 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:36:01.658041 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:01.658002 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:36:02.034285 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:02.034192 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:36:02.156879 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:02.156845 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:36:02.836545 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:02.836221 2556 generic.go:356] "Generic (PLEG): container finished" podID="76c633c5-d34a-430f-a9c1-69a965a457a6" containerID="2976f2123b6d257483d1b92ca1dcf7a84f489dc0afd611684a2fabea1d5f2d6e" exitCode=0 Jul 22 03:36:02.836545 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:02.836318 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" event={"ID":"76c633c5-d34a-430f-a9c1-69a965a457a6","Type":"ContainerDied","Data":"2976f2123b6d257483d1b92ca1dcf7a84f489dc0afd611684a2fabea1d5f2d6e"} Jul 22 03:36:02.836545 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:02.836401 2556 scope.go:117] "RemoveContainer" containerID="eed6fe548957c5683e77b39641a3c66d1da4f493fb4b2efdc5038daedc061df5" Jul 22 03:36:02.920656 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:02.920634 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:36:03.076975 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:03.076930 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76c633c5-d34a-430f-a9c1-69a965a457a6-istio-data\") pod \"76c633c5-d34a-430f-a9c1-69a965a457a6\" (UID: \"76c633c5-d34a-430f-a9c1-69a965a457a6\") " Jul 22 03:36:03.077143 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:03.076988 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76c633c5-d34a-430f-a9c1-69a965a457a6-istio-envoy\") pod \"76c633c5-d34a-430f-a9c1-69a965a457a6\" (UID: \"76c633c5-d34a-430f-a9c1-69a965a457a6\") " Jul 22 03:36:03.077143 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:03.077010 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76c633c5-d34a-430f-a9c1-69a965a457a6-workload-certs\") pod \"76c633c5-d34a-430f-a9c1-69a965a457a6\" (UID: \"76c633c5-d34a-430f-a9c1-69a965a457a6\") " Jul 22 03:36:03.077143 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:03.077039 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76c633c5-d34a-430f-a9c1-69a965a457a6-istio-podinfo\") pod \"76c633c5-d34a-430f-a9c1-69a965a457a6\" (UID: \"76c633c5-d34a-430f-a9c1-69a965a457a6\") " Jul 22 03:36:03.077143 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:03.077055 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76c633c5-d34a-430f-a9c1-69a965a457a6-workload-socket\") pod \"76c633c5-d34a-430f-a9c1-69a965a457a6\" (UID: \"76c633c5-d34a-430f-a9c1-69a965a457a6\") " Jul 22 03:36:03.077143 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:03.077090 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fgq75\" (UniqueName: \"kubernetes.io/projected/76c633c5-d34a-430f-a9c1-69a965a457a6-kube-api-access-fgq75\") pod \"76c633c5-d34a-430f-a9c1-69a965a457a6\" (UID: \"76c633c5-d34a-430f-a9c1-69a965a457a6\") " Jul 22 03:36:03.077414 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:03.077155 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76c633c5-d34a-430f-a9c1-69a965a457a6-istiod-ca-cert\") pod \"76c633c5-d34a-430f-a9c1-69a965a457a6\" (UID: \"76c633c5-d34a-430f-a9c1-69a965a457a6\") " Jul 22 03:36:03.077414 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:03.077316 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76c633c5-d34a-430f-a9c1-69a965a457a6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "76c633c5-d34a-430f-a9c1-69a965a457a6" (UID: "76c633c5-d34a-430f-a9c1-69a965a457a6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:36:03.077414 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:03.077402 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76c633c5-d34a-430f-a9c1-69a965a457a6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "76c633c5-d34a-430f-a9c1-69a965a457a6" (UID: "76c633c5-d34a-430f-a9c1-69a965a457a6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:36:03.077741 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:03.077502 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76c633c5-d34a-430f-a9c1-69a965a457a6-credential-socket\") pod \"76c633c5-d34a-430f-a9c1-69a965a457a6\" (UID: \"76c633c5-d34a-430f-a9c1-69a965a457a6\") " Jul 22 03:36:03.077741 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:03.077564 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76c633c5-d34a-430f-a9c1-69a965a457a6-istio-token\") pod \"76c633c5-d34a-430f-a9c1-69a965a457a6\" (UID: \"76c633c5-d34a-430f-a9c1-69a965a457a6\") " Jul 22 03:36:03.077741 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:03.077628 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76c633c5-d34a-430f-a9c1-69a965a457a6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "76c633c5-d34a-430f-a9c1-69a965a457a6" (UID: "76c633c5-d34a-430f-a9c1-69a965a457a6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:36:03.077917 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:03.077739 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76c633c5-d34a-430f-a9c1-69a965a457a6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "76c633c5-d34a-430f-a9c1-69a965a457a6" (UID: "76c633c5-d34a-430f-a9c1-69a965a457a6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:36:03.077917 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:03.077781 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76c633c5-d34a-430f-a9c1-69a965a457a6-istio-data" (OuterVolumeSpecName: "istio-data") pod "76c633c5-d34a-430f-a9c1-69a965a457a6" (UID: "76c633c5-d34a-430f-a9c1-69a965a457a6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:36:03.078048 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:03.077982 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76c633c5-d34a-430f-a9c1-69a965a457a6-istiod-ca-cert\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:36:03.078048 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:03.078003 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76c633c5-d34a-430f-a9c1-69a965a457a6-credential-socket\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:36:03.078048 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:03.078019 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76c633c5-d34a-430f-a9c1-69a965a457a6-istio-data\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:36:03.078048 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:03.078032 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76c633c5-d34a-430f-a9c1-69a965a457a6-workload-certs\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:36:03.078048 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:03.078045 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76c633c5-d34a-430f-a9c1-69a965a457a6-workload-socket\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:36:03.079347 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:03.079324 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/76c633c5-d34a-430f-a9c1-69a965a457a6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "76c633c5-d34a-430f-a9c1-69a965a457a6" (UID: "76c633c5-d34a-430f-a9c1-69a965a457a6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 03:36:03.079620 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:03.079602 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76c633c5-d34a-430f-a9c1-69a965a457a6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "76c633c5-d34a-430f-a9c1-69a965a457a6" (UID: "76c633c5-d34a-430f-a9c1-69a965a457a6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:36:03.079911 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:03.079894 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76c633c5-d34a-430f-a9c1-69a965a457a6-kube-api-access-fgq75" (OuterVolumeSpecName: "kube-api-access-fgq75") pod "76c633c5-d34a-430f-a9c1-69a965a457a6" (UID: "76c633c5-d34a-430f-a9c1-69a965a457a6"). InnerVolumeSpecName "kube-api-access-fgq75". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:36:03.079998 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:03.079921 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76c633c5-d34a-430f-a9c1-69a965a457a6-istio-token" (OuterVolumeSpecName: "istio-token") pod "76c633c5-d34a-430f-a9c1-69a965a457a6" (UID: "76c633c5-d34a-430f-a9c1-69a965a457a6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:36:03.179146 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:03.178914 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76c633c5-d34a-430f-a9c1-69a965a457a6-istio-token\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:36:03.179146 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:03.178945 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76c633c5-d34a-430f-a9c1-69a965a457a6-istio-envoy\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:36:03.179146 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:03.178973 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76c633c5-d34a-430f-a9c1-69a965a457a6-istio-podinfo\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:36:03.179146 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:03.178988 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fgq75\" (UniqueName: \"kubernetes.io/projected/76c633c5-d34a-430f-a9c1-69a965a457a6-kube-api-access-fgq75\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:36:03.842858 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:03.842831 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" Jul 22 03:36:03.843280 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:03.842830 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4" event={"ID":"76c633c5-d34a-430f-a9c1-69a965a457a6","Type":"ContainerDied","Data":"d767964be5c4612eba8d188478e0d7f8610b3e78596381ebb3a9fbb24e96d6d5"} Jul 22 03:36:03.843280 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:03.842985 2556 scope.go:117] "RemoveContainer" containerID="2976f2123b6d257483d1b92ca1dcf7a84f489dc0afd611684a2fabea1d5f2d6e" Jul 22 03:36:03.869351 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:03.869321 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4"] Jul 22 03:36:03.876884 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:03.876861 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-6abd7a28-openshift-default-d55959fbf-p4xr4"] Jul 22 03:36:04.024577 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:04.024544 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76c633c5-d34a-430f-a9c1-69a965a457a6" path="/var/lib/kubelet/pods/76c633c5-d34a-430f-a9c1-69a965a457a6/volumes" Jul 22 03:36:07.034145 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:07.034107 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:36:14.911800 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:14.911715 2556 generic.go:356] "Generic (PLEG): container finished" podID="22c8ae91-60df-42c3-8378-b6b6f2d0cd48" containerID="0cab384381014de0c9e185f72b1b45a61f62a1682c4b7a0d65a25e52d6e9995d" exitCode=0 Jul 22 03:36:14.912217 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:14.911790 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" event={"ID":"22c8ae91-60df-42c3-8378-b6b6f2d0cd48","Type":"ContainerDied","Data":"0cab384381014de0c9e185f72b1b45a61f62a1682c4b7a0d65a25e52d6e9995d"} Jul 22 03:36:14.912259 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:14.912245 2556 scope.go:117] "RemoveContainer" containerID="0cab384381014de0c9e185f72b1b45a61f62a1682c4b7a0d65a25e52d6e9995d" Jul 22 03:36:14.913310 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:14.913269 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:36:14.913420 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:14.913362 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:36:14.913420 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:14.913400 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:36:15.821859 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:15.821813 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:36:15.918920 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:15.918877 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" event={"ID":"22c8ae91-60df-42c3-8378-b6b6f2d0cd48","Type":"ContainerStarted","Data":"12c6c476afcf4898e03168a195575b3278f80a31e39f73a9be9732dbda66d64c"} Jul 22 03:36:16.183276 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:16.183172 2556 patch_prober.go:28] interesting pod/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 22 03:36:16.183276 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:16.183251 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" podUID="22c8ae91-60df-42c3-8378-b6b6f2d0cd48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 22 03:36:16.183587 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:16.183563 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:36:17.181212 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:17.181174 2556 patch_prober.go:28] interesting pod/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 22 03:36:17.181712 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:17.181251 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" podUID="22c8ae91-60df-42c3-8378-b6b6f2d0cd48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 22 03:36:18.182047 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:18.181572 2556 patch_prober.go:28] interesting pod/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 22 03:36:18.182047 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:18.181637 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" podUID="22c8ae91-60df-42c3-8378-b6b6f2d0cd48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 22 03:36:19.181783 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:19.181751 2556 patch_prober.go:28] interesting pod/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 22 03:36:19.182107 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:19.182066 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" podUID="22c8ae91-60df-42c3-8378-b6b6f2d0cd48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 22 03:36:20.185746 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:20.185710 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:36:20.186149 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:20.185978 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:36:20.186656 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:20.186639 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:36:36.765111 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:36.765070 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:36:39.829686 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:39.829643 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:36:41.345010 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:41.344968 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:36:41.478178 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:41.478141 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt"] Jul 22 03:36:41.478417 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:41.478390 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" podUID="22c8ae91-60df-42c3-8378-b6b6f2d0cd48" containerName="istio-proxy" containerID="cri-o://12c6c476afcf4898e03168a195575b3278f80a31e39f73a9be9732dbda66d64c" gracePeriod=30 Jul 22 03:36:41.502349 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:41.502318 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:36:41.721158 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:41.721076 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:36:41.948371 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:41.948340 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:36:42.245432 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.245396 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc"] Jul 22 03:36:42.245818 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.245805 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76c633c5-d34a-430f-a9c1-69a965a457a6" containerName="istio-proxy" Jul 22 03:36:42.245874 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.245820 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="76c633c5-d34a-430f-a9c1-69a965a457a6" containerName="istio-proxy" Jul 22 03:36:42.245874 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.245833 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76c633c5-d34a-430f-a9c1-69a965a457a6" containerName="istio-proxy" Jul 22 03:36:42.245874 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.245838 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="76c633c5-d34a-430f-a9c1-69a965a457a6" containerName="istio-proxy" Jul 22 03:36:42.245989 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.245899 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="76c633c5-d34a-430f-a9c1-69a965a457a6" containerName="istio-proxy" Jul 22 03:36:42.250073 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.250054 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:36:42.253251 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.253226 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-8cd3c6d0-openshift-default-dockercfg-74hvk\"" Jul 22 03:36:42.264323 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.264293 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc"] Jul 22 03:36:42.270178 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.270148 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:36:42.346153 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.346115 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-workload-socket\") pod \"e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc\" (UID: \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\") " pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:36:42.346607 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.346203 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbqv6\" (UniqueName: \"kubernetes.io/projected/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-kube-api-access-cbqv6\") pod \"e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc\" (UID: \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\") " pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:36:42.346607 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.346265 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-workload-certs\") pod \"e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc\" (UID: \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\") " pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:36:42.346607 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.346360 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-istio-token\") pod \"e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc\" (UID: \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\") " pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:36:42.346607 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.346417 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-istio-podinfo\") pod \"e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc\" (UID: \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\") " pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:36:42.346607 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.346462 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-istio-envoy\") pod \"e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc\" (UID: \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\") " pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:36:42.346607 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.346492 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-istiod-ca-cert\") pod \"e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc\" (UID: \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\") " pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:36:42.346607 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.346546 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-credential-socket\") pod \"e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc\" (UID: \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\") " pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:36:42.346607 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.346571 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-istio-data\") pod \"e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc\" (UID: \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\") " pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:36:42.447450 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.447420 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-istio-token\") pod \"e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc\" (UID: \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\") " pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:36:42.447644 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.447459 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-istio-podinfo\") pod \"e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc\" (UID: \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\") " pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:36:42.447644 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.447483 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-istio-envoy\") pod \"e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc\" (UID: \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\") " pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:36:42.447644 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.447500 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-istiod-ca-cert\") pod \"e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc\" (UID: \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\") " pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:36:42.447644 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.447535 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-credential-socket\") pod \"e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc\" (UID: \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\") " pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:36:42.447644 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.447549 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-istio-data\") pod \"e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc\" (UID: \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\") " pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:36:42.447644 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.447613 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-workload-socket\") pod \"e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc\" (UID: \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\") " pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:36:42.447644 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.447637 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbqv6\" (UniqueName: \"kubernetes.io/projected/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-kube-api-access-cbqv6\") pod \"e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc\" (UID: \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\") " pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:36:42.448050 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.447685 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-workload-certs\") pod \"e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc\" (UID: \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\") " pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:36:42.448050 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.447939 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-credential-socket\") pod \"e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc\" (UID: \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\") " pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:36:42.448194 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.448171 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-workload-certs\") pod \"e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc\" (UID: \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\") " pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:36:42.448251 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.448233 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-workload-socket\") pod \"e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc\" (UID: \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\") " pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:36:42.448413 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.448390 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-istiod-ca-cert\") pod \"e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc\" (UID: \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\") " pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:36:42.448524 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.448416 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-istio-data\") pod \"e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc\" (UID: \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\") " pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:36:42.450292 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.450259 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-istio-envoy\") pod \"e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc\" (UID: \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\") " pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:36:42.450408 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.450353 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-istio-podinfo\") pod \"e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc\" (UID: \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\") " pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:36:42.459178 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.459155 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbqv6\" (UniqueName: \"kubernetes.io/projected/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-kube-api-access-cbqv6\") pod \"e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc\" (UID: \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\") " pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:36:42.460252 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.460231 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-istio-token\") pod \"e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc\" (UID: \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\") " pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:36:42.507062 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.506979 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:36:42.561148 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.561116 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:36:42.708856 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.708821 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc"] Jul 22 03:36:42.710147 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:36:42.710124 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2cb1e23_7a74_44c4_91b1_41c218ecd6c9.slice/crio-82b1fd72b7bf9f159235f75e39bbe8d94b9bcd1c0c1779c83af60c8aa179dbc7 WatchSource:0}: Error finding container 82b1fd72b7bf9f159235f75e39bbe8d94b9bcd1c0c1779c83af60c8aa179dbc7: Status 404 returned error can't find the container with id 82b1fd72b7bf9f159235f75e39bbe8d94b9bcd1c0c1779c83af60c8aa179dbc7 Jul 22 03:36:42.712354 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.712323 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:36:42.712436 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.712384 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:36:42.712436 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:42.712412 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:36:43.071520 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:43.071485 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" event={"ID":"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9","Type":"ContainerStarted","Data":"3892fec4837a084e9c3101aa70c9242ca72d76a65e2b16e1defa096109349212"} Jul 22 03:36:43.071715 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:43.071548 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" event={"ID":"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9","Type":"ContainerStarted","Data":"82b1fd72b7bf9f159235f75e39bbe8d94b9bcd1c0c1779c83af60c8aa179dbc7"} Jul 22 03:36:43.098652 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:43.098603 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" podStartSLOduration=1.098588058 podStartE2EDuration="1.098588058s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:36:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:36:43.093921425 +0000 UTC m=+1743.542904673" watchObservedRunningTime="2026-07-22 03:36:43.098588058 +0000 UTC m=+1743.547571306" Jul 22 03:36:43.561636 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:43.561558 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:36:43.566490 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:43.566466 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:36:44.079111 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:44.079069 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:36:44.079989 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:44.079944 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:36:44.494153 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:44.494067 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:36:44.603345 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:44.603314 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:36:44.776194 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:44.776111 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:36:44.920171 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:44.920133 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:36:45.295186 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:45.295149 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:36:45.407936 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:45.407898 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:36:46.733478 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:46.733455 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:36:46.891665 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:46.891633 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-87xkq\" (UniqueName: \"kubernetes.io/projected/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-kube-api-access-87xkq\") pod \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\" (UID: \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\") " Jul 22 03:36:46.891849 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:46.891702 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-istio-envoy\") pod \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\" (UID: \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\") " Jul 22 03:36:46.891849 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:46.891729 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-istio-data\") pod \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\" (UID: \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\") " Jul 22 03:36:46.891849 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:46.891757 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-istiod-ca-cert\") pod \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\" (UID: \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\") " Jul 22 03:36:46.891849 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:46.891783 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-workload-certs\") pod \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\" (UID: \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\") " Jul 22 03:36:46.891849 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:46.891827 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-workload-socket\") pod \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\" (UID: \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\") " Jul 22 03:36:46.892142 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:46.891852 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-credential-socket\") pod \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\" (UID: \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\") " Jul 22 03:36:46.892142 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:46.891907 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-istio-token\") pod \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\" (UID: \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\") " Jul 22 03:36:46.892142 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:46.891976 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-istio-podinfo\") pod \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\" (UID: \"22c8ae91-60df-42c3-8378-b6b6f2d0cd48\") " Jul 22 03:36:46.892284 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:46.892177 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "22c8ae91-60df-42c3-8378-b6b6f2d0cd48" (UID: "22c8ae91-60df-42c3-8378-b6b6f2d0cd48"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:36:46.892284 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:46.892242 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "22c8ae91-60df-42c3-8378-b6b6f2d0cd48" (UID: "22c8ae91-60df-42c3-8378-b6b6f2d0cd48"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:36:46.892284 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:46.892200 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "22c8ae91-60df-42c3-8378-b6b6f2d0cd48" (UID: "22c8ae91-60df-42c3-8378-b6b6f2d0cd48"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:36:46.892284 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:46.892260 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "22c8ae91-60df-42c3-8378-b6b6f2d0cd48" (UID: "22c8ae91-60df-42c3-8378-b6b6f2d0cd48"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:36:46.892547 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:46.892353 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-istiod-ca-cert\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:36:46.892547 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:46.892372 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-workload-certs\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:36:46.892547 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:46.892385 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-workload-socket\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:36:46.892547 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:46.892398 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-credential-socket\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:36:46.893642 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:46.893058 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-istio-data" (OuterVolumeSpecName: "istio-data") pod "22c8ae91-60df-42c3-8378-b6b6f2d0cd48" (UID: "22c8ae91-60df-42c3-8378-b6b6f2d0cd48"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:36:46.894279 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:46.894250 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "22c8ae91-60df-42c3-8378-b6b6f2d0cd48" (UID: "22c8ae91-60df-42c3-8378-b6b6f2d0cd48"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:36:46.894522 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:46.894421 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "22c8ae91-60df-42c3-8378-b6b6f2d0cd48" (UID: "22c8ae91-60df-42c3-8378-b6b6f2d0cd48"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 03:36:46.894522 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:46.894447 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-kube-api-access-87xkq" (OuterVolumeSpecName: "kube-api-access-87xkq") pod "22c8ae91-60df-42c3-8378-b6b6f2d0cd48" (UID: "22c8ae91-60df-42c3-8378-b6b6f2d0cd48"). InnerVolumeSpecName "kube-api-access-87xkq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:36:46.894716 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:46.894602 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-istio-token" (OuterVolumeSpecName: "istio-token") pod "22c8ae91-60df-42c3-8378-b6b6f2d0cd48" (UID: "22c8ae91-60df-42c3-8378-b6b6f2d0cd48"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:36:46.993576 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:46.993550 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-istio-token\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:36:46.993576 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:46.993574 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-istio-podinfo\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:36:46.993576 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:46.993584 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87xkq\" (UniqueName: \"kubernetes.io/projected/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-kube-api-access-87xkq\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:36:46.993797 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:46.993594 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-istio-envoy\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:36:46.993797 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:46.993603 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22c8ae91-60df-42c3-8378-b6b6f2d0cd48-istio-data\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:36:47.097503 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:47.097466 2556 generic.go:356] "Generic (PLEG): container finished" podID="22c8ae91-60df-42c3-8378-b6b6f2d0cd48" containerID="12c6c476afcf4898e03168a195575b3278f80a31e39f73a9be9732dbda66d64c" exitCode=0 Jul 22 03:36:47.097627 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:47.097518 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" Jul 22 03:36:47.097627 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:47.097550 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" event={"ID":"22c8ae91-60df-42c3-8378-b6b6f2d0cd48","Type":"ContainerDied","Data":"12c6c476afcf4898e03168a195575b3278f80a31e39f73a9be9732dbda66d64c"} Jul 22 03:36:47.097627 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:47.097603 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt" event={"ID":"22c8ae91-60df-42c3-8378-b6b6f2d0cd48","Type":"ContainerDied","Data":"f1c5c24843c37345d7e1618d5ace96fd84ceb10a3b2d3d05a255230c298da7db"} Jul 22 03:36:47.097627 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:47.097627 2556 scope.go:117] "RemoveContainer" containerID="12c6c476afcf4898e03168a195575b3278f80a31e39f73a9be9732dbda66d64c" Jul 22 03:36:47.107163 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:47.107147 2556 scope.go:117] "RemoveContainer" containerID="0cab384381014de0c9e185f72b1b45a61f62a1682c4b7a0d65a25e52d6e9995d" Jul 22 03:36:47.118818 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:47.118799 2556 scope.go:117] "RemoveContainer" containerID="12c6c476afcf4898e03168a195575b3278f80a31e39f73a9be9732dbda66d64c" Jul 22 03:36:47.119111 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:36:47.119090 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12c6c476afcf4898e03168a195575b3278f80a31e39f73a9be9732dbda66d64c\": container with ID starting with 12c6c476afcf4898e03168a195575b3278f80a31e39f73a9be9732dbda66d64c not found: ID does not exist" containerID="12c6c476afcf4898e03168a195575b3278f80a31e39f73a9be9732dbda66d64c" Jul 22 03:36:47.119199 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:47.119124 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12c6c476afcf4898e03168a195575b3278f80a31e39f73a9be9732dbda66d64c"} err="failed to get container status \"12c6c476afcf4898e03168a195575b3278f80a31e39f73a9be9732dbda66d64c\": rpc error: code = NotFound desc = could not find container \"12c6c476afcf4898e03168a195575b3278f80a31e39f73a9be9732dbda66d64c\": container with ID starting with 12c6c476afcf4898e03168a195575b3278f80a31e39f73a9be9732dbda66d64c not found: ID does not exist" Jul 22 03:36:47.119199 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:47.119147 2556 scope.go:117] "RemoveContainer" containerID="0cab384381014de0c9e185f72b1b45a61f62a1682c4b7a0d65a25e52d6e9995d" Jul 22 03:36:47.119378 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:36:47.119361 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cab384381014de0c9e185f72b1b45a61f62a1682c4b7a0d65a25e52d6e9995d\": container with ID starting with 0cab384381014de0c9e185f72b1b45a61f62a1682c4b7a0d65a25e52d6e9995d not found: ID does not exist" containerID="0cab384381014de0c9e185f72b1b45a61f62a1682c4b7a0d65a25e52d6e9995d" Jul 22 03:36:47.119418 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:47.119386 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cab384381014de0c9e185f72b1b45a61f62a1682c4b7a0d65a25e52d6e9995d"} err="failed to get container status \"0cab384381014de0c9e185f72b1b45a61f62a1682c4b7a0d65a25e52d6e9995d\": rpc error: code = NotFound desc = could not find container \"0cab384381014de0c9e185f72b1b45a61f62a1682c4b7a0d65a25e52d6e9995d\": container with ID starting with 0cab384381014de0c9e185f72b1b45a61f62a1682c4b7a0d65a25e52d6e9995d not found: ID does not exist" Jul 22 03:36:47.124898 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:47.124877 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt"] Jul 22 03:36:47.129974 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:47.129941 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-462b7b96-gw-openshift-default-dcd65db9f-qt8pt"] Jul 22 03:36:48.023924 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:48.023893 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22c8ae91-60df-42c3-8378-b6b6f2d0cd48" path="/var/lib/kubelet/pods/22c8ae91-60df-42c3-8378-b6b6f2d0cd48/volumes" Jul 22 03:36:50.929075 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:50.929037 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:36:59.080754 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:59.080715 2556 patch_prober.go:28] interesting pod/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 22 03:36:59.081200 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:59.080775 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" podUID="e2cb1e23-7a74-44c4-91b1-41c218ecd6c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 22 03:36:59.168862 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:59.168830 2556 generic.go:356] "Generic (PLEG): container finished" podID="e2cb1e23-7a74-44c4-91b1-41c218ecd6c9" containerID="3892fec4837a084e9c3101aa70c9242ca72d76a65e2b16e1defa096109349212" exitCode=0 Jul 22 03:36:59.169040 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:59.168898 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" event={"ID":"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9","Type":"ContainerDied","Data":"3892fec4837a084e9c3101aa70c9242ca72d76a65e2b16e1defa096109349212"} Jul 22 03:36:59.169333 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:59.169319 2556 scope.go:117] "RemoveContainer" containerID="3892fec4837a084e9c3101aa70c9242ca72d76a65e2b16e1defa096109349212" Jul 22 03:36:59.170487 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:59.170387 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:36:59.170487 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:59.170477 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:36:59.170881 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:36:59.170527 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:37:00.178766 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:00.178729 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" event={"ID":"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9","Type":"ContainerStarted","Data":"8c5e804000c0391f6d4d1924681866329e44dbaf492d7b693e2dfc0e401b94f6"} Jul 22 03:37:00.561650 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:00.561564 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:37:00.561922 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:00.561896 2556 patch_prober.go:28] interesting pod/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 22 03:37:00.562073 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:00.561942 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" podUID="e2cb1e23-7a74-44c4-91b1-41c218ecd6c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 22 03:37:01.562022 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:01.561947 2556 patch_prober.go:28] interesting pod/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 22 03:37:01.562429 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:01.562045 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" podUID="e2cb1e23-7a74-44c4-91b1-41c218ecd6c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 22 03:37:02.562036 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:02.561998 2556 patch_prober.go:28] interesting pod/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 22 03:37:02.562423 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:02.562063 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" podUID="e2cb1e23-7a74-44c4-91b1-41c218ecd6c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 22 03:37:03.562452 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:03.562424 2556 patch_prober.go:28] interesting pod/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 22 03:37:03.562802 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:03.562476 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" podUID="e2cb1e23-7a74-44c4-91b1-41c218ecd6c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 22 03:37:04.562204 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:04.562172 2556 patch_prober.go:28] interesting pod/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 22 03:37:04.562379 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:04.562226 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" podUID="e2cb1e23-7a74-44c4-91b1-41c218ecd6c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 22 03:37:05.565469 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:05.565440 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:37:05.565827 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:05.565794 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:37:05.566561 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:05.566541 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:37:05.777892 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:05.777863 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:37:20.873091 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:20.873050 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:37:23.646217 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:23.646178 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:37:25.358524 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:25.358477 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:37:26.178442 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:26.178405 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:37:26.289221 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:26.289189 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:37:26.551906 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:26.551814 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc"] Jul 22 03:37:26.552364 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:26.552043 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" podUID="e2cb1e23-7a74-44c4-91b1-41c218ecd6c9" containerName="istio-proxy" containerID="cri-o://8c5e804000c0391f6d4d1924681866329e44dbaf492d7b693e2dfc0e401b94f6" gracePeriod=30 Jul 22 03:37:26.575450 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:26.575414 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:37:26.678196 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:26.678145 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:37:26.894847 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:26.894809 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:37:27.060170 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.060135 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:37:27.335466 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.335434 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267"] Jul 22 03:37:27.335824 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.335812 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22c8ae91-60df-42c3-8378-b6b6f2d0cd48" containerName="istio-proxy" Jul 22 03:37:27.335871 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.335825 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="22c8ae91-60df-42c3-8378-b6b6f2d0cd48" containerName="istio-proxy" Jul 22 03:37:27.335871 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.335837 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22c8ae91-60df-42c3-8378-b6b6f2d0cd48" containerName="istio-proxy" Jul 22 03:37:27.335871 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.335843 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="22c8ae91-60df-42c3-8378-b6b6f2d0cd48" containerName="istio-proxy" Jul 22 03:37:27.335994 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.335913 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="22c8ae91-60df-42c3-8378-b6b6f2d0cd48" containerName="istio-proxy" Jul 22 03:37:27.335994 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.335922 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="76c633c5-d34a-430f-a9c1-69a965a457a6" containerName="istio-proxy" Jul 22 03:37:27.339134 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.339117 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:37:27.342195 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.342174 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-1535280c-openshift-default-dockercfg-zhtc8\"" Jul 22 03:37:27.355111 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.355078 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267"] Jul 22 03:37:27.359134 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.359112 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:37:27.484041 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.484004 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f5d0778-0ee2-454c-932c-ff9276ef6c86-istio-podinfo\") pod \"e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267\" (UID: \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\") " pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:37:27.484231 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.484053 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f5d0778-0ee2-454c-932c-ff9276ef6c86-istiod-ca-cert\") pod \"e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267\" (UID: \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\") " pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:37:27.484231 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.484093 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7c46q\" (UniqueName: \"kubernetes.io/projected/3f5d0778-0ee2-454c-932c-ff9276ef6c86-kube-api-access-7c46q\") pod \"e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267\" (UID: \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\") " pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:37:27.484231 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.484142 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f5d0778-0ee2-454c-932c-ff9276ef6c86-istio-data\") pod \"e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267\" (UID: \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\") " pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:37:27.484231 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.484166 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f5d0778-0ee2-454c-932c-ff9276ef6c86-istio-envoy\") pod \"e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267\" (UID: \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\") " pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:37:27.484231 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.484195 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f5d0778-0ee2-454c-932c-ff9276ef6c86-workload-socket\") pod \"e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267\" (UID: \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\") " pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:37:27.484231 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.484234 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f5d0778-0ee2-454c-932c-ff9276ef6c86-credential-socket\") pod \"e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267\" (UID: \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\") " pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:37:27.484526 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.484293 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f5d0778-0ee2-454c-932c-ff9276ef6c86-istio-token\") pod \"e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267\" (UID: \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\") " pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:37:27.484526 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.484369 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f5d0778-0ee2-454c-932c-ff9276ef6c86-workload-certs\") pod \"e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267\" (UID: \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\") " pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:37:27.585791 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.585710 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f5d0778-0ee2-454c-932c-ff9276ef6c86-credential-socket\") pod \"e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267\" (UID: \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\") " pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:37:27.585791 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.585746 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f5d0778-0ee2-454c-932c-ff9276ef6c86-istio-token\") pod \"e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267\" (UID: \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\") " pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:37:27.585791 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.585772 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f5d0778-0ee2-454c-932c-ff9276ef6c86-workload-certs\") pod \"e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267\" (UID: \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\") " pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:37:27.586380 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.585824 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f5d0778-0ee2-454c-932c-ff9276ef6c86-istio-podinfo\") pod \"e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267\" (UID: \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\") " pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:37:27.586380 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.585850 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f5d0778-0ee2-454c-932c-ff9276ef6c86-istiod-ca-cert\") pod \"e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267\" (UID: \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\") " pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:37:27.586380 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.585921 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7c46q\" (UniqueName: \"kubernetes.io/projected/3f5d0778-0ee2-454c-932c-ff9276ef6c86-kube-api-access-7c46q\") pod \"e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267\" (UID: \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\") " pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:37:27.586380 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.586006 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f5d0778-0ee2-454c-932c-ff9276ef6c86-istio-data\") pod \"e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267\" (UID: \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\") " pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:37:27.586380 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.586043 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f5d0778-0ee2-454c-932c-ff9276ef6c86-istio-envoy\") pod \"e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267\" (UID: \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\") " pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:37:27.586380 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.586096 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f5d0778-0ee2-454c-932c-ff9276ef6c86-workload-socket\") pod \"e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267\" (UID: \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\") " pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:37:27.586380 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.586170 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f5d0778-0ee2-454c-932c-ff9276ef6c86-credential-socket\") pod \"e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267\" (UID: \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\") " pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:37:27.586380 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.586283 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f5d0778-0ee2-454c-932c-ff9276ef6c86-workload-certs\") pod \"e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267\" (UID: \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\") " pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:37:27.586380 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.586338 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f5d0778-0ee2-454c-932c-ff9276ef6c86-istio-data\") pod \"e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267\" (UID: \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\") " pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:37:27.586688 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.586484 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f5d0778-0ee2-454c-932c-ff9276ef6c86-workload-socket\") pod \"e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267\" (UID: \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\") " pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:37:27.586688 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.586535 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f5d0778-0ee2-454c-932c-ff9276ef6c86-istiod-ca-cert\") pod \"e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267\" (UID: \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\") " pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:37:27.588374 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.588353 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f5d0778-0ee2-454c-932c-ff9276ef6c86-istio-envoy\") pod \"e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267\" (UID: \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\") " pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:37:27.588621 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.588603 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f5d0778-0ee2-454c-932c-ff9276ef6c86-istio-podinfo\") pod \"e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267\" (UID: \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\") " pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:37:27.594437 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.594414 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f5d0778-0ee2-454c-932c-ff9276ef6c86-istio-token\") pod \"e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267\" (UID: \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\") " pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:37:27.594685 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.594667 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7c46q\" (UniqueName: \"kubernetes.io/projected/3f5d0778-0ee2-454c-932c-ff9276ef6c86-kube-api-access-7c46q\") pod \"e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267\" (UID: \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\") " pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:37:27.650210 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.650172 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:37:27.807229 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.807199 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267"] Jul 22 03:37:27.808261 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:37:27.808235 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f5d0778_0ee2_454c_932c_ff9276ef6c86.slice/crio-c865998ecc52574c9073c207c359806d1d63b149655def4c57d76181d7047287 WatchSource:0}: Error finding container c865998ecc52574c9073c207c359806d1d63b149655def4c57d76181d7047287: Status 404 returned error can't find the container with id c865998ecc52574c9073c207c359806d1d63b149655def4c57d76181d7047287 Jul 22 03:37:27.810542 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.810506 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:37:27.810619 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.810590 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:37:27.810661 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:27.810631 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:37:28.096290 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:28.096177 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:37:28.351195 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:28.351076 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" event={"ID":"3f5d0778-0ee2-454c-932c-ff9276ef6c86","Type":"ContainerStarted","Data":"227905fd70e55b9e1291d2d57b4494c1b62f34a5c368fdd351405191562d9a33"} Jul 22 03:37:28.351340 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:28.351256 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" event={"ID":"3f5d0778-0ee2-454c-932c-ff9276ef6c86","Type":"ContainerStarted","Data":"c865998ecc52574c9073c207c359806d1d63b149655def4c57d76181d7047287"} Jul 22 03:37:28.371914 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:28.371867 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" podStartSLOduration=1.371854144 podStartE2EDuration="1.371854144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:37:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:37:28.371517094 +0000 UTC m=+1788.820500353" watchObservedRunningTime="2026-07-22 03:37:28.371854144 +0000 UTC m=+1788.820837393" Jul 22 03:37:28.650478 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:28.650389 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:37:28.656047 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:28.656023 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:37:29.356420 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:29.356385 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:37:29.357375 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:29.357353 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:37:29.763772 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:29.763686 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:37:29.895108 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:29.895075 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:37:30.020173 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:30.020061 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:37:30.120297 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:30.120259 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:37:30.740903 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:30.740869 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:37:30.844519 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:30.844485 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:37:31.812939 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:31.812914 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:37:31.822560 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:31.822537 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-istio-token\") pod \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\" (UID: \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\") " Jul 22 03:37:31.822704 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:31.822591 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbqv6\" (UniqueName: \"kubernetes.io/projected/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-kube-api-access-cbqv6\") pod \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\" (UID: \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\") " Jul 22 03:37:31.822704 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:31.822624 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-workload-socket\") pod \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\" (UID: \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\") " Jul 22 03:37:31.822704 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:31.822656 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-istiod-ca-cert\") pod \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\" (UID: \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\") " Jul 22 03:37:31.822704 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:31.822688 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-istio-data\") pod \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\" (UID: \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\") " Jul 22 03:37:31.822913 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:31.822707 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-istio-podinfo\") pod \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\" (UID: \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\") " Jul 22 03:37:31.822913 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:31.822760 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-workload-certs\") pod \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\" (UID: \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\") " Jul 22 03:37:31.822913 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:31.822780 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-istio-envoy\") pod \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\" (UID: \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\") " Jul 22 03:37:31.822913 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:31.822818 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-credential-socket\") pod \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\" (UID: \"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9\") " Jul 22 03:37:31.823169 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:31.822993 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e2cb1e23-7a74-44c4-91b1-41c218ecd6c9" (UID: "e2cb1e23-7a74-44c4-91b1-41c218ecd6c9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:37:31.823169 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:31.823067 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e2cb1e23-7a74-44c4-91b1-41c218ecd6c9" (UID: "e2cb1e23-7a74-44c4-91b1-41c218ecd6c9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:37:31.823267 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:31.823206 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e2cb1e23-7a74-44c4-91b1-41c218ecd6c9" (UID: "e2cb1e23-7a74-44c4-91b1-41c218ecd6c9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:37:31.823396 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:31.823360 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-workload-socket\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:37:31.823396 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:31.823390 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-istiod-ca-cert\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:37:31.823508 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:31.823406 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-workload-certs\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:37:31.823508 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:31.823359 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e2cb1e23-7a74-44c4-91b1-41c218ecd6c9" (UID: "e2cb1e23-7a74-44c4-91b1-41c218ecd6c9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:37:31.823619 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:31.823574 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-istio-data" (OuterVolumeSpecName: "istio-data") pod "e2cb1e23-7a74-44c4-91b1-41c218ecd6c9" (UID: "e2cb1e23-7a74-44c4-91b1-41c218ecd6c9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:37:31.824881 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:31.824856 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-istio-token" (OuterVolumeSpecName: "istio-token") pod "e2cb1e23-7a74-44c4-91b1-41c218ecd6c9" (UID: "e2cb1e23-7a74-44c4-91b1-41c218ecd6c9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:37:31.825003 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:31.824909 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-kube-api-access-cbqv6" (OuterVolumeSpecName: "kube-api-access-cbqv6") pod "e2cb1e23-7a74-44c4-91b1-41c218ecd6c9" (UID: "e2cb1e23-7a74-44c4-91b1-41c218ecd6c9"). InnerVolumeSpecName "kube-api-access-cbqv6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:37:31.825003 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:31.824934 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e2cb1e23-7a74-44c4-91b1-41c218ecd6c9" (UID: "e2cb1e23-7a74-44c4-91b1-41c218ecd6c9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 03:37:31.825090 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:31.825050 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e2cb1e23-7a74-44c4-91b1-41c218ecd6c9" (UID: "e2cb1e23-7a74-44c4-91b1-41c218ecd6c9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:37:31.924328 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:31.924293 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-istio-data\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:37:31.924328 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:31.924323 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-istio-podinfo\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:37:31.924328 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:31.924334 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-istio-envoy\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:37:31.924769 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:31.924343 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-credential-socket\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:37:31.924769 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:31.924353 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-istio-token\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:37:31.924769 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:31.924362 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbqv6\" (UniqueName: \"kubernetes.io/projected/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9-kube-api-access-cbqv6\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:37:32.375026 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:32.374989 2556 generic.go:356] "Generic (PLEG): container finished" podID="e2cb1e23-7a74-44c4-91b1-41c218ecd6c9" containerID="8c5e804000c0391f6d4d1924681866329e44dbaf492d7b693e2dfc0e401b94f6" exitCode=0 Jul 22 03:37:32.375203 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:32.375076 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" event={"ID":"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9","Type":"ContainerDied","Data":"8c5e804000c0391f6d4d1924681866329e44dbaf492d7b693e2dfc0e401b94f6"} Jul 22 03:37:32.375203 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:32.375097 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" Jul 22 03:37:32.375203 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:32.375136 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc" event={"ID":"e2cb1e23-7a74-44c4-91b1-41c218ecd6c9","Type":"ContainerDied","Data":"82b1fd72b7bf9f159235f75e39bbe8d94b9bcd1c0c1779c83af60c8aa179dbc7"} Jul 22 03:37:32.375203 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:32.375159 2556 scope.go:117] "RemoveContainer" containerID="8c5e804000c0391f6d4d1924681866329e44dbaf492d7b693e2dfc0e401b94f6" Jul 22 03:37:32.384552 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:32.384524 2556 scope.go:117] "RemoveContainer" containerID="3892fec4837a084e9c3101aa70c9242ca72d76a65e2b16e1defa096109349212" Jul 22 03:37:32.393506 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:32.393487 2556 scope.go:117] "RemoveContainer" containerID="8c5e804000c0391f6d4d1924681866329e44dbaf492d7b693e2dfc0e401b94f6" Jul 22 03:37:32.393780 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:37:32.393754 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c5e804000c0391f6d4d1924681866329e44dbaf492d7b693e2dfc0e401b94f6\": container with ID starting with 8c5e804000c0391f6d4d1924681866329e44dbaf492d7b693e2dfc0e401b94f6 not found: ID does not exist" containerID="8c5e804000c0391f6d4d1924681866329e44dbaf492d7b693e2dfc0e401b94f6" Jul 22 03:37:32.393886 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:32.393791 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c5e804000c0391f6d4d1924681866329e44dbaf492d7b693e2dfc0e401b94f6"} err="failed to get container status \"8c5e804000c0391f6d4d1924681866329e44dbaf492d7b693e2dfc0e401b94f6\": rpc error: code = NotFound desc = could not find container \"8c5e804000c0391f6d4d1924681866329e44dbaf492d7b693e2dfc0e401b94f6\": container with ID starting with 8c5e804000c0391f6d4d1924681866329e44dbaf492d7b693e2dfc0e401b94f6 not found: ID does not exist" Jul 22 03:37:32.393886 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:32.393814 2556 scope.go:117] "RemoveContainer" containerID="3892fec4837a084e9c3101aa70c9242ca72d76a65e2b16e1defa096109349212" Jul 22 03:37:32.394226 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:37:32.394171 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3892fec4837a084e9c3101aa70c9242ca72d76a65e2b16e1defa096109349212\": container with ID starting with 3892fec4837a084e9c3101aa70c9242ca72d76a65e2b16e1defa096109349212 not found: ID does not exist" containerID="3892fec4837a084e9c3101aa70c9242ca72d76a65e2b16e1defa096109349212" Jul 22 03:37:32.394226 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:32.394202 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3892fec4837a084e9c3101aa70c9242ca72d76a65e2b16e1defa096109349212"} err="failed to get container status \"3892fec4837a084e9c3101aa70c9242ca72d76a65e2b16e1defa096109349212\": rpc error: code = NotFound desc = could not find container \"3892fec4837a084e9c3101aa70c9242ca72d76a65e2b16e1defa096109349212\": container with ID starting with 3892fec4837a084e9c3101aa70c9242ca72d76a65e2b16e1defa096109349212 not found: ID does not exist" Jul 22 03:37:32.396930 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:32.396909 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc"] Jul 22 03:37:32.400369 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:32.400348 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-8cd3c6d0-openshift-default-5755fd59df-v6cxc"] Jul 22 03:37:34.024428 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:34.024395 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2cb1e23-7a74-44c4-91b1-41c218ecd6c9" path="/var/lib/kubelet/pods/e2cb1e23-7a74-44c4-91b1-41c218ecd6c9/volumes" Jul 22 03:37:35.434853 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:35.434818 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:37:40.214762 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:40.214725 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscf8_a01dfba4-f590-412a-8438-1646c51bb1a1/ovn-acl-logging/0.log" Jul 22 03:37:40.217907 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:40.217881 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscf8_a01dfba4-f590-412a-8438-1646c51bb1a1/ovn-acl-logging/0.log" Jul 22 03:37:43.438541 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:43.438457 2556 generic.go:356] "Generic (PLEG): container finished" podID="3f5d0778-0ee2-454c-932c-ff9276ef6c86" containerID="227905fd70e55b9e1291d2d57b4494c1b62f34a5c368fdd351405191562d9a33" exitCode=0 Jul 22 03:37:43.438541 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:43.438504 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" event={"ID":"3f5d0778-0ee2-454c-932c-ff9276ef6c86","Type":"ContainerDied","Data":"227905fd70e55b9e1291d2d57b4494c1b62f34a5c368fdd351405191562d9a33"} Jul 22 03:37:43.439004 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:43.438838 2556 scope.go:117] "RemoveContainer" containerID="227905fd70e55b9e1291d2d57b4494c1b62f34a5c368fdd351405191562d9a33" Jul 22 03:37:43.439772 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:43.439744 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:37:43.439873 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:43.439814 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:37:43.439873 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:43.439847 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:37:44.356866 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:44.356828 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:37:44.446903 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:44.446861 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" event={"ID":"3f5d0778-0ee2-454c-932c-ff9276ef6c86","Type":"ContainerStarted","Data":"50ae960ca3a4d243bfbda021c882267aaeaea6b946e92824461e84d9c63fb774"} Jul 22 03:37:44.651425 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:44.651326 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:37:44.652457 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:44.652427 2556 patch_prober.go:28] interesting pod/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 22 03:37:44.652598 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:44.652479 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" podUID="3f5d0778-0ee2-454c-932c-ff9276ef6c86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 22 03:37:45.650849 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:45.650814 2556 patch_prober.go:28] interesting pod/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 22 03:37:45.651259 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:45.650865 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" podUID="3f5d0778-0ee2-454c-932c-ff9276ef6c86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 22 03:37:46.650750 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:46.650711 2556 patch_prober.go:28] interesting pod/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 22 03:37:46.650947 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:46.650769 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" podUID="3f5d0778-0ee2-454c-932c-ff9276ef6c86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 22 03:37:47.651400 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:47.651359 2556 patch_prober.go:28] interesting pod/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 22 03:37:47.651793 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:47.651413 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" podUID="3f5d0778-0ee2-454c-932c-ff9276ef6c86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 22 03:37:48.654648 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:48.654620 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:37:48.655039 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:48.654864 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:37:48.655516 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:37:48.655501 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:38:05.120455 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:05.120419 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:38:08.043511 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:08.043471 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:38:09.708389 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:09.708357 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:38:09.856377 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:09.856343 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267"] Jul 22 03:38:09.856573 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:09.856547 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" podUID="3f5d0778-0ee2-454c-932c-ff9276ef6c86" containerName="istio-proxy" containerID="cri-o://50ae960ca3a4d243bfbda021c882267aaeaea6b946e92824461e84d9c63fb774" gracePeriod=30 Jul 22 03:38:09.889691 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:09.889660 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:38:10.094063 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:10.094028 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:38:10.562420 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:10.562324 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:38:10.660555 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:10.660523 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:38:10.759713 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:10.759683 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:38:12.147717 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:12.147682 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:38:15.110289 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.110261 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:38:15.121484 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.121438 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f5d0778-0ee2-454c-932c-ff9276ef6c86-istiod-ca-cert\") pod \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\" (UID: \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\") " Jul 22 03:38:15.121484 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.121475 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f5d0778-0ee2-454c-932c-ff9276ef6c86-istio-data\") pod \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\" (UID: \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\") " Jul 22 03:38:15.121605 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.121504 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f5d0778-0ee2-454c-932c-ff9276ef6c86-workload-socket\") pod \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\" (UID: \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\") " Jul 22 03:38:15.121605 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.121545 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7c46q\" (UniqueName: \"kubernetes.io/projected/3f5d0778-0ee2-454c-932c-ff9276ef6c86-kube-api-access-7c46q\") pod \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\" (UID: \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\") " Jul 22 03:38:15.121605 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.121583 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f5d0778-0ee2-454c-932c-ff9276ef6c86-workload-certs\") pod \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\" (UID: \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\") " Jul 22 03:38:15.121761 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.121625 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f5d0778-0ee2-454c-932c-ff9276ef6c86-istio-token\") pod \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\" (UID: \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\") " Jul 22 03:38:15.121761 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.121665 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f5d0778-0ee2-454c-932c-ff9276ef6c86-credential-socket\") pod \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\" (UID: \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\") " Jul 22 03:38:15.121873 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.121847 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f5d0778-0ee2-454c-932c-ff9276ef6c86-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3f5d0778-0ee2-454c-932c-ff9276ef6c86" (UID: "3f5d0778-0ee2-454c-932c-ff9276ef6c86"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:38:15.122067 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.121989 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f5d0778-0ee2-454c-932c-ff9276ef6c86-istiod-ca-cert\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:38:15.122067 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.122003 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f5d0778-0ee2-454c-932c-ff9276ef6c86-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3f5d0778-0ee2-454c-932c-ff9276ef6c86" (UID: "3f5d0778-0ee2-454c-932c-ff9276ef6c86"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:38:15.122198 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.122109 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f5d0778-0ee2-454c-932c-ff9276ef6c86-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3f5d0778-0ee2-454c-932c-ff9276ef6c86" (UID: "3f5d0778-0ee2-454c-932c-ff9276ef6c86"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:38:15.122198 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.122121 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f5d0778-0ee2-454c-932c-ff9276ef6c86-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3f5d0778-0ee2-454c-932c-ff9276ef6c86" (UID: "3f5d0778-0ee2-454c-932c-ff9276ef6c86"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:38:15.122452 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.122429 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f5d0778-0ee2-454c-932c-ff9276ef6c86-istio-data" (OuterVolumeSpecName: "istio-data") pod "3f5d0778-0ee2-454c-932c-ff9276ef6c86" (UID: "3f5d0778-0ee2-454c-932c-ff9276ef6c86"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:38:15.123786 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.123764 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f5d0778-0ee2-454c-932c-ff9276ef6c86-kube-api-access-7c46q" (OuterVolumeSpecName: "kube-api-access-7c46q") pod "3f5d0778-0ee2-454c-932c-ff9276ef6c86" (UID: "3f5d0778-0ee2-454c-932c-ff9276ef6c86"). InnerVolumeSpecName "kube-api-access-7c46q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:38:15.123859 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.123813 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f5d0778-0ee2-454c-932c-ff9276ef6c86-istio-token" (OuterVolumeSpecName: "istio-token") pod "3f5d0778-0ee2-454c-932c-ff9276ef6c86" (UID: "3f5d0778-0ee2-454c-932c-ff9276ef6c86"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:38:15.222869 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.222835 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f5d0778-0ee2-454c-932c-ff9276ef6c86-istio-podinfo\") pod \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\" (UID: \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\") " Jul 22 03:38:15.223062 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.223000 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f5d0778-0ee2-454c-932c-ff9276ef6c86-istio-envoy\") pod \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\" (UID: \"3f5d0778-0ee2-454c-932c-ff9276ef6c86\") " Jul 22 03:38:15.223263 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.223247 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f5d0778-0ee2-454c-932c-ff9276ef6c86-workload-certs\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:38:15.223333 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.223270 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f5d0778-0ee2-454c-932c-ff9276ef6c86-istio-token\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:38:15.223333 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.223283 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f5d0778-0ee2-454c-932c-ff9276ef6c86-credential-socket\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:38:15.223333 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.223296 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f5d0778-0ee2-454c-932c-ff9276ef6c86-istio-data\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:38:15.223333 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.223309 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f5d0778-0ee2-454c-932c-ff9276ef6c86-workload-socket\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:38:15.223333 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.223325 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7c46q\" (UniqueName: \"kubernetes.io/projected/3f5d0778-0ee2-454c-932c-ff9276ef6c86-kube-api-access-7c46q\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:38:15.224893 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.224862 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3f5d0778-0ee2-454c-932c-ff9276ef6c86-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3f5d0778-0ee2-454c-932c-ff9276ef6c86" (UID: "3f5d0778-0ee2-454c-932c-ff9276ef6c86"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 03:38:15.225035 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.224940 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f5d0778-0ee2-454c-932c-ff9276ef6c86-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3f5d0778-0ee2-454c-932c-ff9276ef6c86" (UID: "3f5d0778-0ee2-454c-932c-ff9276ef6c86"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:38:15.324166 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.324129 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f5d0778-0ee2-454c-932c-ff9276ef6c86-istio-envoy\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:38:15.324166 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.324157 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f5d0778-0ee2-454c-932c-ff9276ef6c86-istio-podinfo\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:38:15.627167 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.627135 2556 generic.go:356] "Generic (PLEG): container finished" podID="3f5d0778-0ee2-454c-932c-ff9276ef6c86" containerID="50ae960ca3a4d243bfbda021c882267aaeaea6b946e92824461e84d9c63fb774" exitCode=0 Jul 22 03:38:15.627359 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.627182 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" Jul 22 03:38:15.627359 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.627230 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" event={"ID":"3f5d0778-0ee2-454c-932c-ff9276ef6c86","Type":"ContainerDied","Data":"50ae960ca3a4d243bfbda021c882267aaeaea6b946e92824461e84d9c63fb774"} Jul 22 03:38:15.627359 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.627322 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267" event={"ID":"3f5d0778-0ee2-454c-932c-ff9276ef6c86","Type":"ContainerDied","Data":"c865998ecc52574c9073c207c359806d1d63b149655def4c57d76181d7047287"} Jul 22 03:38:15.627359 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.627350 2556 scope.go:117] "RemoveContainer" containerID="50ae960ca3a4d243bfbda021c882267aaeaea6b946e92824461e84d9c63fb774" Jul 22 03:38:15.637130 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.637112 2556 scope.go:117] "RemoveContainer" containerID="227905fd70e55b9e1291d2d57b4494c1b62f34a5c368fdd351405191562d9a33" Jul 22 03:38:15.644988 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.644971 2556 scope.go:117] "RemoveContainer" containerID="50ae960ca3a4d243bfbda021c882267aaeaea6b946e92824461e84d9c63fb774" Jul 22 03:38:15.645219 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:38:15.645201 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50ae960ca3a4d243bfbda021c882267aaeaea6b946e92824461e84d9c63fb774\": container with ID starting with 50ae960ca3a4d243bfbda021c882267aaeaea6b946e92824461e84d9c63fb774 not found: ID does not exist" containerID="50ae960ca3a4d243bfbda021c882267aaeaea6b946e92824461e84d9c63fb774" Jul 22 03:38:15.645280 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.645226 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50ae960ca3a4d243bfbda021c882267aaeaea6b946e92824461e84d9c63fb774"} err="failed to get container status \"50ae960ca3a4d243bfbda021c882267aaeaea6b946e92824461e84d9c63fb774\": rpc error: code = NotFound desc = could not find container \"50ae960ca3a4d243bfbda021c882267aaeaea6b946e92824461e84d9c63fb774\": container with ID starting with 50ae960ca3a4d243bfbda021c882267aaeaea6b946e92824461e84d9c63fb774 not found: ID does not exist" Jul 22 03:38:15.645280 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.645242 2556 scope.go:117] "RemoveContainer" containerID="227905fd70e55b9e1291d2d57b4494c1b62f34a5c368fdd351405191562d9a33" Jul 22 03:38:15.645451 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:38:15.645432 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"227905fd70e55b9e1291d2d57b4494c1b62f34a5c368fdd351405191562d9a33\": container with ID starting with 227905fd70e55b9e1291d2d57b4494c1b62f34a5c368fdd351405191562d9a33 not found: ID does not exist" containerID="227905fd70e55b9e1291d2d57b4494c1b62f34a5c368fdd351405191562d9a33" Jul 22 03:38:15.645500 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.645459 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"227905fd70e55b9e1291d2d57b4494c1b62f34a5c368fdd351405191562d9a33"} err="failed to get container status \"227905fd70e55b9e1291d2d57b4494c1b62f34a5c368fdd351405191562d9a33\": rpc error: code = NotFound desc = could not find container \"227905fd70e55b9e1291d2d57b4494c1b62f34a5c368fdd351405191562d9a33\": container with ID starting with 227905fd70e55b9e1291d2d57b4494c1b62f34a5c368fdd351405191562d9a33 not found: ID does not exist" Jul 22 03:38:15.659692 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.659670 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267"] Jul 22 03:38:15.671059 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:15.671036 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-1535280c-openshift-default-5bf5fd9b75-k5267"] Jul 22 03:38:16.024432 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:16.024359 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f5d0778-0ee2-454c-932c-ff9276ef6c86" path="/var/lib/kubelet/pods/3f5d0778-0ee2-454c-932c-ff9276ef6c86/volumes" Jul 22 03:38:17.983170 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:17.983120 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:38:18.085216 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:18.085184 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:38:18.187154 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:18.187117 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:38:18.287371 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:18.287263 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:38:18.688454 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:18.688418 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:38:44.629314 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:44.629277 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:38:51.026786 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:51.026747 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:38:51.150974 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:51.150916 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:38:51.263816 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:38:51.263778 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:39:07.099896 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:07.099822 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:39:13.029619 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:13.029578 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:39:13.135235 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:13.135200 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:39:13.229610 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:13.229582 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:39:23.746708 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:23.746647 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:39:24.547332 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:24.547295 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:39:31.590740 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:31.590702 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:39:31.693253 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:31.693221 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:39:31.888480 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:31.888404 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:39:31.997227 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:31.997199 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:39:32.671306 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:32.671270 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:39:32.763781 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:32.763749 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:39:32.880181 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:32.880130 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:39:34.337507 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:34.337470 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:39:34.399720 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:34.399687 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:39:34.470487 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:34.470449 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:39:35.429800 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:35.429765 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:39:40.497263 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:40.497223 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:39:40.596035 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:40.596000 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:39:41.156757 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:41.156718 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:39:45.333722 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:45.333689 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:39:48.269101 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:48.269070 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:39:48.798795 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:48.798761 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct"] Jul 22 03:39:48.799184 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:48.799169 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f5d0778-0ee2-454c-932c-ff9276ef6c86" containerName="istio-proxy" Jul 22 03:39:48.799238 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:48.799185 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f5d0778-0ee2-454c-932c-ff9276ef6c86" containerName="istio-proxy" Jul 22 03:39:48.799238 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:48.799199 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2cb1e23-7a74-44c4-91b1-41c218ecd6c9" containerName="istio-proxy" Jul 22 03:39:48.799238 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:48.799205 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2cb1e23-7a74-44c4-91b1-41c218ecd6c9" containerName="istio-proxy" Jul 22 03:39:48.799238 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:48.799224 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f5d0778-0ee2-454c-932c-ff9276ef6c86" containerName="istio-proxy" Jul 22 03:39:48.799238 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:48.799229 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f5d0778-0ee2-454c-932c-ff9276ef6c86" containerName="istio-proxy" Jul 22 03:39:48.799394 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:48.799282 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="22c8ae91-60df-42c3-8378-b6b6f2d0cd48" containerName="istio-proxy" Jul 22 03:39:48.799394 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:48.799292 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f5d0778-0ee2-454c-932c-ff9276ef6c86" containerName="istio-proxy" Jul 22 03:39:48.799394 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:48.799298 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2cb1e23-7a74-44c4-91b1-41c218ecd6c9" containerName="istio-proxy" Jul 22 03:39:48.799394 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:48.799302 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f5d0778-0ee2-454c-932c-ff9276ef6c86" containerName="istio-proxy" Jul 22 03:39:48.802465 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:48.802445 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:39:48.805467 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:48.805447 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-1c3450ee-openshift-default-dockercfg-88wb5\"" Jul 22 03:39:48.824474 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:48.824449 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct"] Jul 22 03:39:48.945075 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:48.945036 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e0de92f-00f4-4777-9603-2d453584964e-istio-podinfo\") pod \"e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct\" (UID: \"7e0de92f-00f4-4777-9603-2d453584964e\") " pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:39:48.945246 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:48.945089 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e0de92f-00f4-4777-9603-2d453584964e-workload-socket\") pod \"e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct\" (UID: \"7e0de92f-00f4-4777-9603-2d453584964e\") " pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:39:48.945246 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:48.945190 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e0de92f-00f4-4777-9603-2d453584964e-istio-envoy\") pod \"e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct\" (UID: \"7e0de92f-00f4-4777-9603-2d453584964e\") " pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:39:48.945246 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:48.945233 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e0de92f-00f4-4777-9603-2d453584964e-workload-certs\") pod \"e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct\" (UID: \"7e0de92f-00f4-4777-9603-2d453584964e\") " pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:39:48.945360 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:48.945253 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e0de92f-00f4-4777-9603-2d453584964e-istiod-ca-cert\") pod \"e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct\" (UID: \"7e0de92f-00f4-4777-9603-2d453584964e\") " pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:39:48.945360 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:48.945273 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e0de92f-00f4-4777-9603-2d453584964e-credential-socket\") pod \"e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct\" (UID: \"7e0de92f-00f4-4777-9603-2d453584964e\") " pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:39:48.945360 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:48.945306 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e0de92f-00f4-4777-9603-2d453584964e-istio-data\") pod \"e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct\" (UID: \"7e0de92f-00f4-4777-9603-2d453584964e\") " pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:39:48.945460 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:48.945368 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxpdb\" (UniqueName: \"kubernetes.io/projected/7e0de92f-00f4-4777-9603-2d453584964e-kube-api-access-qxpdb\") pod \"e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct\" (UID: \"7e0de92f-00f4-4777-9603-2d453584964e\") " pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:39:48.945460 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:48.945391 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e0de92f-00f4-4777-9603-2d453584964e-istio-token\") pod \"e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct\" (UID: \"7e0de92f-00f4-4777-9603-2d453584964e\") " pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:39:49.045790 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:49.045755 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e0de92f-00f4-4777-9603-2d453584964e-workload-socket\") pod \"e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct\" (UID: \"7e0de92f-00f4-4777-9603-2d453584964e\") " pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:39:49.045790 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:49.045795 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e0de92f-00f4-4777-9603-2d453584964e-istio-envoy\") pod \"e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct\" (UID: \"7e0de92f-00f4-4777-9603-2d453584964e\") " pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:39:49.046054 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:49.045827 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e0de92f-00f4-4777-9603-2d453584964e-workload-certs\") pod \"e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct\" (UID: \"7e0de92f-00f4-4777-9603-2d453584964e\") " pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:39:49.046054 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:49.045850 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e0de92f-00f4-4777-9603-2d453584964e-istiod-ca-cert\") pod \"e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct\" (UID: \"7e0de92f-00f4-4777-9603-2d453584964e\") " pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:39:49.046054 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:49.045875 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e0de92f-00f4-4777-9603-2d453584964e-credential-socket\") pod \"e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct\" (UID: \"7e0de92f-00f4-4777-9603-2d453584964e\") " pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:39:49.046054 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:49.045915 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e0de92f-00f4-4777-9603-2d453584964e-istio-data\") pod \"e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct\" (UID: \"7e0de92f-00f4-4777-9603-2d453584964e\") " pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:39:49.046054 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:49.045986 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxpdb\" (UniqueName: \"kubernetes.io/projected/7e0de92f-00f4-4777-9603-2d453584964e-kube-api-access-qxpdb\") pod \"e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct\" (UID: \"7e0de92f-00f4-4777-9603-2d453584964e\") " pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:39:49.046054 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:49.046018 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e0de92f-00f4-4777-9603-2d453584964e-istio-token\") pod \"e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct\" (UID: \"7e0de92f-00f4-4777-9603-2d453584964e\") " pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:39:49.046349 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:49.046058 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e0de92f-00f4-4777-9603-2d453584964e-istio-podinfo\") pod \"e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct\" (UID: \"7e0de92f-00f4-4777-9603-2d453584964e\") " pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:39:49.046349 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:49.046169 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e0de92f-00f4-4777-9603-2d453584964e-workload-socket\") pod \"e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct\" (UID: \"7e0de92f-00f4-4777-9603-2d453584964e\") " pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:39:49.046550 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:49.046496 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e0de92f-00f4-4777-9603-2d453584964e-istiod-ca-cert\") pod \"e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct\" (UID: \"7e0de92f-00f4-4777-9603-2d453584964e\") " pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:39:49.046671 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:49.046631 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e0de92f-00f4-4777-9603-2d453584964e-workload-certs\") pod \"e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct\" (UID: \"7e0de92f-00f4-4777-9603-2d453584964e\") " pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:39:49.046723 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:49.046667 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e0de92f-00f4-4777-9603-2d453584964e-credential-socket\") pod \"e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct\" (UID: \"7e0de92f-00f4-4777-9603-2d453584964e\") " pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:39:49.046723 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:49.046690 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e0de92f-00f4-4777-9603-2d453584964e-istio-data\") pod \"e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct\" (UID: \"7e0de92f-00f4-4777-9603-2d453584964e\") " pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:39:49.048584 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:49.048562 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e0de92f-00f4-4777-9603-2d453584964e-istio-envoy\") pod \"e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct\" (UID: \"7e0de92f-00f4-4777-9603-2d453584964e\") " pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:39:49.048780 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:49.048763 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e0de92f-00f4-4777-9603-2d453584964e-istio-podinfo\") pod \"e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct\" (UID: \"7e0de92f-00f4-4777-9603-2d453584964e\") " pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:39:49.054172 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:49.054120 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e0de92f-00f4-4777-9603-2d453584964e-istio-token\") pod \"e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct\" (UID: \"7e0de92f-00f4-4777-9603-2d453584964e\") " pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:39:49.054391 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:49.054370 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxpdb\" (UniqueName: \"kubernetes.io/projected/7e0de92f-00f4-4777-9603-2d453584964e-kube-api-access-qxpdb\") pod \"e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct\" (UID: \"7e0de92f-00f4-4777-9603-2d453584964e\") " pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:39:49.113413 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:49.113383 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:39:49.255352 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:49.255322 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct"] Jul 22 03:39:49.257024 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:39:49.256995 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e0de92f_00f4_4777_9603_2d453584964e.slice/crio-4555a743f2e7fc811aeb61232e3d3a2afda7db2271505f9ae2ad042afa003028 WatchSource:0}: Error finding container 4555a743f2e7fc811aeb61232e3d3a2afda7db2271505f9ae2ad042afa003028: Status 404 returned error can't find the container with id 4555a743f2e7fc811aeb61232e3d3a2afda7db2271505f9ae2ad042afa003028 Jul 22 03:39:49.259191 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:49.259150 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:39:49.259294 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:49.259236 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:39:49.259336 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:49.259286 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:39:49.496828 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:49.496796 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:39:50.132539 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:50.132503 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" event={"ID":"7e0de92f-00f4-4777-9603-2d453584964e","Type":"ContainerStarted","Data":"9edbd9c2ae1c10f7d8cb5c0d02200eef7a6887f23a01b66df1d402b1d54ce875"} Jul 22 03:39:50.132737 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:50.132562 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" event={"ID":"7e0de92f-00f4-4777-9603-2d453584964e","Type":"ContainerStarted","Data":"4555a743f2e7fc811aeb61232e3d3a2afda7db2271505f9ae2ad042afa003028"} Jul 22 03:39:50.158575 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:50.158522 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" podStartSLOduration=2.15850445 podStartE2EDuration="2.15850445s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:39:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:39:50.155689267 +0000 UTC m=+1930.604672537" watchObservedRunningTime="2026-07-22 03:39:50.15850445 +0000 UTC m=+1930.607487701" Jul 22 03:39:51.114484 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:51.114451 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:39:51.119842 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:51.119819 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:39:51.137675 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:51.137640 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:39:51.138543 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:51.138523 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:39:52.092068 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:52.092035 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:39:52.182212 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:52.182181 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:39:52.264914 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:52.264881 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:39:52.552223 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:52.552144 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:39:52.920805 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:52.920771 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:39:53.025216 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:53.025188 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:39:56.544305 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:56.544271 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:39:57.183650 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:57.183615 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:39:57.292847 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:57.292813 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:39:57.630249 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:57.630213 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:39:58.036491 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:39:58.036415 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:40:03.561927 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:03.561895 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:40:03.662114 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:03.662078 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:40:06.139162 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:06.139075 2556 patch_prober.go:28] interesting pod/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 22 03:40:06.139631 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:06.139419 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" podUID="7e0de92f-00f4-4777-9603-2d453584964e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 22 03:40:06.230303 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:06.230270 2556 generic.go:356] "Generic (PLEG): container finished" podID="7e0de92f-00f4-4777-9603-2d453584964e" containerID="9edbd9c2ae1c10f7d8cb5c0d02200eef7a6887f23a01b66df1d402b1d54ce875" exitCode=0 Jul 22 03:40:06.230478 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:06.230320 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" event={"ID":"7e0de92f-00f4-4777-9603-2d453584964e","Type":"ContainerDied","Data":"9edbd9c2ae1c10f7d8cb5c0d02200eef7a6887f23a01b66df1d402b1d54ce875"} Jul 22 03:40:06.230721 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:06.230707 2556 scope.go:117] "RemoveContainer" containerID="9edbd9c2ae1c10f7d8cb5c0d02200eef7a6887f23a01b66df1d402b1d54ce875" Jul 22 03:40:06.231723 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:06.231690 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:40:06.231845 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:06.231759 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:40:06.231845 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:06.231794 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:40:07.238044 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:07.238008 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" event={"ID":"7e0de92f-00f4-4777-9603-2d453584964e","Type":"ContainerStarted","Data":"efce981f376912053014f131abc5a70b818df1ee9615a0fc035937edefa01b1f"} Jul 22 03:40:08.113738 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:08.113702 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:40:08.113932 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:08.113912 2556 patch_prober.go:28] interesting pod/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 22 03:40:08.114008 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:08.113986 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" podUID="7e0de92f-00f4-4777-9603-2d453584964e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 22 03:40:09.113977 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:09.113917 2556 patch_prober.go:28] interesting pod/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 22 03:40:09.114389 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:09.114005 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" podUID="7e0de92f-00f4-4777-9603-2d453584964e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 22 03:40:10.114665 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:10.114629 2556 patch_prober.go:28] interesting pod/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 22 03:40:10.115076 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:10.114686 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" podUID="7e0de92f-00f4-4777-9603-2d453584964e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 22 03:40:11.114596 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:11.114548 2556 patch_prober.go:28] interesting pod/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 22 03:40:11.114777 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:11.114622 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" podUID="7e0de92f-00f4-4777-9603-2d453584964e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 22 03:40:12.118269 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:12.118239 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:40:12.118661 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:12.118473 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:40:12.119100 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:12.119078 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:40:14.531126 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:14.531039 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:40:17.334626 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:17.334586 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:40:18.987734 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:18.987647 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:40:19.365773 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:19.365729 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:40:25.460943 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:25.460904 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:40:25.531508 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:25.531467 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:40:25.709040 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:25.709005 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct"] Jul 22 03:40:25.709291 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:25.709265 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" podUID="7e0de92f-00f4-4777-9603-2d453584964e" containerName="istio-proxy" containerID="cri-o://efce981f376912053014f131abc5a70b818df1ee9615a0fc035937edefa01b1f" gracePeriod=30 Jul 22 03:40:25.731377 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:25.731269 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:40:25.942127 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:25.942083 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:40:26.067009 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:26.066892 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:40:26.164105 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:26.164025 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:40:26.293194 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:26.293160 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:40:27.121010 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:27.120973 2556 patch_prober.go:28] interesting pod/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused Jul 22 03:40:27.121378 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:27.121037 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" podUID="7e0de92f-00f4-4777-9603-2d453584964e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 03:40:30.963156 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:30.963131 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:40:31.126767 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.126736 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e0de92f-00f4-4777-9603-2d453584964e-istio-envoy\") pod \"7e0de92f-00f4-4777-9603-2d453584964e\" (UID: \"7e0de92f-00f4-4777-9603-2d453584964e\") " Jul 22 03:40:31.126989 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.126783 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e0de92f-00f4-4777-9603-2d453584964e-workload-socket\") pod \"7e0de92f-00f4-4777-9603-2d453584964e\" (UID: \"7e0de92f-00f4-4777-9603-2d453584964e\") " Jul 22 03:40:31.126989 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.126799 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e0de92f-00f4-4777-9603-2d453584964e-istio-data\") pod \"7e0de92f-00f4-4777-9603-2d453584964e\" (UID: \"7e0de92f-00f4-4777-9603-2d453584964e\") " Jul 22 03:40:31.126989 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.126841 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e0de92f-00f4-4777-9603-2d453584964e-workload-certs\") pod \"7e0de92f-00f4-4777-9603-2d453584964e\" (UID: \"7e0de92f-00f4-4777-9603-2d453584964e\") " Jul 22 03:40:31.126989 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.126857 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e0de92f-00f4-4777-9603-2d453584964e-istio-token\") pod \"7e0de92f-00f4-4777-9603-2d453584964e\" (UID: \"7e0de92f-00f4-4777-9603-2d453584964e\") " Jul 22 03:40:31.126989 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.126881 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e0de92f-00f4-4777-9603-2d453584964e-istiod-ca-cert\") pod \"7e0de92f-00f4-4777-9603-2d453584964e\" (UID: \"7e0de92f-00f4-4777-9603-2d453584964e\") " Jul 22 03:40:31.126989 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.126907 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxpdb\" (UniqueName: \"kubernetes.io/projected/7e0de92f-00f4-4777-9603-2d453584964e-kube-api-access-qxpdb\") pod \"7e0de92f-00f4-4777-9603-2d453584964e\" (UID: \"7e0de92f-00f4-4777-9603-2d453584964e\") " Jul 22 03:40:31.126989 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.126943 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e0de92f-00f4-4777-9603-2d453584964e-credential-socket\") pod \"7e0de92f-00f4-4777-9603-2d453584964e\" (UID: \"7e0de92f-00f4-4777-9603-2d453584964e\") " Jul 22 03:40:31.127360 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.127056 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e0de92f-00f4-4777-9603-2d453584964e-istio-podinfo\") pod \"7e0de92f-00f4-4777-9603-2d453584964e\" (UID: \"7e0de92f-00f4-4777-9603-2d453584964e\") " Jul 22 03:40:31.127360 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.127153 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e0de92f-00f4-4777-9603-2d453584964e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7e0de92f-00f4-4777-9603-2d453584964e" (UID: "7e0de92f-00f4-4777-9603-2d453584964e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:40:31.127360 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.127179 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e0de92f-00f4-4777-9603-2d453584964e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7e0de92f-00f4-4777-9603-2d453584964e" (UID: "7e0de92f-00f4-4777-9603-2d453584964e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:40:31.127360 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.127340 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e0de92f-00f4-4777-9603-2d453584964e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7e0de92f-00f4-4777-9603-2d453584964e" (UID: "7e0de92f-00f4-4777-9603-2d453584964e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:40:31.127558 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.127410 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e0de92f-00f4-4777-9603-2d453584964e-workload-socket\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:40:31.127558 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.127429 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e0de92f-00f4-4777-9603-2d453584964e-workload-certs\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:40:31.127558 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.127541 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e0de92f-00f4-4777-9603-2d453584964e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7e0de92f-00f4-4777-9603-2d453584964e" (UID: "7e0de92f-00f4-4777-9603-2d453584964e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:40:31.128070 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.128048 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e0de92f-00f4-4777-9603-2d453584964e-istio-data" (OuterVolumeSpecName: "istio-data") pod "7e0de92f-00f4-4777-9603-2d453584964e" (UID: "7e0de92f-00f4-4777-9603-2d453584964e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:40:31.129531 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.129507 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e0de92f-00f4-4777-9603-2d453584964e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7e0de92f-00f4-4777-9603-2d453584964e" (UID: "7e0de92f-00f4-4777-9603-2d453584964e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:40:31.129644 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.129537 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e0de92f-00f4-4777-9603-2d453584964e-kube-api-access-qxpdb" (OuterVolumeSpecName: "kube-api-access-qxpdb") pod "7e0de92f-00f4-4777-9603-2d453584964e" (UID: "7e0de92f-00f4-4777-9603-2d453584964e"). InnerVolumeSpecName "kube-api-access-qxpdb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:40:31.129718 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.129702 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7e0de92f-00f4-4777-9603-2d453584964e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7e0de92f-00f4-4777-9603-2d453584964e" (UID: "7e0de92f-00f4-4777-9603-2d453584964e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 03:40:31.129774 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.129717 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e0de92f-00f4-4777-9603-2d453584964e-istio-token" (OuterVolumeSpecName: "istio-token") pod "7e0de92f-00f4-4777-9603-2d453584964e" (UID: "7e0de92f-00f4-4777-9603-2d453584964e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:40:31.228419 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.228377 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e0de92f-00f4-4777-9603-2d453584964e-istio-podinfo\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:40:31.228419 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.228411 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e0de92f-00f4-4777-9603-2d453584964e-istio-envoy\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:40:31.228419 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.228420 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e0de92f-00f4-4777-9603-2d453584964e-istio-data\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:40:31.228419 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.228430 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e0de92f-00f4-4777-9603-2d453584964e-istio-token\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:40:31.228707 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.228438 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e0de92f-00f4-4777-9603-2d453584964e-istiod-ca-cert\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:40:31.228707 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.228446 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxpdb\" (UniqueName: \"kubernetes.io/projected/7e0de92f-00f4-4777-9603-2d453584964e-kube-api-access-qxpdb\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:40:31.228707 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.228455 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e0de92f-00f4-4777-9603-2d453584964e-credential-socket\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:40:31.369878 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.369844 2556 generic.go:356] "Generic (PLEG): container finished" podID="7e0de92f-00f4-4777-9603-2d453584964e" containerID="efce981f376912053014f131abc5a70b818df1ee9615a0fc035937edefa01b1f" exitCode=0 Jul 22 03:40:31.370096 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.369894 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" Jul 22 03:40:31.370096 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.369931 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" event={"ID":"7e0de92f-00f4-4777-9603-2d453584964e","Type":"ContainerDied","Data":"efce981f376912053014f131abc5a70b818df1ee9615a0fc035937edefa01b1f"} Jul 22 03:40:31.370096 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.370014 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct" event={"ID":"7e0de92f-00f4-4777-9603-2d453584964e","Type":"ContainerDied","Data":"4555a743f2e7fc811aeb61232e3d3a2afda7db2271505f9ae2ad042afa003028"} Jul 22 03:40:31.370096 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.370045 2556 scope.go:117] "RemoveContainer" containerID="efce981f376912053014f131abc5a70b818df1ee9615a0fc035937edefa01b1f" Jul 22 03:40:31.379568 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.379548 2556 scope.go:117] "RemoveContainer" containerID="9edbd9c2ae1c10f7d8cb5c0d02200eef7a6887f23a01b66df1d402b1d54ce875" Jul 22 03:40:31.388005 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.387986 2556 scope.go:117] "RemoveContainer" containerID="efce981f376912053014f131abc5a70b818df1ee9615a0fc035937edefa01b1f" Jul 22 03:40:31.388255 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:40:31.388234 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efce981f376912053014f131abc5a70b818df1ee9615a0fc035937edefa01b1f\": container with ID starting with efce981f376912053014f131abc5a70b818df1ee9615a0fc035937edefa01b1f not found: ID does not exist" containerID="efce981f376912053014f131abc5a70b818df1ee9615a0fc035937edefa01b1f" Jul 22 03:40:31.388321 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.388263 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efce981f376912053014f131abc5a70b818df1ee9615a0fc035937edefa01b1f"} err="failed to get container status \"efce981f376912053014f131abc5a70b818df1ee9615a0fc035937edefa01b1f\": rpc error: code = NotFound desc = could not find container \"efce981f376912053014f131abc5a70b818df1ee9615a0fc035937edefa01b1f\": container with ID starting with efce981f376912053014f131abc5a70b818df1ee9615a0fc035937edefa01b1f not found: ID does not exist" Jul 22 03:40:31.388321 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.388281 2556 scope.go:117] "RemoveContainer" containerID="9edbd9c2ae1c10f7d8cb5c0d02200eef7a6887f23a01b66df1d402b1d54ce875" Jul 22 03:40:31.388517 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:40:31.388499 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9edbd9c2ae1c10f7d8cb5c0d02200eef7a6887f23a01b66df1d402b1d54ce875\": container with ID starting with 9edbd9c2ae1c10f7d8cb5c0d02200eef7a6887f23a01b66df1d402b1d54ce875 not found: ID does not exist" containerID="9edbd9c2ae1c10f7d8cb5c0d02200eef7a6887f23a01b66df1d402b1d54ce875" Jul 22 03:40:31.388558 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.388523 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9edbd9c2ae1c10f7d8cb5c0d02200eef7a6887f23a01b66df1d402b1d54ce875"} err="failed to get container status \"9edbd9c2ae1c10f7d8cb5c0d02200eef7a6887f23a01b66df1d402b1d54ce875\": rpc error: code = NotFound desc = could not find container \"9edbd9c2ae1c10f7d8cb5c0d02200eef7a6887f23a01b66df1d402b1d54ce875\": container with ID starting with 9edbd9c2ae1c10f7d8cb5c0d02200eef7a6887f23a01b66df1d402b1d54ce875 not found: ID does not exist" Jul 22 03:40:31.396706 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.396684 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct"] Jul 22 03:40:31.402390 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:31.402370 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-1c3450ee-openshift-default-86f8c54d6c-94dct"] Jul 22 03:40:32.029347 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:32.029312 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e0de92f-00f4-4777-9603-2d453584964e" path="/var/lib/kubelet/pods/7e0de92f-00f4-4777-9603-2d453584964e/volumes" Jul 22 03:40:32.438438 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:32.438407 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:40:32.556189 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:32.556155 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:40:32.647583 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:32.647549 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:40:32.772209 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:32.772131 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:40:38.710848 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:38.710807 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:40:38.774435 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:38.774405 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:40:47.556991 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.556933 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:40:47.836394 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.836354 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l"] Jul 22 03:40:47.836918 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.836902 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e0de92f-00f4-4777-9603-2d453584964e" containerName="istio-proxy" Jul 22 03:40:47.837001 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.836921 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e0de92f-00f4-4777-9603-2d453584964e" containerName="istio-proxy" Jul 22 03:40:47.837001 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.836943 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2cb1e23-7a74-44c4-91b1-41c218ecd6c9" containerName="istio-proxy" Jul 22 03:40:47.837001 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.836966 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2cb1e23-7a74-44c4-91b1-41c218ecd6c9" containerName="istio-proxy" Jul 22 03:40:47.837001 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.836994 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e0de92f-00f4-4777-9603-2d453584964e" containerName="istio-proxy" Jul 22 03:40:47.837132 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.837003 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e0de92f-00f4-4777-9603-2d453584964e" containerName="istio-proxy" Jul 22 03:40:47.837132 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.837095 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e0de92f-00f4-4777-9603-2d453584964e" containerName="istio-proxy" Jul 22 03:40:47.837132 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.837106 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2cb1e23-7a74-44c4-91b1-41c218ecd6c9" containerName="istio-proxy" Jul 22 03:40:47.837132 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.837120 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e0de92f-00f4-4777-9603-2d453584964e" containerName="istio-proxy" Jul 22 03:40:47.840453 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.840432 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:40:47.843359 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.843341 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-513b5fd3-openshift-default-dockercfg-pl72r\"" Jul 22 03:40:47.852674 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.852655 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l"] Jul 22 03:40:47.864231 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.864208 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:40:47.883428 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.883398 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-istio-podinfo\") pod \"e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l\" (UID: \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\") " pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:40:47.883554 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.883432 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-istiod-ca-cert\") pod \"e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l\" (UID: \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\") " pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:40:47.883554 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.883459 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-credential-socket\") pod \"e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l\" (UID: \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\") " pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:40:47.883645 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.883554 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-istio-data\") pod \"e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l\" (UID: \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\") " pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:40:47.883645 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.883590 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-istio-token\") pod \"e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l\" (UID: \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\") " pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:40:47.883727 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.883656 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-istio-envoy\") pod \"e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l\" (UID: \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\") " pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:40:47.883727 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.883690 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggxtj\" (UniqueName: \"kubernetes.io/projected/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-kube-api-access-ggxtj\") pod \"e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l\" (UID: \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\") " pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:40:47.883727 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.883717 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-workload-certs\") pod \"e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l\" (UID: \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\") " pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:40:47.883837 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.883743 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-workload-socket\") pod \"e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l\" (UID: \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\") " pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:40:47.984898 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.984857 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-istio-podinfo\") pod \"e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l\" (UID: \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\") " pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:40:47.985104 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.984911 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-istiod-ca-cert\") pod \"e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l\" (UID: \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\") " pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:40:47.985104 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.984974 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-credential-socket\") pod \"e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l\" (UID: \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\") " pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:40:47.985104 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.985051 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-istio-data\") pod \"e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l\" (UID: \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\") " pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:40:47.985104 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.985076 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-istio-token\") pod \"e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l\" (UID: \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\") " pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:40:47.985335 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.985122 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-istio-envoy\") pod \"e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l\" (UID: \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\") " pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:40:47.985335 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.985161 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggxtj\" (UniqueName: \"kubernetes.io/projected/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-kube-api-access-ggxtj\") pod \"e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l\" (UID: \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\") " pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:40:47.985335 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.985190 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-workload-certs\") pod \"e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l\" (UID: \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\") " pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:40:47.985335 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.985218 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-workload-socket\") pod \"e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l\" (UID: \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\") " pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:40:47.985761 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.985730 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-workload-certs\") pod \"e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l\" (UID: \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\") " pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:40:47.985887 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.985773 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-istio-data\") pod \"e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l\" (UID: \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\") " pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:40:47.985887 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.985809 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-istiod-ca-cert\") pod \"e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l\" (UID: \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\") " pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:40:47.986073 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.986050 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-credential-socket\") pod \"e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l\" (UID: \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\") " pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:40:47.986142 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.986074 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-workload-socket\") pod \"e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l\" (UID: \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\") " pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:40:47.987969 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.987929 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-istio-podinfo\") pod \"e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l\" (UID: \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\") " pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:40:47.988073 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.987971 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-istio-envoy\") pod \"e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l\" (UID: \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\") " pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:40:47.993343 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.993323 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-istio-token\") pod \"e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l\" (UID: \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\") " pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:40:47.993537 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:47.993518 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggxtj\" (UniqueName: \"kubernetes.io/projected/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-kube-api-access-ggxtj\") pod \"e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l\" (UID: \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\") " pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:40:48.152757 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:48.152669 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:40:48.301482 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:48.298930 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l"] Jul 22 03:40:48.310442 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:48.310405 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:40:48.310548 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:48.310477 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:40:48.310548 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:48.310507 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:40:48.343997 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:48.343925 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:40:48.470470 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:48.470367 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" event={"ID":"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f","Type":"ContainerStarted","Data":"50313dfdbb7d1bba0faed5914b40cf6a1a75dd8ce3654c729cc6f404edda6b7b"} Jul 22 03:40:48.470470 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:48.470430 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" event={"ID":"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f","Type":"ContainerStarted","Data":"47b0455615b0e9a684b59134b43002f28cf7f35f3abbcea24ad44bd68078e8ab"} Jul 22 03:40:48.489968 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:48.489898 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" podStartSLOduration=1.489882742 podStartE2EDuration="1.489882742s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:40:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:40:48.489126144 +0000 UTC m=+1988.938109417" watchObservedRunningTime="2026-07-22 03:40:48.489882742 +0000 UTC m=+1988.938866055" Jul 22 03:40:49.153644 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:49.153605 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:40:49.158548 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:49.158521 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:40:49.478535 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:49.478452 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:40:49.479376 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:49.479355 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:40:50.364909 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:50.364875 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:40:50.455724 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:50.455672 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:40:50.537404 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:50.537376 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:40:51.266928 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:51.266895 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:40:51.364393 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:51.364356 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:40:54.636369 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:54.636327 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:40:54.905560 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:40:54.905479 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:41:03.557899 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:03.557867 2556 generic.go:356] "Generic (PLEG): container finished" podID="286858ff-71f1-4b1c-bc3f-d1859d6e8c3f" containerID="50313dfdbb7d1bba0faed5914b40cf6a1a75dd8ce3654c729cc6f404edda6b7b" exitCode=0 Jul 22 03:41:03.558355 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:03.557970 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" event={"ID":"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f","Type":"ContainerDied","Data":"50313dfdbb7d1bba0faed5914b40cf6a1a75dd8ce3654c729cc6f404edda6b7b"} Jul 22 03:41:03.558399 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:03.558373 2556 scope.go:117] "RemoveContainer" containerID="50313dfdbb7d1bba0faed5914b40cf6a1a75dd8ce3654c729cc6f404edda6b7b" Jul 22 03:41:03.559482 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:03.559455 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:41:03.559578 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:03.559523 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:41:03.559578 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:03.559564 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:41:04.478636 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:04.478590 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:41:04.564981 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:04.564921 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" event={"ID":"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f","Type":"ContainerStarted","Data":"37b18f41d69932b7c1754d85197dc27428c5fe7c33b354f9c78b2f85a25a3746"} Jul 22 03:41:05.071636 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:05.071600 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs"] Jul 22 03:41:05.086295 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:05.086266 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs" Jul 22 03:41:05.092332 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:05.092303 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs"] Jul 22 03:41:05.092537 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:05.092343 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-513b5fd3\"/\"default-dockercfg-kv6c4\"" Jul 22 03:41:05.096974 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:05.096465 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-513b5fd3\"/\"kube-root-ca.crt\"" Jul 22 03:41:05.096974 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:05.096725 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-513b5fd3\"/\"e2e-lifecycle-model-513b5fd3-kserve-self-signed-certs\"" Jul 22 03:41:05.097170 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:05.097023 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-513b5fd3\"/\"openshift-service-ca.crt\"" Jul 22 03:41:05.099446 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:05.097636 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:41:05.144349 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:05.144322 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/959d1a7d-8681-4712-a3b3-997ad580434a-tmp-dir\") pod \"e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs\" (UID: \"959d1a7d-8681-4712-a3b3-997ad580434a\") " pod="ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs" Jul 22 03:41:05.144479 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:05.144378 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/959d1a7d-8681-4712-a3b3-997ad580434a-dshm\") pod \"e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs\" (UID: \"959d1a7d-8681-4712-a3b3-997ad580434a\") " pod="ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs" Jul 22 03:41:05.144479 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:05.144394 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/959d1a7d-8681-4712-a3b3-997ad580434a-model-cache\") pod \"e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs\" (UID: \"959d1a7d-8681-4712-a3b3-997ad580434a\") " pod="ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs" Jul 22 03:41:05.144479 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:05.144412 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/959d1a7d-8681-4712-a3b3-997ad580434a-tls-certs\") pod \"e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs\" (UID: \"959d1a7d-8681-4712-a3b3-997ad580434a\") " pod="ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs" Jul 22 03:41:05.144479 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:05.144470 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdgqs\" (UniqueName: \"kubernetes.io/projected/959d1a7d-8681-4712-a3b3-997ad580434a-kube-api-access-wdgqs\") pod \"e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs\" (UID: \"959d1a7d-8681-4712-a3b3-997ad580434a\") " pod="ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs" Jul 22 03:41:05.144637 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:05.144502 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/959d1a7d-8681-4712-a3b3-997ad580434a-home\") pod \"e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs\" (UID: \"959d1a7d-8681-4712-a3b3-997ad580434a\") " pod="ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs" Jul 22 03:41:05.153247 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:05.153219 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:41:05.154555 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:05.154529 2556 patch_prober.go:28] interesting pod/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 22 03:41:05.154658 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:05.154594 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" podUID="286858ff-71f1-4b1c-bc3f-d1859d6e8c3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 22 03:41:05.245156 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:05.245121 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/959d1a7d-8681-4712-a3b3-997ad580434a-tmp-dir\") pod \"e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs\" (UID: \"959d1a7d-8681-4712-a3b3-997ad580434a\") " pod="ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs" Jul 22 03:41:05.245357 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:05.245182 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/959d1a7d-8681-4712-a3b3-997ad580434a-dshm\") pod \"e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs\" (UID: \"959d1a7d-8681-4712-a3b3-997ad580434a\") " pod="ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs" Jul 22 03:41:05.245357 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:05.245200 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/959d1a7d-8681-4712-a3b3-997ad580434a-model-cache\") pod \"e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs\" (UID: \"959d1a7d-8681-4712-a3b3-997ad580434a\") " pod="ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs" Jul 22 03:41:05.245357 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:05.245214 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/959d1a7d-8681-4712-a3b3-997ad580434a-tls-certs\") pod \"e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs\" (UID: \"959d1a7d-8681-4712-a3b3-997ad580434a\") " pod="ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs" Jul 22 03:41:05.245357 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:05.245245 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdgqs\" (UniqueName: \"kubernetes.io/projected/959d1a7d-8681-4712-a3b3-997ad580434a-kube-api-access-wdgqs\") pod \"e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs\" (UID: \"959d1a7d-8681-4712-a3b3-997ad580434a\") " pod="ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs" Jul 22 03:41:05.245357 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:05.245272 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/959d1a7d-8681-4712-a3b3-997ad580434a-home\") pod \"e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs\" (UID: \"959d1a7d-8681-4712-a3b3-997ad580434a\") " pod="ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs" Jul 22 03:41:05.245642 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:05.245580 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/959d1a7d-8681-4712-a3b3-997ad580434a-tmp-dir\") pod \"e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs\" (UID: \"959d1a7d-8681-4712-a3b3-997ad580434a\") " pod="ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs" Jul 22 03:41:05.245642 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:05.245620 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/959d1a7d-8681-4712-a3b3-997ad580434a-model-cache\") pod \"e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs\" (UID: \"959d1a7d-8681-4712-a3b3-997ad580434a\") " pod="ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs" Jul 22 03:41:05.245726 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:05.245674 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/959d1a7d-8681-4712-a3b3-997ad580434a-home\") pod \"e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs\" (UID: \"959d1a7d-8681-4712-a3b3-997ad580434a\") " pod="ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs" Jul 22 03:41:05.247510 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:05.247487 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/959d1a7d-8681-4712-a3b3-997ad580434a-dshm\") pod \"e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs\" (UID: \"959d1a7d-8681-4712-a3b3-997ad580434a\") " pod="ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs" Jul 22 03:41:05.247802 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:05.247786 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/959d1a7d-8681-4712-a3b3-997ad580434a-tls-certs\") pod \"e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs\" (UID: \"959d1a7d-8681-4712-a3b3-997ad580434a\") " pod="ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs" Jul 22 03:41:05.253196 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:05.253173 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdgqs\" (UniqueName: \"kubernetes.io/projected/959d1a7d-8681-4712-a3b3-997ad580434a-kube-api-access-wdgqs\") pod \"e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs\" (UID: \"959d1a7d-8681-4712-a3b3-997ad580434a\") " pod="ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs" Jul 22 03:41:05.405219 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:05.405129 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs" Jul 22 03:41:05.569472 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:05.569423 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs"] Jul 22 03:41:06.153366 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:06.153328 2556 patch_prober.go:28] interesting pod/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 22 03:41:06.153550 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:06.153382 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" podUID="286858ff-71f1-4b1c-bc3f-d1859d6e8c3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 22 03:41:06.586476 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:06.586431 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs" event={"ID":"959d1a7d-8681-4712-a3b3-997ad580434a","Type":"ContainerStarted","Data":"4a20590ad63691ca41b5479235b9fc3d631e93da1a890c8aca010a7eacc2aba4"} Jul 22 03:41:06.586980 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:06.586486 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs" event={"ID":"959d1a7d-8681-4712-a3b3-997ad580434a","Type":"ContainerStarted","Data":"5db6a12c841bb1e9bd8c9d543b55c036821d19e8f9679f05e0151e6b28390af8"} Jul 22 03:41:06.605968 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:06.605903 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs" podStartSLOduration=1.605885886 podStartE2EDuration="1.605885886s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:41:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:41:06.603451387 +0000 UTC m=+2007.052434658" watchObservedRunningTime="2026-07-22 03:41:06.605885886 +0000 UTC m=+2007.054869174" Jul 22 03:41:07.154106 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:07.154064 2556 patch_prober.go:28] interesting pod/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 22 03:41:07.154311 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:07.154130 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" podUID="286858ff-71f1-4b1c-bc3f-d1859d6e8c3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 22 03:41:08.153293 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:08.153264 2556 patch_prober.go:28] interesting pod/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 22 03:41:08.153666 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:08.153316 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" podUID="286858ff-71f1-4b1c-bc3f-d1859d6e8c3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 22 03:41:09.157577 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:09.157545 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:41:09.157991 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:09.157913 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:41:09.158715 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:09.158694 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:41:09.442855 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:09.442772 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:41:15.406283 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:15.406238 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs" Jul 22 03:41:15.406283 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:15.406290 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs" Jul 22 03:41:15.419042 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:15.419012 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs" Jul 22 03:41:15.647555 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:15.647526 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs" Jul 22 03:41:29.859487 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:29.859449 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:41:47.433941 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:47.433900 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:41:48.816336 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:48.816301 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:41:53.378656 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:53.378616 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:41:54.734137 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:54.734097 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:41:55.503703 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:41:55.503665 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:42:00.495069 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:00.494992 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs"] Jul 22 03:42:00.553501 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:00.553462 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:42:15.518926 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:15.518863 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs" podUID="959d1a7d-8681-4712-a3b3-997ad580434a" containerName="main" containerID="cri-o://4a20590ad63691ca41b5479235b9fc3d631e93da1a890c8aca010a7eacc2aba4" gracePeriod=45 Jul 22 03:42:15.637509 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:15.637464 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs" podUID="959d1a7d-8681-4712-a3b3-997ad580434a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": dial tcp 10.133.0.61:8000: connect: connection refused" Jul 22 03:42:15.775479 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:15.775415 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs" Jul 22 03:42:15.806521 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:15.806498 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wdgqs\" (UniqueName: \"kubernetes.io/projected/959d1a7d-8681-4712-a3b3-997ad580434a-kube-api-access-wdgqs\") pod \"959d1a7d-8681-4712-a3b3-997ad580434a\" (UID: \"959d1a7d-8681-4712-a3b3-997ad580434a\") " Jul 22 03:42:15.806670 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:15.806549 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/959d1a7d-8681-4712-a3b3-997ad580434a-tls-certs\") pod \"959d1a7d-8681-4712-a3b3-997ad580434a\" (UID: \"959d1a7d-8681-4712-a3b3-997ad580434a\") " Jul 22 03:42:15.806670 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:15.806601 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/959d1a7d-8681-4712-a3b3-997ad580434a-home\") pod \"959d1a7d-8681-4712-a3b3-997ad580434a\" (UID: \"959d1a7d-8681-4712-a3b3-997ad580434a\") " Jul 22 03:42:15.806670 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:15.806638 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/959d1a7d-8681-4712-a3b3-997ad580434a-model-cache\") pod \"959d1a7d-8681-4712-a3b3-997ad580434a\" (UID: \"959d1a7d-8681-4712-a3b3-997ad580434a\") " Jul 22 03:42:15.806843 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:15.806681 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/959d1a7d-8681-4712-a3b3-997ad580434a-tmp-dir\") pod \"959d1a7d-8681-4712-a3b3-997ad580434a\" (UID: \"959d1a7d-8681-4712-a3b3-997ad580434a\") " Jul 22 03:42:15.806843 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:15.806704 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/959d1a7d-8681-4712-a3b3-997ad580434a-dshm\") pod \"959d1a7d-8681-4712-a3b3-997ad580434a\" (UID: \"959d1a7d-8681-4712-a3b3-997ad580434a\") " Jul 22 03:42:15.807231 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:15.807159 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/959d1a7d-8681-4712-a3b3-997ad580434a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "959d1a7d-8681-4712-a3b3-997ad580434a" (UID: "959d1a7d-8681-4712-a3b3-997ad580434a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:42:15.807355 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:15.807260 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/959d1a7d-8681-4712-a3b3-997ad580434a-model-cache" (OuterVolumeSpecName: "model-cache") pod "959d1a7d-8681-4712-a3b3-997ad580434a" (UID: "959d1a7d-8681-4712-a3b3-997ad580434a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:42:15.807355 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:15.807314 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/959d1a7d-8681-4712-a3b3-997ad580434a-home" (OuterVolumeSpecName: "home") pod "959d1a7d-8681-4712-a3b3-997ad580434a" (UID: "959d1a7d-8681-4712-a3b3-997ad580434a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:42:15.809052 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:15.809019 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/959d1a7d-8681-4712-a3b3-997ad580434a-dshm" (OuterVolumeSpecName: "dshm") pod "959d1a7d-8681-4712-a3b3-997ad580434a" (UID: "959d1a7d-8681-4712-a3b3-997ad580434a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:42:15.809147 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:15.809021 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/959d1a7d-8681-4712-a3b3-997ad580434a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "959d1a7d-8681-4712-a3b3-997ad580434a" (UID: "959d1a7d-8681-4712-a3b3-997ad580434a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:42:15.809147 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:15.809122 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/959d1a7d-8681-4712-a3b3-997ad580434a-kube-api-access-wdgqs" (OuterVolumeSpecName: "kube-api-access-wdgqs") pod "959d1a7d-8681-4712-a3b3-997ad580434a" (UID: "959d1a7d-8681-4712-a3b3-997ad580434a"). InnerVolumeSpecName "kube-api-access-wdgqs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:42:15.907591 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:15.907560 2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/959d1a7d-8681-4712-a3b3-997ad580434a-model-cache\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:42:15.907754 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:15.907613 2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/959d1a7d-8681-4712-a3b3-997ad580434a-tmp-dir\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:42:15.907754 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:15.907625 2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/959d1a7d-8681-4712-a3b3-997ad580434a-dshm\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:42:15.907754 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:15.907636 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wdgqs\" (UniqueName: \"kubernetes.io/projected/959d1a7d-8681-4712-a3b3-997ad580434a-kube-api-access-wdgqs\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:42:15.907754 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:15.907650 2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/959d1a7d-8681-4712-a3b3-997ad580434a-tls-certs\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:42:15.907754 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:15.907663 2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/959d1a7d-8681-4712-a3b3-997ad580434a-home\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:42:15.969847 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:15.969818 2556 generic.go:356] "Generic (PLEG): container finished" podID="959d1a7d-8681-4712-a3b3-997ad580434a" containerID="4a20590ad63691ca41b5479235b9fc3d631e93da1a890c8aca010a7eacc2aba4" exitCode=0 Jul 22 03:42:15.970028 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:15.969859 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs" Jul 22 03:42:15.970028 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:15.969904 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs" event={"ID":"959d1a7d-8681-4712-a3b3-997ad580434a","Type":"ContainerDied","Data":"4a20590ad63691ca41b5479235b9fc3d631e93da1a890c8aca010a7eacc2aba4"} Jul 22 03:42:15.970028 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:15.969946 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs" event={"ID":"959d1a7d-8681-4712-a3b3-997ad580434a","Type":"ContainerDied","Data":"5db6a12c841bb1e9bd8c9d543b55c036821d19e8f9679f05e0151e6b28390af8"} Jul 22 03:42:15.970028 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:15.969978 2556 scope.go:117] "RemoveContainer" containerID="4a20590ad63691ca41b5479235b9fc3d631e93da1a890c8aca010a7eacc2aba4" Jul 22 03:42:15.979718 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:15.979695 2556 scope.go:117] "RemoveContainer" containerID="4a20590ad63691ca41b5479235b9fc3d631e93da1a890c8aca010a7eacc2aba4" Jul 22 03:42:15.980014 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:42:15.979996 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a20590ad63691ca41b5479235b9fc3d631e93da1a890c8aca010a7eacc2aba4\": container with ID starting with 4a20590ad63691ca41b5479235b9fc3d631e93da1a890c8aca010a7eacc2aba4 not found: ID does not exist" containerID="4a20590ad63691ca41b5479235b9fc3d631e93da1a890c8aca010a7eacc2aba4" Jul 22 03:42:15.980071 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:15.980022 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a20590ad63691ca41b5479235b9fc3d631e93da1a890c8aca010a7eacc2aba4"} err="failed to get container status \"4a20590ad63691ca41b5479235b9fc3d631e93da1a890c8aca010a7eacc2aba4\": rpc error: code = NotFound desc = could not find container \"4a20590ad63691ca41b5479235b9fc3d631e93da1a890c8aca010a7eacc2aba4\": container with ID starting with 4a20590ad63691ca41b5479235b9fc3d631e93da1a890c8aca010a7eacc2aba4 not found: ID does not exist" Jul 22 03:42:15.991654 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:15.991634 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs"] Jul 22 03:42:15.995989 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:15.995967 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-513b5fd3/e2e-lifecycle-model-513b5fd3-kserve-85d78bdb7f-ggqbs"] Jul 22 03:42:16.024412 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:16.024387 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="959d1a7d-8681-4712-a3b3-997ad580434a" path="/var/lib/kubelet/pods/959d1a7d-8681-4712-a3b3-997ad580434a/volumes" Jul 22 03:42:27.078232 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:27.078195 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:42:27.178726 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:27.178691 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:42:36.792475 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:36.792440 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:42:36.969615 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:36.969584 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:42:37.765677 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:37.765642 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:42:40.248271 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:40.248241 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscf8_a01dfba4-f590-412a-8438-1646c51bb1a1/ovn-acl-logging/0.log" Jul 22 03:42:40.253716 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:40.253696 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscf8_a01dfba4-f590-412a-8438-1646c51bb1a1/ovn-acl-logging/0.log" Jul 22 03:42:43.485487 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:43.485456 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:42:43.554378 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:43.554342 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:42:43.773725 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:43.773630 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l"] Jul 22 03:42:43.774213 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:43.774173 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" podUID="286858ff-71f1-4b1c-bc3f-d1859d6e8c3f" containerName="istio-proxy" containerID="cri-o://37b18f41d69932b7c1754d85197dc27428c5fe7c33b354f9c78b2f85a25a3746" gracePeriod=30 Jul 22 03:42:43.799139 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:43.799108 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:42:44.002267 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:44.002232 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:42:45.074161 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:45.074126 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:42:45.191261 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:45.191223 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:42:45.327241 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:45.327153 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:42:45.993579 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:45.993546 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:42:46.090406 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:46.090374 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:42:46.195356 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:46.195325 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:42:47.463831 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:47.463799 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:42:47.929010 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:47.928975 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:42:49.034295 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.034228 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:42:49.126758 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.126726 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-istio-data\") pod \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\" (UID: \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\") " Jul 22 03:42:49.126758 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.126764 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-istio-token\") pod \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\" (UID: \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\") " Jul 22 03:42:49.127011 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.126799 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ggxtj\" (UniqueName: \"kubernetes.io/projected/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-kube-api-access-ggxtj\") pod \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\" (UID: \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\") " Jul 22 03:42:49.127011 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.126824 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-credential-socket\") pod \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\" (UID: \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\") " Jul 22 03:42:49.127011 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.126841 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-istio-envoy\") pod \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\" (UID: \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\") " Jul 22 03:42:49.127011 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.126868 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-istiod-ca-cert\") pod \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\" (UID: \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\") " Jul 22 03:42:49.127177 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.127012 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-workload-certs\") pod \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\" (UID: \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\") " Jul 22 03:42:49.127375 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.127324 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "286858ff-71f1-4b1c-bc3f-d1859d6e8c3f" (UID: "286858ff-71f1-4b1c-bc3f-d1859d6e8c3f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:42:49.127375 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.127347 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "286858ff-71f1-4b1c-bc3f-d1859d6e8c3f" (UID: "286858ff-71f1-4b1c-bc3f-d1859d6e8c3f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:42:49.127375 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.127354 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-istio-podinfo\") pod \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\" (UID: \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\") " Jul 22 03:42:49.127646 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.127380 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "286858ff-71f1-4b1c-bc3f-d1859d6e8c3f" (UID: "286858ff-71f1-4b1c-bc3f-d1859d6e8c3f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:42:49.127646 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.127417 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-workload-socket\") pod \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\" (UID: \"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f\") " Jul 22 03:42:49.127646 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.127594 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "286858ff-71f1-4b1c-bc3f-d1859d6e8c3f" (UID: "286858ff-71f1-4b1c-bc3f-d1859d6e8c3f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:42:49.127831 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.127664 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-istio-data" (OuterVolumeSpecName: "istio-data") pod "286858ff-71f1-4b1c-bc3f-d1859d6e8c3f" (UID: "286858ff-71f1-4b1c-bc3f-d1859d6e8c3f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:42:49.127831 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.127745 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-credential-socket\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:42:49.127831 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.127766 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-istiod-ca-cert\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:42:49.127831 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.127781 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-workload-certs\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:42:49.127831 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.127793 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-workload-socket\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:42:49.127831 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.127805 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-istio-data\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:42:49.129216 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.129184 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "286858ff-71f1-4b1c-bc3f-d1859d6e8c3f" (UID: "286858ff-71f1-4b1c-bc3f-d1859d6e8c3f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:42:49.129310 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.129241 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-istio-token" (OuterVolumeSpecName: "istio-token") pod "286858ff-71f1-4b1c-bc3f-d1859d6e8c3f" (UID: "286858ff-71f1-4b1c-bc3f-d1859d6e8c3f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:42:49.129310 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.129251 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-kube-api-access-ggxtj" (OuterVolumeSpecName: "kube-api-access-ggxtj") pod "286858ff-71f1-4b1c-bc3f-d1859d6e8c3f" (UID: "286858ff-71f1-4b1c-bc3f-d1859d6e8c3f"). InnerVolumeSpecName "kube-api-access-ggxtj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:42:49.129448 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.129430 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "286858ff-71f1-4b1c-bc3f-d1859d6e8c3f" (UID: "286858ff-71f1-4b1c-bc3f-d1859d6e8c3f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 03:42:49.155127 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.155090 2556 generic.go:356] "Generic (PLEG): container finished" podID="286858ff-71f1-4b1c-bc3f-d1859d6e8c3f" containerID="37b18f41d69932b7c1754d85197dc27428c5fe7c33b354f9c78b2f85a25a3746" exitCode=0 Jul 22 03:42:49.155240 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.155142 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" Jul 22 03:42:49.155240 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.155167 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" event={"ID":"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f","Type":"ContainerDied","Data":"37b18f41d69932b7c1754d85197dc27428c5fe7c33b354f9c78b2f85a25a3746"} Jul 22 03:42:49.155240 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.155225 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l" event={"ID":"286858ff-71f1-4b1c-bc3f-d1859d6e8c3f","Type":"ContainerDied","Data":"47b0455615b0e9a684b59134b43002f28cf7f35f3abbcea24ad44bd68078e8ab"} Jul 22 03:42:49.155343 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.155248 2556 scope.go:117] "RemoveContainer" containerID="37b18f41d69932b7c1754d85197dc27428c5fe7c33b354f9c78b2f85a25a3746" Jul 22 03:42:49.165144 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.165128 2556 scope.go:117] "RemoveContainer" containerID="50313dfdbb7d1bba0faed5914b40cf6a1a75dd8ce3654c729cc6f404edda6b7b" Jul 22 03:42:49.174572 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.174554 2556 scope.go:117] "RemoveContainer" containerID="37b18f41d69932b7c1754d85197dc27428c5fe7c33b354f9c78b2f85a25a3746" Jul 22 03:42:49.174865 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:42:49.174847 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37b18f41d69932b7c1754d85197dc27428c5fe7c33b354f9c78b2f85a25a3746\": container with ID starting with 37b18f41d69932b7c1754d85197dc27428c5fe7c33b354f9c78b2f85a25a3746 not found: ID does not exist" containerID="37b18f41d69932b7c1754d85197dc27428c5fe7c33b354f9c78b2f85a25a3746" Jul 22 03:42:49.174936 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.174873 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37b18f41d69932b7c1754d85197dc27428c5fe7c33b354f9c78b2f85a25a3746"} err="failed to get container status \"37b18f41d69932b7c1754d85197dc27428c5fe7c33b354f9c78b2f85a25a3746\": rpc error: code = NotFound desc = could not find container \"37b18f41d69932b7c1754d85197dc27428c5fe7c33b354f9c78b2f85a25a3746\": container with ID starting with 37b18f41d69932b7c1754d85197dc27428c5fe7c33b354f9c78b2f85a25a3746 not found: ID does not exist" Jul 22 03:42:49.174936 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.174890 2556 scope.go:117] "RemoveContainer" containerID="50313dfdbb7d1bba0faed5914b40cf6a1a75dd8ce3654c729cc6f404edda6b7b" Jul 22 03:42:49.175217 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:42:49.175197 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50313dfdbb7d1bba0faed5914b40cf6a1a75dd8ce3654c729cc6f404edda6b7b\": container with ID starting with 50313dfdbb7d1bba0faed5914b40cf6a1a75dd8ce3654c729cc6f404edda6b7b not found: ID does not exist" containerID="50313dfdbb7d1bba0faed5914b40cf6a1a75dd8ce3654c729cc6f404edda6b7b" Jul 22 03:42:49.175279 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.175223 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50313dfdbb7d1bba0faed5914b40cf6a1a75dd8ce3654c729cc6f404edda6b7b"} err="failed to get container status \"50313dfdbb7d1bba0faed5914b40cf6a1a75dd8ce3654c729cc6f404edda6b7b\": rpc error: code = NotFound desc = could not find container \"50313dfdbb7d1bba0faed5914b40cf6a1a75dd8ce3654c729cc6f404edda6b7b\": container with ID starting with 50313dfdbb7d1bba0faed5914b40cf6a1a75dd8ce3654c729cc6f404edda6b7b not found: ID does not exist" Jul 22 03:42:49.178545 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.178524 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l"] Jul 22 03:42:49.183205 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.183186 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-513b5fd3-openshift-default-65dc846df5-zsg2l"] Jul 22 03:42:49.229034 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.229009 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ggxtj\" (UniqueName: \"kubernetes.io/projected/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-kube-api-access-ggxtj\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:42:49.229034 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.229031 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-istio-envoy\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:42:49.229034 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.229040 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-istio-podinfo\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:42:49.229218 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:49.229048 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f-istio-token\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:42:50.024043 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:50.024002 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="286858ff-71f1-4b1c-bc3f-d1859d6e8c3f" path="/var/lib/kubelet/pods/286858ff-71f1-4b1c-bc3f-d1859d6e8c3f/volumes" Jul 22 03:42:53.315946 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:53.315915 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:42:53.474916 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:53.474879 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:42:54.032874 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:54.032837 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:42:59.935148 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:59.935114 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:42:59.947460 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:42:59.947429 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:43:00.055168 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:00.055133 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:43:00.147824 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:00.147791 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:43:10.641437 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:10.641407 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:43:10.977757 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:10.977668 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:43:31.914558 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:31.914473 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:43:33.360131 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:33.360091 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:43:33.461301 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:33.461266 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:43:38.372142 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:38.372109 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:43:38.497170 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:38.497130 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:43:38.790153 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:38.790068 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:43:39.017423 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.017386 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn"] Jul 22 03:43:39.017940 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.017925 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="286858ff-71f1-4b1c-bc3f-d1859d6e8c3f" containerName="istio-proxy" Jul 22 03:43:39.018023 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.017943 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="286858ff-71f1-4b1c-bc3f-d1859d6e8c3f" containerName="istio-proxy" Jul 22 03:43:39.018023 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.017977 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="286858ff-71f1-4b1c-bc3f-d1859d6e8c3f" containerName="istio-proxy" Jul 22 03:43:39.018023 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.017986 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="286858ff-71f1-4b1c-bc3f-d1859d6e8c3f" containerName="istio-proxy" Jul 22 03:43:39.018023 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.018021 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="959d1a7d-8681-4712-a3b3-997ad580434a" containerName="main" Jul 22 03:43:39.018023 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.018030 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="959d1a7d-8681-4712-a3b3-997ad580434a" containerName="main" Jul 22 03:43:39.018191 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.018103 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="286858ff-71f1-4b1c-bc3f-d1859d6e8c3f" containerName="istio-proxy" Jul 22 03:43:39.018191 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.018112 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="286858ff-71f1-4b1c-bc3f-d1859d6e8c3f" containerName="istio-proxy" Jul 22 03:43:39.018191 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.018125 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="959d1a7d-8681-4712-a3b3-997ad580434a" containerName="main" Jul 22 03:43:39.021749 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.021726 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:43:39.027433 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.027413 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-44f1c1-openshift-default-dockercfg-htch7\"" Jul 22 03:43:39.037343 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.037321 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn"] Jul 22 03:43:39.043735 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.043680 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:43:39.080792 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.080765 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c38bf61-f5eb-4af7-89b7-f627eb446979-istio-envoy\") pod \"e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn\" (UID: \"4c38bf61-f5eb-4af7-89b7-f627eb446979\") " pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:43:39.080894 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.080821 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c38bf61-f5eb-4af7-89b7-f627eb446979-workload-certs\") pod \"e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn\" (UID: \"4c38bf61-f5eb-4af7-89b7-f627eb446979\") " pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:43:39.080999 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.080945 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c38bf61-f5eb-4af7-89b7-f627eb446979-istio-podinfo\") pod \"e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn\" (UID: \"4c38bf61-f5eb-4af7-89b7-f627eb446979\") " pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:43:39.080999 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.080990 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c38bf61-f5eb-4af7-89b7-f627eb446979-workload-socket\") pod \"e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn\" (UID: \"4c38bf61-f5eb-4af7-89b7-f627eb446979\") " pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:43:39.081115 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.081012 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c38bf61-f5eb-4af7-89b7-f627eb446979-credential-socket\") pod \"e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn\" (UID: \"4c38bf61-f5eb-4af7-89b7-f627eb446979\") " pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:43:39.081210 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.081190 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c38bf61-f5eb-4af7-89b7-f627eb446979-istio-token\") pod \"e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn\" (UID: \"4c38bf61-f5eb-4af7-89b7-f627eb446979\") " pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:43:39.081270 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.081227 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47k5n\" (UniqueName: \"kubernetes.io/projected/4c38bf61-f5eb-4af7-89b7-f627eb446979-kube-api-access-47k5n\") pod \"e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn\" (UID: \"4c38bf61-f5eb-4af7-89b7-f627eb446979\") " pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:43:39.081329 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.081270 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c38bf61-f5eb-4af7-89b7-f627eb446979-istiod-ca-cert\") pod \"e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn\" (UID: \"4c38bf61-f5eb-4af7-89b7-f627eb446979\") " pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:43:39.081400 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.081383 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c38bf61-f5eb-4af7-89b7-f627eb446979-istio-data\") pod \"e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn\" (UID: \"4c38bf61-f5eb-4af7-89b7-f627eb446979\") " pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:43:39.182384 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.182355 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c38bf61-f5eb-4af7-89b7-f627eb446979-istio-data\") pod \"e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn\" (UID: \"4c38bf61-f5eb-4af7-89b7-f627eb446979\") " pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:43:39.182550 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.182409 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c38bf61-f5eb-4af7-89b7-f627eb446979-istio-envoy\") pod \"e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn\" (UID: \"4c38bf61-f5eb-4af7-89b7-f627eb446979\") " pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:43:39.182550 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.182524 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c38bf61-f5eb-4af7-89b7-f627eb446979-workload-certs\") pod \"e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn\" (UID: \"4c38bf61-f5eb-4af7-89b7-f627eb446979\") " pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:43:39.182685 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.182597 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c38bf61-f5eb-4af7-89b7-f627eb446979-istio-podinfo\") pod \"e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn\" (UID: \"4c38bf61-f5eb-4af7-89b7-f627eb446979\") " pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:43:39.182685 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.182627 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c38bf61-f5eb-4af7-89b7-f627eb446979-workload-socket\") pod \"e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn\" (UID: \"4c38bf61-f5eb-4af7-89b7-f627eb446979\") " pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:43:39.182685 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.182650 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c38bf61-f5eb-4af7-89b7-f627eb446979-credential-socket\") pod \"e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn\" (UID: \"4c38bf61-f5eb-4af7-89b7-f627eb446979\") " pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:43:39.182846 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.182701 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c38bf61-f5eb-4af7-89b7-f627eb446979-istio-token\") pod \"e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn\" (UID: \"4c38bf61-f5eb-4af7-89b7-f627eb446979\") " pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:43:39.182846 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.182726 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47k5n\" (UniqueName: \"kubernetes.io/projected/4c38bf61-f5eb-4af7-89b7-f627eb446979-kube-api-access-47k5n\") pod \"e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn\" (UID: \"4c38bf61-f5eb-4af7-89b7-f627eb446979\") " pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:43:39.182846 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.182769 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c38bf61-f5eb-4af7-89b7-f627eb446979-istiod-ca-cert\") pod \"e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn\" (UID: \"4c38bf61-f5eb-4af7-89b7-f627eb446979\") " pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:43:39.182846 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.182832 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c38bf61-f5eb-4af7-89b7-f627eb446979-istio-data\") pod \"e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn\" (UID: \"4c38bf61-f5eb-4af7-89b7-f627eb446979\") " pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:43:39.183046 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.182897 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c38bf61-f5eb-4af7-89b7-f627eb446979-workload-certs\") pod \"e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn\" (UID: \"4c38bf61-f5eb-4af7-89b7-f627eb446979\") " pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:43:39.183631 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.183243 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c38bf61-f5eb-4af7-89b7-f627eb446979-credential-socket\") pod \"e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn\" (UID: \"4c38bf61-f5eb-4af7-89b7-f627eb446979\") " pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:43:39.183631 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.183269 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c38bf61-f5eb-4af7-89b7-f627eb446979-workload-socket\") pod \"e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn\" (UID: \"4c38bf61-f5eb-4af7-89b7-f627eb446979\") " pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:43:39.183851 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.183830 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c38bf61-f5eb-4af7-89b7-f627eb446979-istiod-ca-cert\") pod \"e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn\" (UID: \"4c38bf61-f5eb-4af7-89b7-f627eb446979\") " pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:43:39.184173 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.184149 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:43:39.185189 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.185164 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c38bf61-f5eb-4af7-89b7-f627eb446979-istio-envoy\") pod \"e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn\" (UID: \"4c38bf61-f5eb-4af7-89b7-f627eb446979\") " pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:43:39.185300 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.185175 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c38bf61-f5eb-4af7-89b7-f627eb446979-istio-podinfo\") pod \"e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn\" (UID: \"4c38bf61-f5eb-4af7-89b7-f627eb446979\") " pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:43:39.190962 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.190916 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c38bf61-f5eb-4af7-89b7-f627eb446979-istio-token\") pod \"e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn\" (UID: \"4c38bf61-f5eb-4af7-89b7-f627eb446979\") " pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:43:39.191375 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.191355 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47k5n\" (UniqueName: \"kubernetes.io/projected/4c38bf61-f5eb-4af7-89b7-f627eb446979-kube-api-access-47k5n\") pod \"e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn\" (UID: \"4c38bf61-f5eb-4af7-89b7-f627eb446979\") " pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:43:39.334340 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.334313 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:43:39.473018 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.472899 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn"] Jul 22 03:43:39.477279 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:43:39.477247 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c38bf61_f5eb_4af7_89b7_f627eb446979.slice/crio-ead9c66369d930db709cb613e6bafa6caf400b886064fcb0f56e640918aeadad WatchSource:0}: Error finding container ead9c66369d930db709cb613e6bafa6caf400b886064fcb0f56e640918aeadad: Status 404 returned error can't find the container with id ead9c66369d930db709cb613e6bafa6caf400b886064fcb0f56e640918aeadad Jul 22 03:43:39.479556 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.479514 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:43:39.479650 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.479583 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:43:39.479650 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:39.479610 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:43:40.428639 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:40.428603 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" event={"ID":"4c38bf61-f5eb-4af7-89b7-f627eb446979","Type":"ContainerStarted","Data":"c7b2a6f97fad2ba597d2c023e72af065694217dcdcef078332e7a46c66425668"} Jul 22 03:43:40.428835 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:40.428665 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" event={"ID":"4c38bf61-f5eb-4af7-89b7-f627eb446979","Type":"ContainerStarted","Data":"ead9c66369d930db709cb613e6bafa6caf400b886064fcb0f56e640918aeadad"} Jul 22 03:43:40.453472 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:40.453426 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" podStartSLOduration=2.453409806 podStartE2EDuration="2.453409806s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:43:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:43:40.451926966 +0000 UTC m=+2160.900910278" watchObservedRunningTime="2026-07-22 03:43:40.453409806 +0000 UTC m=+2160.902393056" Jul 22 03:43:41.334968 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:41.334917 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:43:41.339990 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:41.339944 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:43:41.434329 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:41.434304 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:43:41.435210 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:41.435193 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:43:42.364862 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:42.364826 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:43:42.461277 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:42.461238 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:43:42.622087 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:42.622007 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:43:42.723516 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:42.723483 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:43:43.100288 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:43.100253 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:43:43.211113 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:43.211060 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:43:46.852152 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:46.852117 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:43:47.168621 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.168538 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:43:47.453646 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.453568 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7"] Jul 22 03:43:47.457116 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.457098 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:43:47.460553 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.460529 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-4c93e4-openshift-default-dockercfg-p8484\"" Jul 22 03:43:47.475007 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.474979 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7"] Jul 22 03:43:47.494601 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.494565 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:43:47.553981 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.553902 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d6655c6-26fb-4450-a4ac-925e068eb84d-istio-token\") pod \"e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7\" (UID: \"3d6655c6-26fb-4450-a4ac-925e068eb84d\") " pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:43:47.554176 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.554008 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d6655c6-26fb-4450-a4ac-925e068eb84d-credential-socket\") pod \"e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7\" (UID: \"3d6655c6-26fb-4450-a4ac-925e068eb84d\") " pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:43:47.554176 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.554040 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gv9nh\" (UniqueName: \"kubernetes.io/projected/3d6655c6-26fb-4450-a4ac-925e068eb84d-kube-api-access-gv9nh\") pod \"e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7\" (UID: \"3d6655c6-26fb-4450-a4ac-925e068eb84d\") " pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:43:47.554176 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.554067 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d6655c6-26fb-4450-a4ac-925e068eb84d-workload-certs\") pod \"e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7\" (UID: \"3d6655c6-26fb-4450-a4ac-925e068eb84d\") " pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:43:47.554176 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.554093 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d6655c6-26fb-4450-a4ac-925e068eb84d-istiod-ca-cert\") pod \"e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7\" (UID: \"3d6655c6-26fb-4450-a4ac-925e068eb84d\") " pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:43:47.554176 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.554124 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d6655c6-26fb-4450-a4ac-925e068eb84d-istio-envoy\") pod \"e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7\" (UID: \"3d6655c6-26fb-4450-a4ac-925e068eb84d\") " pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:43:47.554176 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.554153 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d6655c6-26fb-4450-a4ac-925e068eb84d-istio-data\") pod \"e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7\" (UID: \"3d6655c6-26fb-4450-a4ac-925e068eb84d\") " pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:43:47.554507 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.554209 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d6655c6-26fb-4450-a4ac-925e068eb84d-istio-podinfo\") pod \"e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7\" (UID: \"3d6655c6-26fb-4450-a4ac-925e068eb84d\") " pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:43:47.554507 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.554292 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d6655c6-26fb-4450-a4ac-925e068eb84d-workload-socket\") pod \"e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7\" (UID: \"3d6655c6-26fb-4450-a4ac-925e068eb84d\") " pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:43:47.595249 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.595217 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:43:47.655666 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.655630 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d6655c6-26fb-4450-a4ac-925e068eb84d-istio-podinfo\") pod \"e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7\" (UID: \"3d6655c6-26fb-4450-a4ac-925e068eb84d\") " pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:43:47.655819 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.655777 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d6655c6-26fb-4450-a4ac-925e068eb84d-workload-socket\") pod \"e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7\" (UID: \"3d6655c6-26fb-4450-a4ac-925e068eb84d\") " pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:43:47.655819 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.655811 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d6655c6-26fb-4450-a4ac-925e068eb84d-istio-token\") pod \"e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7\" (UID: \"3d6655c6-26fb-4450-a4ac-925e068eb84d\") " pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:43:47.655912 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.655869 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d6655c6-26fb-4450-a4ac-925e068eb84d-credential-socket\") pod \"e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7\" (UID: \"3d6655c6-26fb-4450-a4ac-925e068eb84d\") " pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:43:47.655912 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.655889 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gv9nh\" (UniqueName: \"kubernetes.io/projected/3d6655c6-26fb-4450-a4ac-925e068eb84d-kube-api-access-gv9nh\") pod \"e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7\" (UID: \"3d6655c6-26fb-4450-a4ac-925e068eb84d\") " pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:43:47.655912 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.655907 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d6655c6-26fb-4450-a4ac-925e068eb84d-workload-certs\") pod \"e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7\" (UID: \"3d6655c6-26fb-4450-a4ac-925e068eb84d\") " pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:43:47.656082 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.655931 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d6655c6-26fb-4450-a4ac-925e068eb84d-istiod-ca-cert\") pod \"e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7\" (UID: \"3d6655c6-26fb-4450-a4ac-925e068eb84d\") " pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:43:47.656082 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.655978 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d6655c6-26fb-4450-a4ac-925e068eb84d-istio-envoy\") pod \"e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7\" (UID: \"3d6655c6-26fb-4450-a4ac-925e068eb84d\") " pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:43:47.656082 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.656050 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d6655c6-26fb-4450-a4ac-925e068eb84d-istio-data\") pod \"e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7\" (UID: \"3d6655c6-26fb-4450-a4ac-925e068eb84d\") " pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:43:47.656254 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.656239 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d6655c6-26fb-4450-a4ac-925e068eb84d-workload-socket\") pod \"e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7\" (UID: \"3d6655c6-26fb-4450-a4ac-925e068eb84d\") " pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:43:47.656313 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.656282 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d6655c6-26fb-4450-a4ac-925e068eb84d-credential-socket\") pod \"e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7\" (UID: \"3d6655c6-26fb-4450-a4ac-925e068eb84d\") " pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:43:47.656527 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.656498 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d6655c6-26fb-4450-a4ac-925e068eb84d-istio-data\") pod \"e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7\" (UID: \"3d6655c6-26fb-4450-a4ac-925e068eb84d\") " pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:43:47.656623 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.656576 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d6655c6-26fb-4450-a4ac-925e068eb84d-workload-certs\") pod \"e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7\" (UID: \"3d6655c6-26fb-4450-a4ac-925e068eb84d\") " pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:43:47.656728 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.656711 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d6655c6-26fb-4450-a4ac-925e068eb84d-istiod-ca-cert\") pod \"e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7\" (UID: \"3d6655c6-26fb-4450-a4ac-925e068eb84d\") " pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:43:47.658267 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.658251 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d6655c6-26fb-4450-a4ac-925e068eb84d-istio-envoy\") pod \"e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7\" (UID: \"3d6655c6-26fb-4450-a4ac-925e068eb84d\") " pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:43:47.658322 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.658272 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d6655c6-26fb-4450-a4ac-925e068eb84d-istio-podinfo\") pod \"e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7\" (UID: \"3d6655c6-26fb-4450-a4ac-925e068eb84d\") " pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:43:47.664649 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.664620 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d6655c6-26fb-4450-a4ac-925e068eb84d-istio-token\") pod \"e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7\" (UID: \"3d6655c6-26fb-4450-a4ac-925e068eb84d\") " pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:43:47.664737 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.664704 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gv9nh\" (UniqueName: \"kubernetes.io/projected/3d6655c6-26fb-4450-a4ac-925e068eb84d-kube-api-access-gv9nh\") pod \"e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7\" (UID: \"3d6655c6-26fb-4450-a4ac-925e068eb84d\") " pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:43:47.769996 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.769871 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:43:47.917020 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.916991 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7"] Jul 22 03:43:47.918508 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:43:47.918481 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d6655c6_26fb_4450_a4ac_925e068eb84d.slice/crio-fcf41e6fd69216c3abd49f6479b8b59aa58d109c30cb4e8f2d6357ea126a214e WatchSource:0}: Error finding container fcf41e6fd69216c3abd49f6479b8b59aa58d109c30cb4e8f2d6357ea126a214e: Status 404 returned error can't find the container with id fcf41e6fd69216c3abd49f6479b8b59aa58d109c30cb4e8f2d6357ea126a214e Jul 22 03:43:47.920786 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.920750 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:43:47.920897 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.920835 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:43:47.920897 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:47.920879 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:43:48.045119 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:48.045032 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:43:48.472500 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:48.472464 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" event={"ID":"3d6655c6-26fb-4450-a4ac-925e068eb84d","Type":"ContainerStarted","Data":"cb7d82d8a87066e779a150d89b1a2a7ae30cf7cb19800be7c22c2300b45fa386"} Jul 22 03:43:48.472712 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:48.472527 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" event={"ID":"3d6655c6-26fb-4450-a4ac-925e068eb84d","Type":"ContainerStarted","Data":"fcf41e6fd69216c3abd49f6479b8b59aa58d109c30cb4e8f2d6357ea126a214e"} Jul 22 03:43:48.495602 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:48.495556 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" podStartSLOduration=1.495541741 podStartE2EDuration="1.495541741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:43:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:43:48.495386403 +0000 UTC m=+2168.944369651" watchObservedRunningTime="2026-07-22 03:43:48.495541741 +0000 UTC m=+2168.944524989" Jul 22 03:43:48.771135 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:48.771037 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:43:48.775991 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:48.775936 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:43:49.481270 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:49.481199 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:43:49.489813 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:49.489775 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:43:50.390291 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:50.390210 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:43:50.493234 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:50.493169 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:43:50.658945 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:50.658866 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:43:50.759924 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:50.759879 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:43:51.222392 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:51.222358 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:43:51.323836 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:51.323803 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:43:54.859969 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:54.859922 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:43:56.436123 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:56.436086 2556 patch_prober.go:28] interesting pod/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 22 03:43:56.436534 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:56.436154 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" podUID="4c38bf61-f5eb-4af7-89b7-f627eb446979" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 22 03:43:56.528688 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:56.528655 2556 generic.go:356] "Generic (PLEG): container finished" podID="4c38bf61-f5eb-4af7-89b7-f627eb446979" containerID="c7b2a6f97fad2ba597d2c023e72af065694217dcdcef078332e7a46c66425668" exitCode=0 Jul 22 03:43:56.528869 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:56.528707 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" event={"ID":"4c38bf61-f5eb-4af7-89b7-f627eb446979","Type":"ContainerDied","Data":"c7b2a6f97fad2ba597d2c023e72af065694217dcdcef078332e7a46c66425668"} Jul 22 03:43:56.529122 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:56.529108 2556 scope.go:117] "RemoveContainer" containerID="c7b2a6f97fad2ba597d2c023e72af065694217dcdcef078332e7a46c66425668" Jul 22 03:43:56.530116 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:56.530086 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:43:56.530183 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:56.530156 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:43:56.537110 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:56.530570 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:43:57.535898 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:57.535859 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" event={"ID":"4c38bf61-f5eb-4af7-89b7-f627eb446979","Type":"ContainerStarted","Data":"3dcc5d4f576a15314530e12ab6565aa2ea93cad433a665441ab43efcbf9f4505"} Jul 22 03:43:58.335364 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:58.335326 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:43:58.335537 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:58.335459 2556 patch_prober.go:28] interesting pod/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 22 03:43:58.335537 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:58.335521 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" podUID="4c38bf61-f5eb-4af7-89b7-f627eb446979" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 22 03:43:59.335644 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:59.335614 2556 patch_prober.go:28] interesting pod/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 22 03:43:59.336048 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:43:59.335664 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" podUID="4c38bf61-f5eb-4af7-89b7-f627eb446979" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 22 03:44:00.335631 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:44:00.335597 2556 patch_prober.go:28] interesting pod/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 22 03:44:00.335824 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:44:00.335653 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" podUID="4c38bf61-f5eb-4af7-89b7-f627eb446979" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 22 03:44:01.335378 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:44:01.335336 2556 patch_prober.go:28] interesting pod/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 22 03:44:01.335540 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:44:01.335393 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" podUID="4c38bf61-f5eb-4af7-89b7-f627eb446979" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 22 03:44:02.339533 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:44:02.339502 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:44:02.339917 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:44:02.339872 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:44:02.340636 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:44:02.340615 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:44:03.575469 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:44:03.575433 2556 generic.go:356] "Generic (PLEG): container finished" podID="3d6655c6-26fb-4450-a4ac-925e068eb84d" containerID="cb7d82d8a87066e779a150d89b1a2a7ae30cf7cb19800be7c22c2300b45fa386" exitCode=0 Jul 22 03:44:03.576077 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:44:03.575507 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" event={"ID":"3d6655c6-26fb-4450-a4ac-925e068eb84d","Type":"ContainerDied","Data":"cb7d82d8a87066e779a150d89b1a2a7ae30cf7cb19800be7c22c2300b45fa386"} Jul 22 03:44:03.576077 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:44:03.576004 2556 scope.go:117] "RemoveContainer" containerID="cb7d82d8a87066e779a150d89b1a2a7ae30cf7cb19800be7c22c2300b45fa386" Jul 22 03:44:03.577194 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:44:03.577161 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:44:03.577287 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:44:03.577244 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:44:03.577346 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:44:03.577289 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:44:03.744299 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:44:03.744266 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:44:04.038292 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:44:04.038209 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:44:04.139843 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:44:04.139809 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:44:04.481426 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:44:04.481325 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:44:04.588155 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:44:04.588112 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" event={"ID":"3d6655c6-26fb-4450-a4ac-925e068eb84d","Type":"ContainerStarted","Data":"5161901a6b27ba7b90547f42e5e99a06ec20a3e853aaad11ef7ce4e2afb164ee"} Jul 22 03:44:04.779900 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:44:04.779384 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:44:04.779900 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:44:04.779512 2556 patch_prober.go:28] interesting pod/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 22 03:44:04.779900 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:44:04.779566 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" podUID="3d6655c6-26fb-4450-a4ac-925e068eb84d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 22 03:44:05.771557 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:44:05.771061 2556 patch_prober.go:28] interesting pod/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 22 03:44:05.771557 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:44:05.771135 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" podUID="3d6655c6-26fb-4450-a4ac-925e068eb84d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 22 03:44:06.770563 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:44:06.770524 2556 patch_prober.go:28] interesting pod/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 22 03:44:06.770886 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:44:06.770854 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" podUID="3d6655c6-26fb-4450-a4ac-925e068eb84d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 22 03:44:07.770938 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:44:07.770897 2556 patch_prober.go:28] interesting pod/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 22 03:44:07.771604 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:44:07.771565 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" podUID="3d6655c6-26fb-4450-a4ac-925e068eb84d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 22 03:44:08.774747 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:44:08.774717 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:44:08.775181 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:44:08.775078 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:44:08.775894 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:44:08.775872 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:44:12.212071 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:44:12.212031 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:44:12.312638 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:44:12.312605 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:44:14.172191 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:44:14.172146 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:44:17.554907 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:44:17.554868 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:44:19.204026 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:44:19.203987 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:44:19.305522 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:44:19.305492 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:44:35.875463 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:44:35.875428 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:44:41.046436 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:44:41.046406 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:45:19.664753 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:19.664663 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:45:21.225455 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:21.225418 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:45:21.324974 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:21.324912 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:45:21.427648 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:21.427610 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:45:25.037920 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:25.037890 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:45:31.271563 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:31.271523 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:45:35.809066 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:35.809025 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:45:36.197336 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:36.197300 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:45:40.736414 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:40.736375 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:45:42.600101 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:42.600045 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:45:42.700172 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:42.700136 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:45:42.858597 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:42.858508 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn"] Jul 22 03:45:42.858759 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:42.858728 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" podUID="4c38bf61-f5eb-4af7-89b7-f627eb446979" containerName="istio-proxy" containerID="cri-o://3dcc5d4f576a15314530e12ab6565aa2ea93cad433a665441ab43efcbf9f4505" gracePeriod=30 Jul 22 03:45:42.884468 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:42.884423 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:45:43.091154 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:43.091118 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:45:43.738194 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:43.738151 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:45:43.838494 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:43.838454 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:45:47.342540 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:47.342501 2556 patch_prober.go:28] interesting pod/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 03:45:47.343032 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:47.342582 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" podUID="4c38bf61-f5eb-4af7-89b7-f627eb446979" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 03:45:48.122015 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.121937 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:45:48.159213 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.159175 2556 generic.go:356] "Generic (PLEG): container finished" podID="4c38bf61-f5eb-4af7-89b7-f627eb446979" containerID="3dcc5d4f576a15314530e12ab6565aa2ea93cad433a665441ab43efcbf9f4505" exitCode=0 Jul 22 03:45:48.159367 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.159219 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" Jul 22 03:45:48.159367 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.159234 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" event={"ID":"4c38bf61-f5eb-4af7-89b7-f627eb446979","Type":"ContainerDied","Data":"3dcc5d4f576a15314530e12ab6565aa2ea93cad433a665441ab43efcbf9f4505"} Jul 22 03:45:48.159367 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.159304 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn" event={"ID":"4c38bf61-f5eb-4af7-89b7-f627eb446979","Type":"ContainerDied","Data":"ead9c66369d930db709cb613e6bafa6caf400b886064fcb0f56e640918aeadad"} Jul 22 03:45:48.159367 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.159337 2556 scope.go:117] "RemoveContainer" containerID="3dcc5d4f576a15314530e12ab6565aa2ea93cad433a665441ab43efcbf9f4505" Jul 22 03:45:48.168399 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.168389 2556 scope.go:117] "RemoveContainer" containerID="c7b2a6f97fad2ba597d2c023e72af065694217dcdcef078332e7a46c66425668" Jul 22 03:45:48.177738 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.177721 2556 scope.go:117] "RemoveContainer" containerID="3dcc5d4f576a15314530e12ab6565aa2ea93cad433a665441ab43efcbf9f4505" Jul 22 03:45:48.178014 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:45:48.177997 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3dcc5d4f576a15314530e12ab6565aa2ea93cad433a665441ab43efcbf9f4505\": container with ID starting with 3dcc5d4f576a15314530e12ab6565aa2ea93cad433a665441ab43efcbf9f4505 not found: ID does not exist" containerID="3dcc5d4f576a15314530e12ab6565aa2ea93cad433a665441ab43efcbf9f4505" Jul 22 03:45:48.178069 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.178024 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3dcc5d4f576a15314530e12ab6565aa2ea93cad433a665441ab43efcbf9f4505"} err="failed to get container status \"3dcc5d4f576a15314530e12ab6565aa2ea93cad433a665441ab43efcbf9f4505\": rpc error: code = NotFound desc = could not find container \"3dcc5d4f576a15314530e12ab6565aa2ea93cad433a665441ab43efcbf9f4505\": container with ID starting with 3dcc5d4f576a15314530e12ab6565aa2ea93cad433a665441ab43efcbf9f4505 not found: ID does not exist" Jul 22 03:45:48.178069 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.178042 2556 scope.go:117] "RemoveContainer" containerID="c7b2a6f97fad2ba597d2c023e72af065694217dcdcef078332e7a46c66425668" Jul 22 03:45:48.178302 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:45:48.178287 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7b2a6f97fad2ba597d2c023e72af065694217dcdcef078332e7a46c66425668\": container with ID starting with c7b2a6f97fad2ba597d2c023e72af065694217dcdcef078332e7a46c66425668 not found: ID does not exist" containerID="c7b2a6f97fad2ba597d2c023e72af065694217dcdcef078332e7a46c66425668" Jul 22 03:45:48.178349 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.178308 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7b2a6f97fad2ba597d2c023e72af065694217dcdcef078332e7a46c66425668"} err="failed to get container status \"c7b2a6f97fad2ba597d2c023e72af065694217dcdcef078332e7a46c66425668\": rpc error: code = NotFound desc = could not find container \"c7b2a6f97fad2ba597d2c023e72af065694217dcdcef078332e7a46c66425668\": container with ID starting with c7b2a6f97fad2ba597d2c023e72af065694217dcdcef078332e7a46c66425668 not found: ID does not exist" Jul 22 03:45:48.209823 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.209794 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47k5n\" (UniqueName: \"kubernetes.io/projected/4c38bf61-f5eb-4af7-89b7-f627eb446979-kube-api-access-47k5n\") pod \"4c38bf61-f5eb-4af7-89b7-f627eb446979\" (UID: \"4c38bf61-f5eb-4af7-89b7-f627eb446979\") " Jul 22 03:45:48.209985 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.209862 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c38bf61-f5eb-4af7-89b7-f627eb446979-credential-socket\") pod \"4c38bf61-f5eb-4af7-89b7-f627eb446979\" (UID: \"4c38bf61-f5eb-4af7-89b7-f627eb446979\") " Jul 22 03:45:48.209985 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.209883 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c38bf61-f5eb-4af7-89b7-f627eb446979-workload-socket\") pod \"4c38bf61-f5eb-4af7-89b7-f627eb446979\" (UID: \"4c38bf61-f5eb-4af7-89b7-f627eb446979\") " Jul 22 03:45:48.209985 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.209913 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c38bf61-f5eb-4af7-89b7-f627eb446979-istio-podinfo\") pod \"4c38bf61-f5eb-4af7-89b7-f627eb446979\" (UID: \"4c38bf61-f5eb-4af7-89b7-f627eb446979\") " Jul 22 03:45:48.209985 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.209935 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c38bf61-f5eb-4af7-89b7-f627eb446979-istio-envoy\") pod \"4c38bf61-f5eb-4af7-89b7-f627eb446979\" (UID: \"4c38bf61-f5eb-4af7-89b7-f627eb446979\") " Jul 22 03:45:48.210209 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.209988 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c38bf61-f5eb-4af7-89b7-f627eb446979-istio-data\") pod \"4c38bf61-f5eb-4af7-89b7-f627eb446979\" (UID: \"4c38bf61-f5eb-4af7-89b7-f627eb446979\") " Jul 22 03:45:48.210209 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.210014 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c38bf61-f5eb-4af7-89b7-f627eb446979-istio-token\") pod \"4c38bf61-f5eb-4af7-89b7-f627eb446979\" (UID: \"4c38bf61-f5eb-4af7-89b7-f627eb446979\") " Jul 22 03:45:48.210209 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.210048 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c38bf61-f5eb-4af7-89b7-f627eb446979-workload-certs\") pod \"4c38bf61-f5eb-4af7-89b7-f627eb446979\" (UID: \"4c38bf61-f5eb-4af7-89b7-f627eb446979\") " Jul 22 03:45:48.210209 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.210089 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c38bf61-f5eb-4af7-89b7-f627eb446979-istiod-ca-cert\") pod \"4c38bf61-f5eb-4af7-89b7-f627eb446979\" (UID: \"4c38bf61-f5eb-4af7-89b7-f627eb446979\") " Jul 22 03:45:48.210427 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.210183 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c38bf61-f5eb-4af7-89b7-f627eb446979-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4c38bf61-f5eb-4af7-89b7-f627eb446979" (UID: "4c38bf61-f5eb-4af7-89b7-f627eb446979"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:45:48.210427 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.210224 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c38bf61-f5eb-4af7-89b7-f627eb446979-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4c38bf61-f5eb-4af7-89b7-f627eb446979" (UID: "4c38bf61-f5eb-4af7-89b7-f627eb446979"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:45:48.210531 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.210437 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c38bf61-f5eb-4af7-89b7-f627eb446979-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4c38bf61-f5eb-4af7-89b7-f627eb446979" (UID: "4c38bf61-f5eb-4af7-89b7-f627eb446979"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:45:48.210531 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.210452 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c38bf61-f5eb-4af7-89b7-f627eb446979-credential-socket\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:45:48.210531 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.210472 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c38bf61-f5eb-4af7-89b7-f627eb446979-workload-socket\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:45:48.210531 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.210492 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c38bf61-f5eb-4af7-89b7-f627eb446979-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4c38bf61-f5eb-4af7-89b7-f627eb446979" (UID: "4c38bf61-f5eb-4af7-89b7-f627eb446979"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:45:48.211106 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.211070 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c38bf61-f5eb-4af7-89b7-f627eb446979-istio-data" (OuterVolumeSpecName: "istio-data") pod "4c38bf61-f5eb-4af7-89b7-f627eb446979" (UID: "4c38bf61-f5eb-4af7-89b7-f627eb446979"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:45:48.212161 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.212135 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c38bf61-f5eb-4af7-89b7-f627eb446979-kube-api-access-47k5n" (OuterVolumeSpecName: "kube-api-access-47k5n") pod "4c38bf61-f5eb-4af7-89b7-f627eb446979" (UID: "4c38bf61-f5eb-4af7-89b7-f627eb446979"). InnerVolumeSpecName "kube-api-access-47k5n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:45:48.212383 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.212356 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c38bf61-f5eb-4af7-89b7-f627eb446979-istio-token" (OuterVolumeSpecName: "istio-token") pod "4c38bf61-f5eb-4af7-89b7-f627eb446979" (UID: "4c38bf61-f5eb-4af7-89b7-f627eb446979"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:45:48.212529 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.212511 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c38bf61-f5eb-4af7-89b7-f627eb446979-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4c38bf61-f5eb-4af7-89b7-f627eb446979" (UID: "4c38bf61-f5eb-4af7-89b7-f627eb446979"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:45:48.212576 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.212518 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4c38bf61-f5eb-4af7-89b7-f627eb446979-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4c38bf61-f5eb-4af7-89b7-f627eb446979" (UID: "4c38bf61-f5eb-4af7-89b7-f627eb446979"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 03:45:48.311399 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.311358 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47k5n\" (UniqueName: \"kubernetes.io/projected/4c38bf61-f5eb-4af7-89b7-f627eb446979-kube-api-access-47k5n\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:45:48.311399 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.311393 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c38bf61-f5eb-4af7-89b7-f627eb446979-istio-podinfo\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:45:48.311611 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.311416 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c38bf61-f5eb-4af7-89b7-f627eb446979-istio-envoy\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:45:48.311611 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.311427 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c38bf61-f5eb-4af7-89b7-f627eb446979-istio-data\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:45:48.311611 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.311437 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c38bf61-f5eb-4af7-89b7-f627eb446979-istio-token\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:45:48.311611 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.311447 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c38bf61-f5eb-4af7-89b7-f627eb446979-workload-certs\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:45:48.311611 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.311457 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c38bf61-f5eb-4af7-89b7-f627eb446979-istiod-ca-cert\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:45:48.482934 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.482896 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn"] Jul 22 03:45:48.486915 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:48.486887 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-44f1c1-openshift-default-7c86ccf799-lpvfn"] Jul 22 03:45:50.024510 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:50.024479 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c38bf61-f5eb-4af7-89b7-f627eb446979" path="/var/lib/kubelet/pods/4c38bf61-f5eb-4af7-89b7-f627eb446979/volumes" Jul 22 03:45:53.824883 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:53.824848 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:45:56.834132 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:56.834096 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:45:58.285624 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:58.285585 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:45:58.653003 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:45:58.652940 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:04.605845 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:04.605797 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:04.709822 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:04.709781 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:04.864978 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:04.864880 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7"] Jul 22 03:46:04.865164 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:04.865117 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" podUID="3d6655c6-26fb-4450-a4ac-925e068eb84d" containerName="istio-proxy" containerID="cri-o://5161901a6b27ba7b90547f42e5e99a06ec20a3e853aaad11ef7ce4e2afb164ee" gracePeriod=30 Jul 22 03:46:04.887070 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:04.887035 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:05.118068 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.117975 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:05.451302 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.451213 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:05.749967 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.749870 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x"] Jul 22 03:46:05.750469 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.750450 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c38bf61-f5eb-4af7-89b7-f627eb446979" containerName="istio-proxy" Jul 22 03:46:05.750553 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.750472 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c38bf61-f5eb-4af7-89b7-f627eb446979" containerName="istio-proxy" Jul 22 03:46:05.750553 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.750487 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c38bf61-f5eb-4af7-89b7-f627eb446979" containerName="istio-proxy" Jul 22 03:46:05.750553 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.750495 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c38bf61-f5eb-4af7-89b7-f627eb446979" containerName="istio-proxy" Jul 22 03:46:05.750702 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.750600 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c38bf61-f5eb-4af7-89b7-f627eb446979" containerName="istio-proxy" Jul 22 03:46:05.755316 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.755296 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:46:05.757768 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.757746 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-7a2eef-openshift-default-dockercfg-dv2cs\"" Jul 22 03:46:05.766353 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.766329 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x"] Jul 22 03:46:05.781422 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.781377 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:05.882506 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.882465 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f819a251-38f7-456e-9f36-ed830f8e1c53-istio-token\") pod \"e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x\" (UID: \"f819a251-38f7-456e-9f36-ed830f8e1c53\") " pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:46:05.882709 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.882530 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f819a251-38f7-456e-9f36-ed830f8e1c53-istio-data\") pod \"e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x\" (UID: \"f819a251-38f7-456e-9f36-ed830f8e1c53\") " pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:46:05.882709 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.882560 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f819a251-38f7-456e-9f36-ed830f8e1c53-credential-socket\") pod \"e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x\" (UID: \"f819a251-38f7-456e-9f36-ed830f8e1c53\") " pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:46:05.882709 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.882603 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f819a251-38f7-456e-9f36-ed830f8e1c53-workload-socket\") pod \"e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x\" (UID: \"f819a251-38f7-456e-9f36-ed830f8e1c53\") " pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:46:05.882709 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.882628 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f819a251-38f7-456e-9f36-ed830f8e1c53-workload-certs\") pod \"e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x\" (UID: \"f819a251-38f7-456e-9f36-ed830f8e1c53\") " pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:46:05.882709 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.882669 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bw7p\" (UniqueName: \"kubernetes.io/projected/f819a251-38f7-456e-9f36-ed830f8e1c53-kube-api-access-8bw7p\") pod \"e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x\" (UID: \"f819a251-38f7-456e-9f36-ed830f8e1c53\") " pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:46:05.882941 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.882715 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f819a251-38f7-456e-9f36-ed830f8e1c53-istiod-ca-cert\") pod \"e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x\" (UID: \"f819a251-38f7-456e-9f36-ed830f8e1c53\") " pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:46:05.882941 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.882771 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f819a251-38f7-456e-9f36-ed830f8e1c53-istio-envoy\") pod \"e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x\" (UID: \"f819a251-38f7-456e-9f36-ed830f8e1c53\") " pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:46:05.882941 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.882794 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f819a251-38f7-456e-9f36-ed830f8e1c53-istio-podinfo\") pod \"e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x\" (UID: \"f819a251-38f7-456e-9f36-ed830f8e1c53\") " pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:46:05.983407 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.983377 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f819a251-38f7-456e-9f36-ed830f8e1c53-istio-token\") pod \"e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x\" (UID: \"f819a251-38f7-456e-9f36-ed830f8e1c53\") " pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:46:05.983573 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.983438 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f819a251-38f7-456e-9f36-ed830f8e1c53-istio-data\") pod \"e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x\" (UID: \"f819a251-38f7-456e-9f36-ed830f8e1c53\") " pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:46:05.983573 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.983467 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f819a251-38f7-456e-9f36-ed830f8e1c53-credential-socket\") pod \"e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x\" (UID: \"f819a251-38f7-456e-9f36-ed830f8e1c53\") " pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:46:05.983573 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.983494 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f819a251-38f7-456e-9f36-ed830f8e1c53-workload-socket\") pod \"e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x\" (UID: \"f819a251-38f7-456e-9f36-ed830f8e1c53\") " pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:46:05.983573 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.983510 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f819a251-38f7-456e-9f36-ed830f8e1c53-workload-certs\") pod \"e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x\" (UID: \"f819a251-38f7-456e-9f36-ed830f8e1c53\") " pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:46:05.983573 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.983539 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bw7p\" (UniqueName: \"kubernetes.io/projected/f819a251-38f7-456e-9f36-ed830f8e1c53-kube-api-access-8bw7p\") pod \"e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x\" (UID: \"f819a251-38f7-456e-9f36-ed830f8e1c53\") " pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:46:05.983841 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.983584 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f819a251-38f7-456e-9f36-ed830f8e1c53-istiod-ca-cert\") pod \"e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x\" (UID: \"f819a251-38f7-456e-9f36-ed830f8e1c53\") " pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:46:05.983841 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.983824 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f819a251-38f7-456e-9f36-ed830f8e1c53-istio-envoy\") pod \"e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x\" (UID: \"f819a251-38f7-456e-9f36-ed830f8e1c53\") " pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:46:05.983986 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.983862 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f819a251-38f7-456e-9f36-ed830f8e1c53-istio-podinfo\") pod \"e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x\" (UID: \"f819a251-38f7-456e-9f36-ed830f8e1c53\") " pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:46:05.983986 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.983923 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f819a251-38f7-456e-9f36-ed830f8e1c53-workload-certs\") pod \"e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x\" (UID: \"f819a251-38f7-456e-9f36-ed830f8e1c53\") " pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:46:05.984117 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.983988 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f819a251-38f7-456e-9f36-ed830f8e1c53-credential-socket\") pod \"e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x\" (UID: \"f819a251-38f7-456e-9f36-ed830f8e1c53\") " pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:46:05.984117 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.984082 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f819a251-38f7-456e-9f36-ed830f8e1c53-istio-data\") pod \"e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x\" (UID: \"f819a251-38f7-456e-9f36-ed830f8e1c53\") " pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:46:05.984117 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.984105 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f819a251-38f7-456e-9f36-ed830f8e1c53-workload-socket\") pod \"e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x\" (UID: \"f819a251-38f7-456e-9f36-ed830f8e1c53\") " pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:46:05.984338 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.984320 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f819a251-38f7-456e-9f36-ed830f8e1c53-istiod-ca-cert\") pod \"e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x\" (UID: \"f819a251-38f7-456e-9f36-ed830f8e1c53\") " pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:46:05.985966 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.985929 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f819a251-38f7-456e-9f36-ed830f8e1c53-istio-envoy\") pod \"e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x\" (UID: \"f819a251-38f7-456e-9f36-ed830f8e1c53\") " pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:46:05.986330 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.986315 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f819a251-38f7-456e-9f36-ed830f8e1c53-istio-podinfo\") pod \"e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x\" (UID: \"f819a251-38f7-456e-9f36-ed830f8e1c53\") " pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:46:05.999122 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.992408 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:05.999122 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.993263 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bw7p\" (UniqueName: \"kubernetes.io/projected/f819a251-38f7-456e-9f36-ed830f8e1c53-kube-api-access-8bw7p\") pod \"e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x\" (UID: \"f819a251-38f7-456e-9f36-ed830f8e1c53\") " pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:46:05.999122 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:05.995794 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f819a251-38f7-456e-9f36-ed830f8e1c53-istio-token\") pod \"e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x\" (UID: \"f819a251-38f7-456e-9f36-ed830f8e1c53\") " pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:46:06.066794 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:06.066714 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:46:06.208846 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:46:06.208815 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf819a251_38f7_456e_9f36_ed830f8e1c53.slice/crio-79478cb55e84a84080a98f82ea6f68d41b550d6cdbd6e30508c3a482f300a95c WatchSource:0}: Error finding container 79478cb55e84a84080a98f82ea6f68d41b550d6cdbd6e30508c3a482f300a95c: Status 404 returned error can't find the container with id 79478cb55e84a84080a98f82ea6f68d41b550d6cdbd6e30508c3a482f300a95c Jul 22 03:46:06.208846 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:06.208837 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x"] Jul 22 03:46:06.211041 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:06.210999 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:46:06.211163 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:06.211089 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:46:06.211163 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:06.211147 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:46:06.259077 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:06.259049 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" event={"ID":"f819a251-38f7-456e-9f36-ed830f8e1c53","Type":"ContainerStarted","Data":"79478cb55e84a84080a98f82ea6f68d41b550d6cdbd6e30508c3a482f300a95c"} Jul 22 03:46:07.265698 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:07.265659 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" event={"ID":"f819a251-38f7-456e-9f36-ed830f8e1c53","Type":"ContainerStarted","Data":"61188f0c27d51a97caf5a4a62e1fa725004d003e8f3310d1e4b98efc91656c41"} Jul 22 03:46:07.293545 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:07.293483 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" podStartSLOduration=2.29346395 podStartE2EDuration="2.29346395s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:46:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:46:07.292714009 +0000 UTC m=+2307.741697301" watchObservedRunningTime="2026-07-22 03:46:07.29346395 +0000 UTC m=+2307.742447199" Jul 22 03:46:08.067856 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:08.067824 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:46:08.072886 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:08.072859 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:46:08.272276 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:08.272242 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:46:08.273134 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:08.273112 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:46:08.777388 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:08.777350 2556 patch_prober.go:28] interesting pod/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 03:46:08.777580 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:08.777405 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" podUID="3d6655c6-26fb-4450-a4ac-925e068eb84d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 03:46:09.199186 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:09.199149 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:09.296181 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:09.296150 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:09.407308 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:09.407271 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:09.865089 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:09.865054 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:09.969429 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:09.969396 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:10.132914 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.132887 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:46:10.287341 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.287303 2556 generic.go:356] "Generic (PLEG): container finished" podID="3d6655c6-26fb-4450-a4ac-925e068eb84d" containerID="5161901a6b27ba7b90547f42e5e99a06ec20a3e853aaad11ef7ce4e2afb164ee" exitCode=0 Jul 22 03:46:10.287522 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.287375 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" Jul 22 03:46:10.287522 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.287391 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" event={"ID":"3d6655c6-26fb-4450-a4ac-925e068eb84d","Type":"ContainerDied","Data":"5161901a6b27ba7b90547f42e5e99a06ec20a3e853aaad11ef7ce4e2afb164ee"} Jul 22 03:46:10.287522 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.287447 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7" event={"ID":"3d6655c6-26fb-4450-a4ac-925e068eb84d","Type":"ContainerDied","Data":"fcf41e6fd69216c3abd49f6479b8b59aa58d109c30cb4e8f2d6357ea126a214e"} Jul 22 03:46:10.287522 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.287471 2556 scope.go:117] "RemoveContainer" containerID="5161901a6b27ba7b90547f42e5e99a06ec20a3e853aaad11ef7ce4e2afb164ee" Jul 22 03:46:10.299686 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.299661 2556 scope.go:117] "RemoveContainer" containerID="cb7d82d8a87066e779a150d89b1a2a7ae30cf7cb19800be7c22c2300b45fa386" Jul 22 03:46:10.308011 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.307936 2556 scope.go:117] "RemoveContainer" containerID="5161901a6b27ba7b90547f42e5e99a06ec20a3e853aaad11ef7ce4e2afb164ee" Jul 22 03:46:10.308218 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:46:10.308200 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5161901a6b27ba7b90547f42e5e99a06ec20a3e853aaad11ef7ce4e2afb164ee\": container with ID starting with 5161901a6b27ba7b90547f42e5e99a06ec20a3e853aaad11ef7ce4e2afb164ee not found: ID does not exist" containerID="5161901a6b27ba7b90547f42e5e99a06ec20a3e853aaad11ef7ce4e2afb164ee" Jul 22 03:46:10.308272 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.308231 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5161901a6b27ba7b90547f42e5e99a06ec20a3e853aaad11ef7ce4e2afb164ee"} err="failed to get container status \"5161901a6b27ba7b90547f42e5e99a06ec20a3e853aaad11ef7ce4e2afb164ee\": rpc error: code = NotFound desc = could not find container \"5161901a6b27ba7b90547f42e5e99a06ec20a3e853aaad11ef7ce4e2afb164ee\": container with ID starting with 5161901a6b27ba7b90547f42e5e99a06ec20a3e853aaad11ef7ce4e2afb164ee not found: ID does not exist" Jul 22 03:46:10.308272 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.308257 2556 scope.go:117] "RemoveContainer" containerID="cb7d82d8a87066e779a150d89b1a2a7ae30cf7cb19800be7c22c2300b45fa386" Jul 22 03:46:10.308463 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:46:10.308445 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb7d82d8a87066e779a150d89b1a2a7ae30cf7cb19800be7c22c2300b45fa386\": container with ID starting with cb7d82d8a87066e779a150d89b1a2a7ae30cf7cb19800be7c22c2300b45fa386 not found: ID does not exist" containerID="cb7d82d8a87066e779a150d89b1a2a7ae30cf7cb19800be7c22c2300b45fa386" Jul 22 03:46:10.308520 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.308471 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb7d82d8a87066e779a150d89b1a2a7ae30cf7cb19800be7c22c2300b45fa386"} err="failed to get container status \"cb7d82d8a87066e779a150d89b1a2a7ae30cf7cb19800be7c22c2300b45fa386\": rpc error: code = NotFound desc = could not find container \"cb7d82d8a87066e779a150d89b1a2a7ae30cf7cb19800be7c22c2300b45fa386\": container with ID starting with cb7d82d8a87066e779a150d89b1a2a7ae30cf7cb19800be7c22c2300b45fa386 not found: ID does not exist" Jul 22 03:46:10.324869 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.324846 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gv9nh\" (UniqueName: \"kubernetes.io/projected/3d6655c6-26fb-4450-a4ac-925e068eb84d-kube-api-access-gv9nh\") pod \"3d6655c6-26fb-4450-a4ac-925e068eb84d\" (UID: \"3d6655c6-26fb-4450-a4ac-925e068eb84d\") " Jul 22 03:46:10.325004 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.324884 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d6655c6-26fb-4450-a4ac-925e068eb84d-istio-envoy\") pod \"3d6655c6-26fb-4450-a4ac-925e068eb84d\" (UID: \"3d6655c6-26fb-4450-a4ac-925e068eb84d\") " Jul 22 03:46:10.325004 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.324916 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d6655c6-26fb-4450-a4ac-925e068eb84d-credential-socket\") pod \"3d6655c6-26fb-4450-a4ac-925e068eb84d\" (UID: \"3d6655c6-26fb-4450-a4ac-925e068eb84d\") " Jul 22 03:46:10.325004 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.324943 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d6655c6-26fb-4450-a4ac-925e068eb84d-workload-certs\") pod \"3d6655c6-26fb-4450-a4ac-925e068eb84d\" (UID: \"3d6655c6-26fb-4450-a4ac-925e068eb84d\") " Jul 22 03:46:10.325004 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.324991 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d6655c6-26fb-4450-a4ac-925e068eb84d-workload-socket\") pod \"3d6655c6-26fb-4450-a4ac-925e068eb84d\" (UID: \"3d6655c6-26fb-4450-a4ac-925e068eb84d\") " Jul 22 03:46:10.325218 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.325070 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d6655c6-26fb-4450-a4ac-925e068eb84d-istio-podinfo\") pod \"3d6655c6-26fb-4450-a4ac-925e068eb84d\" (UID: \"3d6655c6-26fb-4450-a4ac-925e068eb84d\") " Jul 22 03:46:10.325218 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.325155 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d6655c6-26fb-4450-a4ac-925e068eb84d-istiod-ca-cert\") pod \"3d6655c6-26fb-4450-a4ac-925e068eb84d\" (UID: \"3d6655c6-26fb-4450-a4ac-925e068eb84d\") " Jul 22 03:46:10.325218 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.325179 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d6655c6-26fb-4450-a4ac-925e068eb84d-istio-data\") pod \"3d6655c6-26fb-4450-a4ac-925e068eb84d\" (UID: \"3d6655c6-26fb-4450-a4ac-925e068eb84d\") " Jul 22 03:46:10.325359 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.325225 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d6655c6-26fb-4450-a4ac-925e068eb84d-istio-token\") pod \"3d6655c6-26fb-4450-a4ac-925e068eb84d\" (UID: \"3d6655c6-26fb-4450-a4ac-925e068eb84d\") " Jul 22 03:46:10.325359 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.325243 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d6655c6-26fb-4450-a4ac-925e068eb84d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3d6655c6-26fb-4450-a4ac-925e068eb84d" (UID: "3d6655c6-26fb-4450-a4ac-925e068eb84d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:46:10.325359 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.325297 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d6655c6-26fb-4450-a4ac-925e068eb84d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3d6655c6-26fb-4450-a4ac-925e068eb84d" (UID: "3d6655c6-26fb-4450-a4ac-925e068eb84d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:46:10.325563 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.325542 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d6655c6-26fb-4450-a4ac-925e068eb84d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3d6655c6-26fb-4450-a4ac-925e068eb84d" (UID: "3d6655c6-26fb-4450-a4ac-925e068eb84d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:46:10.325617 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.325553 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d6655c6-26fb-4450-a4ac-925e068eb84d-workload-socket\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:46:10.325617 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.325591 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d6655c6-26fb-4450-a4ac-925e068eb84d-workload-certs\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:46:10.325723 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.325644 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d6655c6-26fb-4450-a4ac-925e068eb84d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3d6655c6-26fb-4450-a4ac-925e068eb84d" (UID: "3d6655c6-26fb-4450-a4ac-925e068eb84d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:46:10.326159 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.326134 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d6655c6-26fb-4450-a4ac-925e068eb84d-istio-data" (OuterVolumeSpecName: "istio-data") pod "3d6655c6-26fb-4450-a4ac-925e068eb84d" (UID: "3d6655c6-26fb-4450-a4ac-925e068eb84d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:46:10.327252 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.327220 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d6655c6-26fb-4450-a4ac-925e068eb84d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3d6655c6-26fb-4450-a4ac-925e068eb84d" (UID: "3d6655c6-26fb-4450-a4ac-925e068eb84d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:46:10.327337 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.327265 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d6655c6-26fb-4450-a4ac-925e068eb84d-kube-api-access-gv9nh" (OuterVolumeSpecName: "kube-api-access-gv9nh") pod "3d6655c6-26fb-4450-a4ac-925e068eb84d" (UID: "3d6655c6-26fb-4450-a4ac-925e068eb84d"). InnerVolumeSpecName "kube-api-access-gv9nh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:46:10.327337 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.327289 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3d6655c6-26fb-4450-a4ac-925e068eb84d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3d6655c6-26fb-4450-a4ac-925e068eb84d" (UID: "3d6655c6-26fb-4450-a4ac-925e068eb84d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 03:46:10.327434 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.327376 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d6655c6-26fb-4450-a4ac-925e068eb84d-istio-token" (OuterVolumeSpecName: "istio-token") pod "3d6655c6-26fb-4450-a4ac-925e068eb84d" (UID: "3d6655c6-26fb-4450-a4ac-925e068eb84d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:46:10.344321 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.344294 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:10.427364 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.427249 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d6655c6-26fb-4450-a4ac-925e068eb84d-istiod-ca-cert\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:46:10.427364 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.427295 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d6655c6-26fb-4450-a4ac-925e068eb84d-istio-data\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:46:10.427364 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.427308 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d6655c6-26fb-4450-a4ac-925e068eb84d-istio-token\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:46:10.427364 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.427330 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gv9nh\" (UniqueName: \"kubernetes.io/projected/3d6655c6-26fb-4450-a4ac-925e068eb84d-kube-api-access-gv9nh\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:46:10.427364 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.427347 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d6655c6-26fb-4450-a4ac-925e068eb84d-istio-envoy\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:46:10.427364 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.427361 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d6655c6-26fb-4450-a4ac-925e068eb84d-credential-socket\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:46:10.427775 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.427373 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d6655c6-26fb-4450-a4ac-925e068eb84d-istio-podinfo\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:46:10.613699 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.613670 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7"] Jul 22 03:46:10.618484 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:10.618457 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-4c93e4-openshift-default-7967d58459-5c7k7"] Jul 22 03:46:12.023892 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:12.023860 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d6655c6-26fb-4450-a4ac-925e068eb84d" path="/var/lib/kubelet/pods/3d6655c6-26fb-4450-a4ac-925e068eb84d/volumes" Jul 22 03:46:12.261205 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:12.261167 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:13.666920 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:13.666885 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:14.137933 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.137888 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:14.418347 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.418264 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8"] Jul 22 03:46:14.418904 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.418882 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d6655c6-26fb-4450-a4ac-925e068eb84d" containerName="istio-proxy" Jul 22 03:46:14.419007 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.418908 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d6655c6-26fb-4450-a4ac-925e068eb84d" containerName="istio-proxy" Jul 22 03:46:14.419007 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.418926 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d6655c6-26fb-4450-a4ac-925e068eb84d" containerName="istio-proxy" Jul 22 03:46:14.419007 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.418935 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d6655c6-26fb-4450-a4ac-925e068eb84d" containerName="istio-proxy" Jul 22 03:46:14.419128 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.419102 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d6655c6-26fb-4450-a4ac-925e068eb84d" containerName="istio-proxy" Jul 22 03:46:14.419128 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.419116 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d6655c6-26fb-4450-a4ac-925e068eb84d" containerName="istio-proxy" Jul 22 03:46:14.419128 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.419125 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c38bf61-f5eb-4af7-89b7-f627eb446979" containerName="istio-proxy" Jul 22 03:46:14.422666 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.422643 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:46:14.427830 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.427813 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-92a8f6-openshift-default-dockercfg-gbxhq\"" Jul 22 03:46:14.443644 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.443619 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8"] Jul 22 03:46:14.459991 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.459942 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e52ba23-75cf-4789-af93-b2b561de956e-istio-envoy\") pod \"e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8\" (UID: \"2e52ba23-75cf-4789-af93-b2b561de956e\") " pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:46:14.460081 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.460000 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e52ba23-75cf-4789-af93-b2b561de956e-workload-socket\") pod \"e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8\" (UID: \"2e52ba23-75cf-4789-af93-b2b561de956e\") " pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:46:14.460081 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.460044 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e52ba23-75cf-4789-af93-b2b561de956e-istio-token\") pod \"e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8\" (UID: \"2e52ba23-75cf-4789-af93-b2b561de956e\") " pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:46:14.460081 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.460060 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e52ba23-75cf-4789-af93-b2b561de956e-credential-socket\") pod \"e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8\" (UID: \"2e52ba23-75cf-4789-af93-b2b561de956e\") " pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:46:14.460081 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.460075 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e52ba23-75cf-4789-af93-b2b561de956e-istiod-ca-cert\") pod \"e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8\" (UID: \"2e52ba23-75cf-4789-af93-b2b561de956e\") " pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:46:14.460269 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.460167 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e52ba23-75cf-4789-af93-b2b561de956e-istio-podinfo\") pod \"e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8\" (UID: \"2e52ba23-75cf-4789-af93-b2b561de956e\") " pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:46:14.460269 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.460195 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e52ba23-75cf-4789-af93-b2b561de956e-workload-certs\") pod \"e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8\" (UID: \"2e52ba23-75cf-4789-af93-b2b561de956e\") " pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:46:14.460269 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.460217 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5ljh\" (UniqueName: \"kubernetes.io/projected/2e52ba23-75cf-4789-af93-b2b561de956e-kube-api-access-x5ljh\") pod \"e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8\" (UID: \"2e52ba23-75cf-4789-af93-b2b561de956e\") " pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:46:14.460363 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.460334 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e52ba23-75cf-4789-af93-b2b561de956e-istio-data\") pod \"e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8\" (UID: \"2e52ba23-75cf-4789-af93-b2b561de956e\") " pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:46:14.467454 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.467420 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:14.561902 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.561864 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e52ba23-75cf-4789-af93-b2b561de956e-istio-envoy\") pod \"e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8\" (UID: \"2e52ba23-75cf-4789-af93-b2b561de956e\") " pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:46:14.562086 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.561909 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e52ba23-75cf-4789-af93-b2b561de956e-workload-socket\") pod \"e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8\" (UID: \"2e52ba23-75cf-4789-af93-b2b561de956e\") " pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:46:14.562086 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.561941 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e52ba23-75cf-4789-af93-b2b561de956e-istio-token\") pod \"e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8\" (UID: \"2e52ba23-75cf-4789-af93-b2b561de956e\") " pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:46:14.562086 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.561995 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e52ba23-75cf-4789-af93-b2b561de956e-credential-socket\") pod \"e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8\" (UID: \"2e52ba23-75cf-4789-af93-b2b561de956e\") " pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:46:14.562086 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.562019 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e52ba23-75cf-4789-af93-b2b561de956e-istiod-ca-cert\") pod \"e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8\" (UID: \"2e52ba23-75cf-4789-af93-b2b561de956e\") " pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:46:14.562086 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.562068 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e52ba23-75cf-4789-af93-b2b561de956e-istio-podinfo\") pod \"e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8\" (UID: \"2e52ba23-75cf-4789-af93-b2b561de956e\") " pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:46:14.562380 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.562093 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e52ba23-75cf-4789-af93-b2b561de956e-workload-certs\") pod \"e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8\" (UID: \"2e52ba23-75cf-4789-af93-b2b561de956e\") " pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:46:14.562380 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.562121 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5ljh\" (UniqueName: \"kubernetes.io/projected/2e52ba23-75cf-4789-af93-b2b561de956e-kube-api-access-x5ljh\") pod \"e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8\" (UID: \"2e52ba23-75cf-4789-af93-b2b561de956e\") " pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:46:14.562380 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.562182 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e52ba23-75cf-4789-af93-b2b561de956e-istio-data\") pod \"e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8\" (UID: \"2e52ba23-75cf-4789-af93-b2b561de956e\") " pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:46:14.562380 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.562338 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e52ba23-75cf-4789-af93-b2b561de956e-workload-socket\") pod \"e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8\" (UID: \"2e52ba23-75cf-4789-af93-b2b561de956e\") " pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:46:14.562606 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.562385 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e52ba23-75cf-4789-af93-b2b561de956e-credential-socket\") pod \"e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8\" (UID: \"2e52ba23-75cf-4789-af93-b2b561de956e\") " pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:46:14.562606 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.562595 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e52ba23-75cf-4789-af93-b2b561de956e-workload-certs\") pod \"e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8\" (UID: \"2e52ba23-75cf-4789-af93-b2b561de956e\") " pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:46:14.562791 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.562771 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e52ba23-75cf-4789-af93-b2b561de956e-istio-data\") pod \"e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8\" (UID: \"2e52ba23-75cf-4789-af93-b2b561de956e\") " pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:46:14.562849 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.562783 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e52ba23-75cf-4789-af93-b2b561de956e-istiod-ca-cert\") pod \"e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8\" (UID: \"2e52ba23-75cf-4789-af93-b2b561de956e\") " pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:46:14.564480 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.564459 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e52ba23-75cf-4789-af93-b2b561de956e-istio-envoy\") pod \"e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8\" (UID: \"2e52ba23-75cf-4789-af93-b2b561de956e\") " pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:46:14.564664 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.564647 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e52ba23-75cf-4789-af93-b2b561de956e-istio-podinfo\") pod \"e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8\" (UID: \"2e52ba23-75cf-4789-af93-b2b561de956e\") " pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:46:14.571535 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.571512 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5ljh\" (UniqueName: \"kubernetes.io/projected/2e52ba23-75cf-4789-af93-b2b561de956e-kube-api-access-x5ljh\") pod \"e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8\" (UID: \"2e52ba23-75cf-4789-af93-b2b561de956e\") " pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:46:14.573065 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.573043 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e52ba23-75cf-4789-af93-b2b561de956e-istio-token\") pod \"e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8\" (UID: \"2e52ba23-75cf-4789-af93-b2b561de956e\") " pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:46:14.733238 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.733158 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:46:14.880266 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.880239 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8"] Jul 22 03:46:14.881280 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:46:14.881256 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e52ba23_75cf_4789_af93_b2b561de956e.slice/crio-c0f478881fc0f3a1a9d6249be85a01ef379125ad1a5f4af3b722525eed66c4ac WatchSource:0}: Error finding container c0f478881fc0f3a1a9d6249be85a01ef379125ad1a5f4af3b722525eed66c4ac: Status 404 returned error can't find the container with id c0f478881fc0f3a1a9d6249be85a01ef379125ad1a5f4af3b722525eed66c4ac Jul 22 03:46:14.883729 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.883697 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:46:14.883827 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.883758 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:46:14.883827 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:14.883787 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:46:15.199609 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:15.199581 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:15.298167 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:15.298132 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:15.316512 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:15.316476 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" event={"ID":"2e52ba23-75cf-4789-af93-b2b561de956e","Type":"ContainerStarted","Data":"c41de0484b21588bff4c8d9a84f457b7c0d61d0cde3909589532a9e6745a2ffd"} Jul 22 03:46:15.316646 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:15.316538 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" event={"ID":"2e52ba23-75cf-4789-af93-b2b561de956e","Type":"ContainerStarted","Data":"c0f478881fc0f3a1a9d6249be85a01ef379125ad1a5f4af3b722525eed66c4ac"} Jul 22 03:46:15.338620 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:15.338553 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" podStartSLOduration=1.3385329910000001 podStartE2EDuration="1.338532991s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:46:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:46:15.338236753 +0000 UTC m=+2315.787220002" watchObservedRunningTime="2026-07-22 03:46:15.338532991 +0000 UTC m=+2315.787516240" Jul 22 03:46:15.734025 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:15.733986 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:46:15.739074 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:15.739049 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:46:16.324762 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:16.324727 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:46:16.325581 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:16.325564 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:46:16.562250 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:16.562217 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:46:17.294195 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:17.294146 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:17.558992 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:17.558852 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:17.660365 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:17.660329 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:17.802328 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:17.802294 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:18.017762 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:18.017726 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:18.117472 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:18.117435 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:21.707074 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:21.707039 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:24.314116 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:24.314075 2556 patch_prober.go:28] interesting pod/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 22 03:46:24.314605 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:24.314149 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" podUID="f819a251-38f7-456e-9f36-ed830f8e1c53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 22 03:46:25.377260 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:25.377224 2556 generic.go:356] "Generic (PLEG): container finished" podID="f819a251-38f7-456e-9f36-ed830f8e1c53" containerID="61188f0c27d51a97caf5a4a62e1fa725004d003e8f3310d1e4b98efc91656c41" exitCode=0 Jul 22 03:46:25.377692 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:25.377303 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" event={"ID":"f819a251-38f7-456e-9f36-ed830f8e1c53","Type":"ContainerDied","Data":"61188f0c27d51a97caf5a4a62e1fa725004d003e8f3310d1e4b98efc91656c41"} Jul 22 03:46:25.377839 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:25.377823 2556 scope.go:117] "RemoveContainer" containerID="61188f0c27d51a97caf5a4a62e1fa725004d003e8f3310d1e4b98efc91656c41" Jul 22 03:46:25.379009 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:25.378977 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:46:25.379091 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:25.379063 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:46:25.379155 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:25.379140 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:46:26.388861 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:26.388826 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" event={"ID":"f819a251-38f7-456e-9f36-ed830f8e1c53","Type":"ContainerStarted","Data":"376e4b6a2067aac909fa41f067f0fbb3c8acdba8e5a7f217a46da0ba2cd69c9a"} Jul 22 03:46:27.067274 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:27.067241 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:46:27.067466 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:27.067332 2556 patch_prober.go:28] interesting pod/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Jul 22 03:46:27.067466 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:27.067364 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" podUID="f819a251-38f7-456e-9f36-ed830f8e1c53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Jul 22 03:46:28.067519 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:28.067475 2556 patch_prober.go:28] interesting pod/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Jul 22 03:46:28.067899 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:28.067550 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" podUID="f819a251-38f7-456e-9f36-ed830f8e1c53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Jul 22 03:46:29.067926 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:29.067887 2556 patch_prober.go:28] interesting pod/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Jul 22 03:46:29.068340 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:29.067947 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" podUID="f819a251-38f7-456e-9f36-ed830f8e1c53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Jul 22 03:46:30.067281 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:30.067202 2556 patch_prober.go:28] interesting pod/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Jul 22 03:46:30.067281 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:30.067257 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" podUID="f819a251-38f7-456e-9f36-ed830f8e1c53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Jul 22 03:46:30.237802 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:30.237756 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:30.340942 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:30.340906 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:31.070607 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:31.070577 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:46:31.070960 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:31.070927 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:46:31.071658 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:31.071638 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:46:32.366142 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:32.366094 2556 patch_prober.go:28] interesting pod/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 22 03:46:32.366635 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:32.366162 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" podUID="2e52ba23-75cf-4789-af93-b2b561de956e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 22 03:46:34.445490 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:34.445455 2556 generic.go:356] "Generic (PLEG): container finished" podID="2e52ba23-75cf-4789-af93-b2b561de956e" containerID="c41de0484b21588bff4c8d9a84f457b7c0d61d0cde3909589532a9e6745a2ffd" exitCode=0 Jul 22 03:46:34.445996 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:34.445526 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" event={"ID":"2e52ba23-75cf-4789-af93-b2b561de956e","Type":"ContainerDied","Data":"c41de0484b21588bff4c8d9a84f457b7c0d61d0cde3909589532a9e6745a2ffd"} Jul 22 03:46:34.445996 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:34.445909 2556 scope.go:117] "RemoveContainer" containerID="c41de0484b21588bff4c8d9a84f457b7c0d61d0cde3909589532a9e6745a2ffd" Jul 22 03:46:34.446906 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:34.446879 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:46:34.446979 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:34.446947 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:46:34.447022 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:34.446995 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:46:34.827156 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:34.827122 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:46:35.453690 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:35.453652 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" event={"ID":"2e52ba23-75cf-4789-af93-b2b561de956e","Type":"ContainerStarted","Data":"27112a35d26ed38820962163c803ef88410c13f9a3f9d08d689dc47b18d6940c"} Jul 22 03:46:35.733623 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:35.733542 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:46:35.733848 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:35.733750 2556 patch_prober.go:28] interesting pod/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Jul 22 03:46:35.733848 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:35.733826 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" podUID="2e52ba23-75cf-4789-af93-b2b561de956e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Jul 22 03:46:36.734360 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:36.734326 2556 patch_prober.go:28] interesting pod/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Jul 22 03:46:36.734739 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:36.734382 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" podUID="2e52ba23-75cf-4789-af93-b2b561de956e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Jul 22 03:46:37.733763 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:37.733732 2556 patch_prober.go:28] interesting pod/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Jul 22 03:46:37.733971 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:37.733789 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" podUID="2e52ba23-75cf-4789-af93-b2b561de956e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Jul 22 03:46:38.733697 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:38.733658 2556 patch_prober.go:28] interesting pod/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Jul 22 03:46:38.734194 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:38.733727 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" podUID="2e52ba23-75cf-4789-af93-b2b561de956e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Jul 22 03:46:38.796433 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:38.796395 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:38.894208 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:38.894154 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:39.737478 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:39.737452 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:46:39.737936 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:39.737733 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:46:39.738403 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:39.738382 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:46:40.212090 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:40.212055 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:40.313390 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:40.313355 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:45.472563 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:45.472504 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:49.661912 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:49.661877 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:50.056587 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:50.056488 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:50.648230 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:50.648199 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:46:55.842399 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:55.842338 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:55.942154 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:55.942111 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:56.086383 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:56.086350 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x"] Jul 22 03:46:56.086622 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:56.086595 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" podUID="f819a251-38f7-456e-9f36-ed830f8e1c53" containerName="istio-proxy" containerID="cri-o://376e4b6a2067aac909fa41f067f0fbb3c8acdba8e5a7f217a46da0ba2cd69c9a" gracePeriod=30 Jul 22 03:46:56.111201 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:56.111125 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:56.347165 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:56.347132 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:57.201280 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:57.201246 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:46:57.352530 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:46:57.352502 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:47:01.073352 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.073314 2556 patch_prober.go:28] interesting pod/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused Jul 22 03:47:01.073827 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.073388 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" podUID="f819a251-38f7-456e-9f36-ed830f8e1c53" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 03:47:01.333230 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.333171 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:47:01.437658 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.437630 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f819a251-38f7-456e-9f36-ed830f8e1c53-istio-envoy\") pod \"f819a251-38f7-456e-9f36-ed830f8e1c53\" (UID: \"f819a251-38f7-456e-9f36-ed830f8e1c53\") " Jul 22 03:47:01.437658 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.437664 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f819a251-38f7-456e-9f36-ed830f8e1c53-workload-socket\") pod \"f819a251-38f7-456e-9f36-ed830f8e1c53\" (UID: \"f819a251-38f7-456e-9f36-ed830f8e1c53\") " Jul 22 03:47:01.437882 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.437694 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f819a251-38f7-456e-9f36-ed830f8e1c53-istio-podinfo\") pod \"f819a251-38f7-456e-9f36-ed830f8e1c53\" (UID: \"f819a251-38f7-456e-9f36-ed830f8e1c53\") " Jul 22 03:47:01.437882 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.437787 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f819a251-38f7-456e-9f36-ed830f8e1c53-istio-token\") pod \"f819a251-38f7-456e-9f36-ed830f8e1c53\" (UID: \"f819a251-38f7-456e-9f36-ed830f8e1c53\") " Jul 22 03:47:01.437882 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.437828 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f819a251-38f7-456e-9f36-ed830f8e1c53-workload-certs\") pod \"f819a251-38f7-456e-9f36-ed830f8e1c53\" (UID: \"f819a251-38f7-456e-9f36-ed830f8e1c53\") " Jul 22 03:47:01.438087 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.437892 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f819a251-38f7-456e-9f36-ed830f8e1c53-istiod-ca-cert\") pod \"f819a251-38f7-456e-9f36-ed830f8e1c53\" (UID: \"f819a251-38f7-456e-9f36-ed830f8e1c53\") " Jul 22 03:47:01.438087 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.437919 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f819a251-38f7-456e-9f36-ed830f8e1c53-istio-data\") pod \"f819a251-38f7-456e-9f36-ed830f8e1c53\" (UID: \"f819a251-38f7-456e-9f36-ed830f8e1c53\") " Jul 22 03:47:01.438087 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.437942 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f819a251-38f7-456e-9f36-ed830f8e1c53-credential-socket\") pod \"f819a251-38f7-456e-9f36-ed830f8e1c53\" (UID: \"f819a251-38f7-456e-9f36-ed830f8e1c53\") " Jul 22 03:47:01.438087 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.437970 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f819a251-38f7-456e-9f36-ed830f8e1c53-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f819a251-38f7-456e-9f36-ed830f8e1c53" (UID: "f819a251-38f7-456e-9f36-ed830f8e1c53"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:47:01.438087 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.438008 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8bw7p\" (UniqueName: \"kubernetes.io/projected/f819a251-38f7-456e-9f36-ed830f8e1c53-kube-api-access-8bw7p\") pod \"f819a251-38f7-456e-9f36-ed830f8e1c53\" (UID: \"f819a251-38f7-456e-9f36-ed830f8e1c53\") " Jul 22 03:47:01.438594 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.438567 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f819a251-38f7-456e-9f36-ed830f8e1c53-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f819a251-38f7-456e-9f36-ed830f8e1c53" (UID: "f819a251-38f7-456e-9f36-ed830f8e1c53"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:47:01.438981 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.438925 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f819a251-38f7-456e-9f36-ed830f8e1c53-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f819a251-38f7-456e-9f36-ed830f8e1c53" (UID: "f819a251-38f7-456e-9f36-ed830f8e1c53"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:47:01.439127 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.438987 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f819a251-38f7-456e-9f36-ed830f8e1c53-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f819a251-38f7-456e-9f36-ed830f8e1c53" (UID: "f819a251-38f7-456e-9f36-ed830f8e1c53"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:47:01.439730 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.439698 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f819a251-38f7-456e-9f36-ed830f8e1c53-istio-data" (OuterVolumeSpecName: "istio-data") pod "f819a251-38f7-456e-9f36-ed830f8e1c53" (UID: "f819a251-38f7-456e-9f36-ed830f8e1c53"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:47:01.440582 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.439989 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f819a251-38f7-456e-9f36-ed830f8e1c53-workload-certs\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:47:01.440582 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.440091 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f819a251-38f7-456e-9f36-ed830f8e1c53-istiod-ca-cert\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:47:01.440582 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.440110 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f819a251-38f7-456e-9f36-ed830f8e1c53-istio-data\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:47:01.440582 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.440133 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f819a251-38f7-456e-9f36-ed830f8e1c53-credential-socket\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:47:01.440582 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.440173 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f819a251-38f7-456e-9f36-ed830f8e1c53-workload-socket\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:47:01.440879 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.440607 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f819a251-38f7-456e-9f36-ed830f8e1c53-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f819a251-38f7-456e-9f36-ed830f8e1c53" (UID: "f819a251-38f7-456e-9f36-ed830f8e1c53"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:47:01.442996 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.441291 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f819a251-38f7-456e-9f36-ed830f8e1c53-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f819a251-38f7-456e-9f36-ed830f8e1c53" (UID: "f819a251-38f7-456e-9f36-ed830f8e1c53"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 03:47:01.447459 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.447428 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f819a251-38f7-456e-9f36-ed830f8e1c53-kube-api-access-8bw7p" (OuterVolumeSpecName: "kube-api-access-8bw7p") pod "f819a251-38f7-456e-9f36-ed830f8e1c53" (UID: "f819a251-38f7-456e-9f36-ed830f8e1c53"). InnerVolumeSpecName "kube-api-access-8bw7p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:47:01.447774 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.447748 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f819a251-38f7-456e-9f36-ed830f8e1c53-istio-token" (OuterVolumeSpecName: "istio-token") pod "f819a251-38f7-456e-9f36-ed830f8e1c53" (UID: "f819a251-38f7-456e-9f36-ed830f8e1c53"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:47:01.541107 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.541075 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f819a251-38f7-456e-9f36-ed830f8e1c53-istio-token\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:47:01.541107 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.541102 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8bw7p\" (UniqueName: \"kubernetes.io/projected/f819a251-38f7-456e-9f36-ed830f8e1c53-kube-api-access-8bw7p\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:47:01.541107 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.541113 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f819a251-38f7-456e-9f36-ed830f8e1c53-istio-envoy\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:47:01.541306 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.541122 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f819a251-38f7-456e-9f36-ed830f8e1c53-istio-podinfo\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:47:01.604642 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.604564 2556 generic.go:356] "Generic (PLEG): container finished" podID="f819a251-38f7-456e-9f36-ed830f8e1c53" containerID="376e4b6a2067aac909fa41f067f0fbb3c8acdba8e5a7f217a46da0ba2cd69c9a" exitCode=0 Jul 22 03:47:01.604642 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.604625 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" Jul 22 03:47:01.604642 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.604634 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" event={"ID":"f819a251-38f7-456e-9f36-ed830f8e1c53","Type":"ContainerDied","Data":"376e4b6a2067aac909fa41f067f0fbb3c8acdba8e5a7f217a46da0ba2cd69c9a"} Jul 22 03:47:01.604860 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.604700 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x" event={"ID":"f819a251-38f7-456e-9f36-ed830f8e1c53","Type":"ContainerDied","Data":"79478cb55e84a84080a98f82ea6f68d41b550d6cdbd6e30508c3a482f300a95c"} Jul 22 03:47:01.604860 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.604721 2556 scope.go:117] "RemoveContainer" containerID="376e4b6a2067aac909fa41f067f0fbb3c8acdba8e5a7f217a46da0ba2cd69c9a" Jul 22 03:47:01.614751 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.614734 2556 scope.go:117] "RemoveContainer" containerID="61188f0c27d51a97caf5a4a62e1fa725004d003e8f3310d1e4b98efc91656c41" Jul 22 03:47:01.622858 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.622840 2556 scope.go:117] "RemoveContainer" containerID="376e4b6a2067aac909fa41f067f0fbb3c8acdba8e5a7f217a46da0ba2cd69c9a" Jul 22 03:47:01.623161 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:47:01.623140 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"376e4b6a2067aac909fa41f067f0fbb3c8acdba8e5a7f217a46da0ba2cd69c9a\": container with ID starting with 376e4b6a2067aac909fa41f067f0fbb3c8acdba8e5a7f217a46da0ba2cd69c9a not found: ID does not exist" containerID="376e4b6a2067aac909fa41f067f0fbb3c8acdba8e5a7f217a46da0ba2cd69c9a" Jul 22 03:47:01.623258 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.623173 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"376e4b6a2067aac909fa41f067f0fbb3c8acdba8e5a7f217a46da0ba2cd69c9a"} err="failed to get container status \"376e4b6a2067aac909fa41f067f0fbb3c8acdba8e5a7f217a46da0ba2cd69c9a\": rpc error: code = NotFound desc = could not find container \"376e4b6a2067aac909fa41f067f0fbb3c8acdba8e5a7f217a46da0ba2cd69c9a\": container with ID starting with 376e4b6a2067aac909fa41f067f0fbb3c8acdba8e5a7f217a46da0ba2cd69c9a not found: ID does not exist" Jul 22 03:47:01.623258 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.623197 2556 scope.go:117] "RemoveContainer" containerID="61188f0c27d51a97caf5a4a62e1fa725004d003e8f3310d1e4b98efc91656c41" Jul 22 03:47:01.623468 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:47:01.623451 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61188f0c27d51a97caf5a4a62e1fa725004d003e8f3310d1e4b98efc91656c41\": container with ID starting with 61188f0c27d51a97caf5a4a62e1fa725004d003e8f3310d1e4b98efc91656c41 not found: ID does not exist" containerID="61188f0c27d51a97caf5a4a62e1fa725004d003e8f3310d1e4b98efc91656c41" Jul 22 03:47:01.623511 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.623474 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61188f0c27d51a97caf5a4a62e1fa725004d003e8f3310d1e4b98efc91656c41"} err="failed to get container status \"61188f0c27d51a97caf5a4a62e1fa725004d003e8f3310d1e4b98efc91656c41\": rpc error: code = NotFound desc = could not find container \"61188f0c27d51a97caf5a4a62e1fa725004d003e8f3310d1e4b98efc91656c41\": container with ID starting with 61188f0c27d51a97caf5a4a62e1fa725004d003e8f3310d1e4b98efc91656c41 not found: ID does not exist" Jul 22 03:47:01.626639 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.626618 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x"] Jul 22 03:47:01.631358 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:01.631337 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-7a2eef-openshift-default-6f7f7f98cf-bl74x"] Jul 22 03:47:02.024641 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:02.024564 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f819a251-38f7-456e-9f36-ed830f8e1c53" path="/var/lib/kubelet/pods/f819a251-38f7-456e-9f36-ed830f8e1c53/volumes" Jul 22 03:47:12.033424 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:12.033389 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:47:16.540981 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:16.540931 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:47:16.564544 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:16.564517 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:47:16.956384 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:16.956348 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:47:22.833722 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:22.833690 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:47:22.943665 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:22.943634 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:47:23.114125 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:23.099895 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8"] Jul 22 03:47:23.114125 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:23.107843 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" podUID="2e52ba23-75cf-4789-af93-b2b561de956e" containerName="istio-proxy" containerID="cri-o://27112a35d26ed38820962163c803ef88410c13f9a3f9d08d689dc47b18d6940c" gracePeriod=30 Jul 22 03:47:23.122222 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:23.122182 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:47:23.344283 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:23.344251 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:47:23.684801 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:23.684770 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:47:24.012616 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.012536 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4"] Jul 22 03:47:24.013061 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.012925 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f819a251-38f7-456e-9f36-ed830f8e1c53" containerName="istio-proxy" Jul 22 03:47:24.013061 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.012940 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f819a251-38f7-456e-9f36-ed830f8e1c53" containerName="istio-proxy" Jul 22 03:47:24.013061 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.013039 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f819a251-38f7-456e-9f36-ed830f8e1c53" containerName="istio-proxy" Jul 22 03:47:24.016206 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.016183 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:47:24.019768 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.019745 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-c713ac-openshift-default-dockercfg-4gnsh\"" Jul 22 03:47:24.037547 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.037519 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4"] Jul 22 03:47:24.044591 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.044570 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-istio-envoy\") pod \"e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4\" (UID: \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\") " pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:47:24.044709 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.044615 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhzzs\" (UniqueName: \"kubernetes.io/projected/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-kube-api-access-lhzzs\") pod \"e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4\" (UID: \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\") " pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:47:24.044709 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.044683 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-istio-token\") pod \"e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4\" (UID: \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\") " pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:47:24.044829 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.044744 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-workload-socket\") pod \"e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4\" (UID: \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\") " pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:47:24.045004 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.044941 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-istio-data\") pod \"e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4\" (UID: \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\") " pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:47:24.045100 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.045084 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-istiod-ca-cert\") pod \"e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4\" (UID: \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\") " pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:47:24.045182 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.045156 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-credential-socket\") pod \"e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4\" (UID: \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\") " pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:47:24.045330 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.045197 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-workload-certs\") pod \"e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4\" (UID: \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\") " pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:47:24.045330 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.045293 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-istio-podinfo\") pod \"e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4\" (UID: \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\") " pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:47:24.088282 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.088244 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:47:24.146631 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.146599 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-istio-podinfo\") pod \"e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4\" (UID: \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\") " pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:47:24.146821 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.146657 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-istio-envoy\") pod \"e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4\" (UID: \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\") " pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:47:24.146821 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.146681 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhzzs\" (UniqueName: \"kubernetes.io/projected/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-kube-api-access-lhzzs\") pod \"e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4\" (UID: \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\") " pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:47:24.146821 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.146708 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-istio-token\") pod \"e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4\" (UID: \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\") " pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:47:24.146821 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.146733 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-workload-socket\") pod \"e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4\" (UID: \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\") " pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:47:24.147143 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.146892 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-istio-data\") pod \"e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4\" (UID: \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\") " pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:47:24.147143 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.147007 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-istiod-ca-cert\") pod \"e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4\" (UID: \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\") " pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:47:24.147143 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.147043 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-credential-socket\") pod \"e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4\" (UID: \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\") " pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:47:24.147143 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.147073 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-workload-certs\") pod \"e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4\" (UID: \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\") " pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:47:24.147369 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.147312 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-workload-socket\") pod \"e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4\" (UID: \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\") " pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:47:24.147426 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.147366 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-istio-data\") pod \"e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4\" (UID: \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\") " pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:47:24.147426 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.147412 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-workload-certs\") pod \"e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4\" (UID: \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\") " pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:47:24.147513 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.147487 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-credential-socket\") pod \"e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4\" (UID: \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\") " pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:47:24.147697 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.147678 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-istiod-ca-cert\") pod \"e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4\" (UID: \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\") " pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:47:24.149323 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.149303 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-istio-podinfo\") pod \"e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4\" (UID: \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\") " pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:47:24.149423 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.149360 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-istio-envoy\") pod \"e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4\" (UID: \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\") " pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:47:24.155158 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.155133 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-istio-token\") pod \"e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4\" (UID: \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\") " pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:47:24.155285 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.155206 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhzzs\" (UniqueName: \"kubernetes.io/projected/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-kube-api-access-lhzzs\") pod \"e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4\" (UID: \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\") " pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:47:24.327816 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.327788 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:47:24.477009 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.476976 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4"] Jul 22 03:47:24.478444 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:47:24.478415 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0b0a353_ccc8_4f2f_9f11_a0fe24e16341.slice/crio-b0de11f8bed158bf2dbb1014a056b2e4fb967b86bfe664c836f3f4d092eb007d WatchSource:0}: Error finding container b0de11f8bed158bf2dbb1014a056b2e4fb967b86bfe664c836f3f4d092eb007d: Status 404 returned error can't find the container with id b0de11f8bed158bf2dbb1014a056b2e4fb967b86bfe664c836f3f4d092eb007d Jul 22 03:47:24.481465 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.481428 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:47:24.481574 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.481519 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:47:24.481574 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.481566 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:47:24.597737 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.597650 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:47:24.740813 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.740774 2556 patch_prober.go:28] interesting pod/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 03:47:24.741018 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.740849 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" podUID="2e52ba23-75cf-4789-af93-b2b561de956e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 03:47:24.741580 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.741556 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" event={"ID":"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341","Type":"ContainerStarted","Data":"e7bf63f322348fe590f698f35b1d00db28d89bc4b05f97bec9445ac242efbce4"} Jul 22 03:47:24.741684 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.741606 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" event={"ID":"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341","Type":"ContainerStarted","Data":"b0de11f8bed158bf2dbb1014a056b2e4fb967b86bfe664c836f3f4d092eb007d"} Jul 22 03:47:24.763802 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:24.763753 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" podStartSLOduration=1.763737941 podStartE2EDuration="1.763737941s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:47:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:47:24.761050874 +0000 UTC m=+2385.210034137" watchObservedRunningTime="2026-07-22 03:47:24.763737941 +0000 UTC m=+2385.212721189" Jul 22 03:47:25.328184 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:25.328150 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:47:25.332805 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:25.332782 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:47:25.746666 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:25.746590 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:47:25.747489 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:25.747471 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:47:26.652182 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:26.652149 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:47:26.761729 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:26.761697 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:47:26.942013 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:26.941924 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:47:27.042859 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:27.042799 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:47:27.542914 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:27.542869 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:47:27.646263 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:27.646233 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:47:28.366264 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.366241 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:47:28.391217 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.391143 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e52ba23-75cf-4789-af93-b2b561de956e-credential-socket\") pod \"2e52ba23-75cf-4789-af93-b2b561de956e\" (UID: \"2e52ba23-75cf-4789-af93-b2b561de956e\") " Jul 22 03:47:28.391217 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.391190 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e52ba23-75cf-4789-af93-b2b561de956e-istio-envoy\") pod \"2e52ba23-75cf-4789-af93-b2b561de956e\" (UID: \"2e52ba23-75cf-4789-af93-b2b561de956e\") " Jul 22 03:47:28.391419 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.391225 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e52ba23-75cf-4789-af93-b2b561de956e-istiod-ca-cert\") pod \"2e52ba23-75cf-4789-af93-b2b561de956e\" (UID: \"2e52ba23-75cf-4789-af93-b2b561de956e\") " Jul 22 03:47:28.391419 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.391291 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e52ba23-75cf-4789-af93-b2b561de956e-workload-certs\") pod \"2e52ba23-75cf-4789-af93-b2b561de956e\" (UID: \"2e52ba23-75cf-4789-af93-b2b561de956e\") " Jul 22 03:47:28.391419 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.391322 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x5ljh\" (UniqueName: \"kubernetes.io/projected/2e52ba23-75cf-4789-af93-b2b561de956e-kube-api-access-x5ljh\") pod \"2e52ba23-75cf-4789-af93-b2b561de956e\" (UID: \"2e52ba23-75cf-4789-af93-b2b561de956e\") " Jul 22 03:47:28.391419 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.391398 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e52ba23-75cf-4789-af93-b2b561de956e-istio-podinfo\") pod \"2e52ba23-75cf-4789-af93-b2b561de956e\" (UID: \"2e52ba23-75cf-4789-af93-b2b561de956e\") " Jul 22 03:47:28.391419 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.391399 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e52ba23-75cf-4789-af93-b2b561de956e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2e52ba23-75cf-4789-af93-b2b561de956e" (UID: "2e52ba23-75cf-4789-af93-b2b561de956e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:47:28.391678 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.391431 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e52ba23-75cf-4789-af93-b2b561de956e-istio-data\") pod \"2e52ba23-75cf-4789-af93-b2b561de956e\" (UID: \"2e52ba23-75cf-4789-af93-b2b561de956e\") " Jul 22 03:47:28.391678 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.391471 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e52ba23-75cf-4789-af93-b2b561de956e-workload-socket\") pod \"2e52ba23-75cf-4789-af93-b2b561de956e\" (UID: \"2e52ba23-75cf-4789-af93-b2b561de956e\") " Jul 22 03:47:28.391678 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.391499 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e52ba23-75cf-4789-af93-b2b561de956e-istio-token\") pod \"2e52ba23-75cf-4789-af93-b2b561de956e\" (UID: \"2e52ba23-75cf-4789-af93-b2b561de956e\") " Jul 22 03:47:28.391678 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.391650 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e52ba23-75cf-4789-af93-b2b561de956e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2e52ba23-75cf-4789-af93-b2b561de956e" (UID: "2e52ba23-75cf-4789-af93-b2b561de956e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:47:28.391894 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.391852 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e52ba23-75cf-4789-af93-b2b561de956e-istiod-ca-cert\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:47:28.391894 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.391872 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e52ba23-75cf-4789-af93-b2b561de956e-credential-socket\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:47:28.392107 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.392071 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e52ba23-75cf-4789-af93-b2b561de956e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2e52ba23-75cf-4789-af93-b2b561de956e" (UID: "2e52ba23-75cf-4789-af93-b2b561de956e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:47:28.392285 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.392241 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e52ba23-75cf-4789-af93-b2b561de956e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2e52ba23-75cf-4789-af93-b2b561de956e" (UID: "2e52ba23-75cf-4789-af93-b2b561de956e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:47:28.392572 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.392552 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e52ba23-75cf-4789-af93-b2b561de956e-istio-data" (OuterVolumeSpecName: "istio-data") pod "2e52ba23-75cf-4789-af93-b2b561de956e" (UID: "2e52ba23-75cf-4789-af93-b2b561de956e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:47:28.393976 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.393916 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e52ba23-75cf-4789-af93-b2b561de956e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2e52ba23-75cf-4789-af93-b2b561de956e" (UID: "2e52ba23-75cf-4789-af93-b2b561de956e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:47:28.393976 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.393919 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e52ba23-75cf-4789-af93-b2b561de956e-istio-token" (OuterVolumeSpecName: "istio-token") pod "2e52ba23-75cf-4789-af93-b2b561de956e" (UID: "2e52ba23-75cf-4789-af93-b2b561de956e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:47:28.394123 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.394048 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2e52ba23-75cf-4789-af93-b2b561de956e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2e52ba23-75cf-4789-af93-b2b561de956e" (UID: "2e52ba23-75cf-4789-af93-b2b561de956e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 03:47:28.394123 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.394105 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e52ba23-75cf-4789-af93-b2b561de956e-kube-api-access-x5ljh" (OuterVolumeSpecName: "kube-api-access-x5ljh") pod "2e52ba23-75cf-4789-af93-b2b561de956e" (UID: "2e52ba23-75cf-4789-af93-b2b561de956e"). InnerVolumeSpecName "kube-api-access-x5ljh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:47:28.419257 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.419232 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:47:28.492791 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.492760 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x5ljh\" (UniqueName: \"kubernetes.io/projected/2e52ba23-75cf-4789-af93-b2b561de956e-kube-api-access-x5ljh\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:47:28.492791 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.492787 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e52ba23-75cf-4789-af93-b2b561de956e-istio-podinfo\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:47:28.492791 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.492797 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e52ba23-75cf-4789-af93-b2b561de956e-istio-data\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:47:28.493027 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.492804 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e52ba23-75cf-4789-af93-b2b561de956e-workload-socket\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:47:28.493027 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.492813 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e52ba23-75cf-4789-af93-b2b561de956e-istio-token\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:47:28.493027 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.492821 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e52ba23-75cf-4789-af93-b2b561de956e-istio-envoy\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:47:28.493027 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.492829 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e52ba23-75cf-4789-af93-b2b561de956e-workload-certs\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:47:28.520204 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.520176 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:47:28.765748 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.765623 2556 generic.go:356] "Generic (PLEG): container finished" podID="2e52ba23-75cf-4789-af93-b2b561de956e" containerID="27112a35d26ed38820962163c803ef88410c13f9a3f9d08d689dc47b18d6940c" exitCode=0 Jul 22 03:47:28.765748 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.765683 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" Jul 22 03:47:28.765748 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.765704 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" event={"ID":"2e52ba23-75cf-4789-af93-b2b561de956e","Type":"ContainerDied","Data":"27112a35d26ed38820962163c803ef88410c13f9a3f9d08d689dc47b18d6940c"} Jul 22 03:47:28.766066 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.765753 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8" event={"ID":"2e52ba23-75cf-4789-af93-b2b561de956e","Type":"ContainerDied","Data":"c0f478881fc0f3a1a9d6249be85a01ef379125ad1a5f4af3b722525eed66c4ac"} Jul 22 03:47:28.766066 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.765779 2556 scope.go:117] "RemoveContainer" containerID="27112a35d26ed38820962163c803ef88410c13f9a3f9d08d689dc47b18d6940c" Jul 22 03:47:28.775776 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.775760 2556 scope.go:117] "RemoveContainer" containerID="c41de0484b21588bff4c8d9a84f457b7c0d61d0cde3909589532a9e6745a2ffd" Jul 22 03:47:28.784112 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.784092 2556 scope.go:117] "RemoveContainer" containerID="27112a35d26ed38820962163c803ef88410c13f9a3f9d08d689dc47b18d6940c" Jul 22 03:47:28.784406 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:47:28.784385 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27112a35d26ed38820962163c803ef88410c13f9a3f9d08d689dc47b18d6940c\": container with ID starting with 27112a35d26ed38820962163c803ef88410c13f9a3f9d08d689dc47b18d6940c not found: ID does not exist" containerID="27112a35d26ed38820962163c803ef88410c13f9a3f9d08d689dc47b18d6940c" Jul 22 03:47:28.784495 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.784412 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27112a35d26ed38820962163c803ef88410c13f9a3f9d08d689dc47b18d6940c"} err="failed to get container status \"27112a35d26ed38820962163c803ef88410c13f9a3f9d08d689dc47b18d6940c\": rpc error: code = NotFound desc = could not find container \"27112a35d26ed38820962163c803ef88410c13f9a3f9d08d689dc47b18d6940c\": container with ID starting with 27112a35d26ed38820962163c803ef88410c13f9a3f9d08d689dc47b18d6940c not found: ID does not exist" Jul 22 03:47:28.784495 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.784434 2556 scope.go:117] "RemoveContainer" containerID="c41de0484b21588bff4c8d9a84f457b7c0d61d0cde3909589532a9e6745a2ffd" Jul 22 03:47:28.784721 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:47:28.784698 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c41de0484b21588bff4c8d9a84f457b7c0d61d0cde3909589532a9e6745a2ffd\": container with ID starting with c41de0484b21588bff4c8d9a84f457b7c0d61d0cde3909589532a9e6745a2ffd not found: ID does not exist" containerID="c41de0484b21588bff4c8d9a84f457b7c0d61d0cde3909589532a9e6745a2ffd" Jul 22 03:47:28.784791 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.784731 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c41de0484b21588bff4c8d9a84f457b7c0d61d0cde3909589532a9e6745a2ffd"} err="failed to get container status \"c41de0484b21588bff4c8d9a84f457b7c0d61d0cde3909589532a9e6745a2ffd\": rpc error: code = NotFound desc = could not find container \"c41de0484b21588bff4c8d9a84f457b7c0d61d0cde3909589532a9e6745a2ffd\": container with ID starting with c41de0484b21588bff4c8d9a84f457b7c0d61d0cde3909589532a9e6745a2ffd not found: ID does not exist" Jul 22 03:47:28.792556 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.792535 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8"] Jul 22 03:47:28.794779 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:28.794758 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-92a8f6-openshift-default-9b5b68b9-dsxk8"] Jul 22 03:47:29.276905 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:29.276866 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:47:29.374732 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:29.374691 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:47:30.025034 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:30.025004 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e52ba23-75cf-4789-af93-b2b561de956e" path="/var/lib/kubelet/pods/2e52ba23-75cf-4789-af93-b2b561de956e/volumes" Jul 22 03:47:31.153778 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:31.153744 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:47:31.615377 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:31.615341 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:47:31.844855 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:31.844820 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs"] Jul 22 03:47:31.845429 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:31.845414 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f819a251-38f7-456e-9f36-ed830f8e1c53" containerName="istio-proxy" Jul 22 03:47:31.845479 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:31.845442 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f819a251-38f7-456e-9f36-ed830f8e1c53" containerName="istio-proxy" Jul 22 03:47:31.845479 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:31.845471 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e52ba23-75cf-4789-af93-b2b561de956e" containerName="istio-proxy" Jul 22 03:47:31.845548 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:31.845480 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e52ba23-75cf-4789-af93-b2b561de956e" containerName="istio-proxy" Jul 22 03:47:31.845548 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:31.845491 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e52ba23-75cf-4789-af93-b2b561de956e" containerName="istio-proxy" Jul 22 03:47:31.845548 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:31.845498 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e52ba23-75cf-4789-af93-b2b561de956e" containerName="istio-proxy" Jul 22 03:47:31.845639 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:31.845593 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f819a251-38f7-456e-9f36-ed830f8e1c53" containerName="istio-proxy" Jul 22 03:47:31.845639 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:31.845609 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e52ba23-75cf-4789-af93-b2b561de956e" containerName="istio-proxy" Jul 22 03:47:31.845639 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:31.845618 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e52ba23-75cf-4789-af93-b2b561de956e" containerName="istio-proxy" Jul 22 03:47:31.849297 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:31.849277 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:47:31.852165 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:31.852137 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-be39c1-openshift-default-dockercfg-rtpnz\"" Jul 22 03:47:31.866391 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:31.866327 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs"] Jul 22 03:47:31.887783 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:31.887756 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:47:31.923319 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:31.923286 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kg6z\" (UniqueName: \"kubernetes.io/projected/79ca697d-2b42-4e98-985d-6d9797094db3-kube-api-access-6kg6z\") pod \"e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs\" (UID: \"79ca697d-2b42-4e98-985d-6d9797094db3\") " pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:47:31.923486 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:31.923334 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79ca697d-2b42-4e98-985d-6d9797094db3-credential-socket\") pod \"e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs\" (UID: \"79ca697d-2b42-4e98-985d-6d9797094db3\") " pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:47:31.923486 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:31.923371 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79ca697d-2b42-4e98-985d-6d9797094db3-istiod-ca-cert\") pod \"e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs\" (UID: \"79ca697d-2b42-4e98-985d-6d9797094db3\") " pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:47:31.923486 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:31.923405 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79ca697d-2b42-4e98-985d-6d9797094db3-istio-token\") pod \"e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs\" (UID: \"79ca697d-2b42-4e98-985d-6d9797094db3\") " pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:47:31.923486 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:31.923430 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79ca697d-2b42-4e98-985d-6d9797094db3-workload-certs\") pod \"e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs\" (UID: \"79ca697d-2b42-4e98-985d-6d9797094db3\") " pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:47:31.923628 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:31.923531 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79ca697d-2b42-4e98-985d-6d9797094db3-istio-data\") pod \"e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs\" (UID: \"79ca697d-2b42-4e98-985d-6d9797094db3\") " pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:47:31.923670 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:31.923641 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79ca697d-2b42-4e98-985d-6d9797094db3-istio-envoy\") pod \"e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs\" (UID: \"79ca697d-2b42-4e98-985d-6d9797094db3\") " pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:47:31.923725 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:31.923675 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79ca697d-2b42-4e98-985d-6d9797094db3-workload-socket\") pod \"e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs\" (UID: \"79ca697d-2b42-4e98-985d-6d9797094db3\") " pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:47:31.923725 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:31.923719 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79ca697d-2b42-4e98-985d-6d9797094db3-istio-podinfo\") pod \"e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs\" (UID: \"79ca697d-2b42-4e98-985d-6d9797094db3\") " pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:47:32.025167 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:32.025132 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79ca697d-2b42-4e98-985d-6d9797094db3-istio-envoy\") pod \"e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs\" (UID: \"79ca697d-2b42-4e98-985d-6d9797094db3\") " pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:47:32.025167 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:32.025170 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79ca697d-2b42-4e98-985d-6d9797094db3-workload-socket\") pod \"e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs\" (UID: \"79ca697d-2b42-4e98-985d-6d9797094db3\") " pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:47:32.025505 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:32.025199 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79ca697d-2b42-4e98-985d-6d9797094db3-istio-podinfo\") pod \"e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs\" (UID: \"79ca697d-2b42-4e98-985d-6d9797094db3\") " pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:47:32.025505 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:32.025231 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kg6z\" (UniqueName: \"kubernetes.io/projected/79ca697d-2b42-4e98-985d-6d9797094db3-kube-api-access-6kg6z\") pod \"e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs\" (UID: \"79ca697d-2b42-4e98-985d-6d9797094db3\") " pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:47:32.025505 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:32.025255 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79ca697d-2b42-4e98-985d-6d9797094db3-credential-socket\") pod \"e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs\" (UID: \"79ca697d-2b42-4e98-985d-6d9797094db3\") " pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:47:32.025505 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:32.025270 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79ca697d-2b42-4e98-985d-6d9797094db3-istiod-ca-cert\") pod \"e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs\" (UID: \"79ca697d-2b42-4e98-985d-6d9797094db3\") " pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:47:32.025505 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:32.025307 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79ca697d-2b42-4e98-985d-6d9797094db3-istio-token\") pod \"e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs\" (UID: \"79ca697d-2b42-4e98-985d-6d9797094db3\") " pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:47:32.025505 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:32.025331 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79ca697d-2b42-4e98-985d-6d9797094db3-workload-certs\") pod \"e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs\" (UID: \"79ca697d-2b42-4e98-985d-6d9797094db3\") " pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:47:32.025505 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:32.025350 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79ca697d-2b42-4e98-985d-6d9797094db3-istio-data\") pod \"e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs\" (UID: \"79ca697d-2b42-4e98-985d-6d9797094db3\") " pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:47:32.025873 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:32.025610 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79ca697d-2b42-4e98-985d-6d9797094db3-credential-socket\") pod \"e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs\" (UID: \"79ca697d-2b42-4e98-985d-6d9797094db3\") " pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:47:32.025873 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:32.025659 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79ca697d-2b42-4e98-985d-6d9797094db3-istio-data\") pod \"e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs\" (UID: \"79ca697d-2b42-4e98-985d-6d9797094db3\") " pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:47:32.026011 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:32.025880 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79ca697d-2b42-4e98-985d-6d9797094db3-workload-socket\") pod \"e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs\" (UID: \"79ca697d-2b42-4e98-985d-6d9797094db3\") " pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:47:32.026011 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:32.025900 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79ca697d-2b42-4e98-985d-6d9797094db3-workload-certs\") pod \"e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs\" (UID: \"79ca697d-2b42-4e98-985d-6d9797094db3\") " pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:47:32.026288 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:32.026263 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79ca697d-2b42-4e98-985d-6d9797094db3-istiod-ca-cert\") pod \"e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs\" (UID: \"79ca697d-2b42-4e98-985d-6d9797094db3\") " pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:47:32.027616 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:32.027586 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79ca697d-2b42-4e98-985d-6d9797094db3-istio-envoy\") pod \"e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs\" (UID: \"79ca697d-2b42-4e98-985d-6d9797094db3\") " pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:47:32.027887 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:32.027868 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79ca697d-2b42-4e98-985d-6d9797094db3-istio-podinfo\") pod \"e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs\" (UID: \"79ca697d-2b42-4e98-985d-6d9797094db3\") " pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:47:32.033779 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:32.033760 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kg6z\" (UniqueName: \"kubernetes.io/projected/79ca697d-2b42-4e98-985d-6d9797094db3-kube-api-access-6kg6z\") pod \"e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs\" (UID: \"79ca697d-2b42-4e98-985d-6d9797094db3\") " pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:47:32.034018 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:32.033972 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79ca697d-2b42-4e98-985d-6d9797094db3-istio-token\") pod \"e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs\" (UID: \"79ca697d-2b42-4e98-985d-6d9797094db3\") " pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:47:32.160448 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:32.160359 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:47:32.193422 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:32.193391 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:47:32.313416 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:32.313386 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs"] Jul 22 03:47:32.315848 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:47:32.315822 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79ca697d_2b42_4e98_985d_6d9797094db3.slice/crio-64de8a94b37fa2e03fa8a95bcc206e291b13a4e99f878c437f87062e81207ec1 WatchSource:0}: Error finding container 64de8a94b37fa2e03fa8a95bcc206e291b13a4e99f878c437f87062e81207ec1: Status 404 returned error can't find the container with id 64de8a94b37fa2e03fa8a95bcc206e291b13a4e99f878c437f87062e81207ec1 Jul 22 03:47:32.318066 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:32.318032 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:47:32.318177 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:32.318113 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:47:32.318177 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:32.318160 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:47:32.346420 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:32.346393 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:47:32.793472 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:32.793430 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" event={"ID":"79ca697d-2b42-4e98-985d-6d9797094db3","Type":"ContainerStarted","Data":"c55f216e8e65f553a06c47d67ec4cf4484489aa07e477a011cb072489d269dea"} Jul 22 03:47:32.793655 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:32.793495 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" event={"ID":"79ca697d-2b42-4e98-985d-6d9797094db3","Type":"ContainerStarted","Data":"64de8a94b37fa2e03fa8a95bcc206e291b13a4e99f878c437f87062e81207ec1"} Jul 22 03:47:32.832238 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:32.832184 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" podStartSLOduration=1.8321664690000001 podStartE2EDuration="1.832166469s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:47:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:47:32.829185303 +0000 UTC m=+2393.278168554" watchObservedRunningTime="2026-07-22 03:47:32.832166469 +0000 UTC m=+2393.281149718" Jul 22 03:47:33.161256 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:33.161223 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:47:33.166363 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:33.166336 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:47:33.802401 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:33.802359 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:47:33.803353 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:33.803331 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:47:34.861450 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:34.861420 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:47:34.952391 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:34.952355 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:47:35.054460 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:35.054414 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:47:35.622202 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:35.622166 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:47:35.724435 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:35.724405 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:47:35.802184 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:35.802120 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:47:39.257633 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:39.257597 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:47:40.323131 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:40.323030 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscf8_a01dfba4-f590-412a-8438-1646c51bb1a1/ovn-acl-logging/0.log" Jul 22 03:47:40.329190 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:40.329172 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscf8_a01dfba4-f590-412a-8438-1646c51bb1a1/ovn-acl-logging/0.log" Jul 22 03:47:40.748180 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:40.748100 2556 patch_prober.go:28] interesting pod/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 22 03:47:40.748180 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:40.748165 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" podUID="b0b0a353-ccc8-4f2f-9f11-a0fe24e16341" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 22 03:47:40.845538 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:40.845504 2556 generic.go:356] "Generic (PLEG): container finished" podID="b0b0a353-ccc8-4f2f-9f11-a0fe24e16341" containerID="e7bf63f322348fe590f698f35b1d00db28d89bc4b05f97bec9445ac242efbce4" exitCode=0 Jul 22 03:47:40.845733 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:40.845582 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" event={"ID":"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341","Type":"ContainerDied","Data":"e7bf63f322348fe590f698f35b1d00db28d89bc4b05f97bec9445ac242efbce4"} Jul 22 03:47:40.846106 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:40.846089 2556 scope.go:117] "RemoveContainer" containerID="e7bf63f322348fe590f698f35b1d00db28d89bc4b05f97bec9445ac242efbce4" Jul 22 03:47:40.847150 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:40.847121 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:47:40.847234 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:40.847188 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:47:40.847234 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:40.847221 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:47:41.853326 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:41.853294 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" event={"ID":"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341","Type":"ContainerStarted","Data":"1b9c4cb43414c4ce5d7a1cb7e3cb0fb46ac07e3af940f906e284312f484d0df9"} Jul 22 03:47:42.328312 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:42.328273 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:47:42.328545 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:42.328518 2556 patch_prober.go:28] interesting pod/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 22 03:47:42.328745 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:42.328718 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" podUID="b0b0a353-ccc8-4f2f-9f11-a0fe24e16341" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 22 03:47:43.328904 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:43.328864 2556 patch_prober.go:28] interesting pod/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 22 03:47:43.329321 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:43.328917 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" podUID="b0b0a353-ccc8-4f2f-9f11-a0fe24e16341" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 22 03:47:44.328841 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:44.328804 2556 patch_prober.go:28] interesting pod/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 22 03:47:44.329032 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:44.328865 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" podUID="b0b0a353-ccc8-4f2f-9f11-a0fe24e16341" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 22 03:47:45.328797 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:45.328754 2556 patch_prober.go:28] interesting pod/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 22 03:47:45.329000 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:45.328815 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" podUID="b0b0a353-ccc8-4f2f-9f11-a0fe24e16341" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 22 03:47:46.333049 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:46.333013 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:47:46.333523 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:46.333394 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:47:46.334028 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:46.334008 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:47:47.916655 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:47.916620 2556 generic.go:356] "Generic (PLEG): container finished" podID="79ca697d-2b42-4e98-985d-6d9797094db3" containerID="c55f216e8e65f553a06c47d67ec4cf4484489aa07e477a011cb072489d269dea" exitCode=0 Jul 22 03:47:47.917164 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:47.916696 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" event={"ID":"79ca697d-2b42-4e98-985d-6d9797094db3","Type":"ContainerDied","Data":"c55f216e8e65f553a06c47d67ec4cf4484489aa07e477a011cb072489d269dea"} Jul 22 03:47:47.917272 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:47.917255 2556 scope.go:117] "RemoveContainer" containerID="c55f216e8e65f553a06c47d67ec4cf4484489aa07e477a011cb072489d269dea" Jul 22 03:47:47.918211 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:47.918184 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:47:47.918270 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:47.918251 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:47:47.918311 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:47.918283 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 03:47:48.237918 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:48.237840 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:47:48.309002 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:48.308943 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:47:48.416071 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:48.416038 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:47:48.802533 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:48.802489 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:47:48.924347 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:48.924314 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" event={"ID":"79ca697d-2b42-4e98-985d-6d9797094db3","Type":"ContainerStarted","Data":"1fe8d477b39018290ecd420544cff5777e2e6dcc38d12f5d141d1c56a81c4c7e"} Jul 22 03:47:49.161529 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:49.161497 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:47:49.162596 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:49.162574 2556 patch_prober.go:28] interesting pod/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Jul 22 03:47:49.162704 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:49.162632 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" podUID="79ca697d-2b42-4e98-985d-6d9797094db3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Jul 22 03:47:50.160993 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:50.160937 2556 patch_prober.go:28] interesting pod/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Jul 22 03:47:50.161402 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:50.161023 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" podUID="79ca697d-2b42-4e98-985d-6d9797094db3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Jul 22 03:47:51.161167 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:51.161131 2556 patch_prober.go:28] interesting pod/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Jul 22 03:47:51.161559 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:51.161200 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" podUID="79ca697d-2b42-4e98-985d-6d9797094db3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Jul 22 03:47:52.161216 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:52.161171 2556 patch_prober.go:28] interesting pod/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Jul 22 03:47:52.161611 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:52.161252 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" podUID="79ca697d-2b42-4e98-985d-6d9797094db3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Jul 22 03:47:53.164917 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:53.164885 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:47:53.165301 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:53.165176 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:47:53.165892 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:53.165873 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:47:56.075923 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:56.075874 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:47:56.406137 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:56.406048 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:47:56.489106 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:47:56.489073 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:48:01.230596 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:48:01.230520 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:48:18.091664 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:48:18.091619 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:48:21.938888 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:48:21.938857 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:48:51.035717 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:48:51.035684 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:48:54.699507 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:48:54.699468 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:48:54.776145 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:48:54.776110 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:49:10.234346 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:49:10.234311 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:49:24.733961 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:49:24.733903 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:49:43.137231 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:49:43.137153 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:50:01.134999 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:50:01.134961 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:50:08.272322 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:50:08.272286 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:50:20.626202 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:50:20.626167 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:50:35.205495 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:50:35.205451 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:50:36.436312 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:50:36.436277 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:50:39.735973 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:50:39.735917 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:51:10.489278 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:51:10.489192 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:51:13.937884 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:51:13.937849 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:51:30.501423 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:51:30.501373 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:51:34.347316 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:51:34.347282 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:51:48.341149 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:51:48.341109 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:51:55.801032 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:51:55.800988 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:52:07.322410 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:07.322372 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:52:08.242703 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:08.242661 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:52:11.553271 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:11.553234 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:52:12.463554 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:12.463523 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-c713ac/rate-test-model-c713ac-kserve-55cb595c9f-4qmnn"] Jul 22 03:52:12.467246 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:12.467228 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c713ac/rate-test-model-c713ac-kserve-55cb595c9f-4qmnn" Jul 22 03:52:12.473087 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:12.472750 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c713ac\"/\"kube-root-ca.crt\"" Jul 22 03:52:12.473087 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:12.472812 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c713ac\"/\"rate-test-model-c713ac-kserve-self-signed-certs\"" Jul 22 03:52:12.473087 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:12.472748 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c713ac\"/\"openshift-service-ca.crt\"" Jul 22 03:52:12.473087 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:12.472758 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c713ac\"/\"default-dockercfg-7v8hk\"" Jul 22 03:52:12.496298 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:12.496270 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c713ac/rate-test-model-c713ac-kserve-55cb595c9f-4qmnn"] Jul 22 03:52:12.546592 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:12.546564 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:52:12.555851 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:12.555829 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4adaf440-a408-42b2-8cc8-d5ed76e8184a-tls-certs\") pod \"rate-test-model-c713ac-kserve-55cb595c9f-4qmnn\" (UID: \"4adaf440-a408-42b2-8cc8-d5ed76e8184a\") " pod="ai-tenant-e2e-shared-a-c713ac/rate-test-model-c713ac-kserve-55cb595c9f-4qmnn" Jul 22 03:52:12.556197 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:12.555891 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wx6tc\" (UniqueName: \"kubernetes.io/projected/4adaf440-a408-42b2-8cc8-d5ed76e8184a-kube-api-access-wx6tc\") pod \"rate-test-model-c713ac-kserve-55cb595c9f-4qmnn\" (UID: \"4adaf440-a408-42b2-8cc8-d5ed76e8184a\") " pod="ai-tenant-e2e-shared-a-c713ac/rate-test-model-c713ac-kserve-55cb595c9f-4qmnn" Jul 22 03:52:12.556197 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:12.555937 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4adaf440-a408-42b2-8cc8-d5ed76e8184a-dshm\") pod \"rate-test-model-c713ac-kserve-55cb595c9f-4qmnn\" (UID: \"4adaf440-a408-42b2-8cc8-d5ed76e8184a\") " pod="ai-tenant-e2e-shared-a-c713ac/rate-test-model-c713ac-kserve-55cb595c9f-4qmnn" Jul 22 03:52:12.556197 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:12.556004 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4adaf440-a408-42b2-8cc8-d5ed76e8184a-home\") pod \"rate-test-model-c713ac-kserve-55cb595c9f-4qmnn\" (UID: \"4adaf440-a408-42b2-8cc8-d5ed76e8184a\") " pod="ai-tenant-e2e-shared-a-c713ac/rate-test-model-c713ac-kserve-55cb595c9f-4qmnn" Jul 22 03:52:12.556197 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:12.556029 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4adaf440-a408-42b2-8cc8-d5ed76e8184a-model-cache\") pod \"rate-test-model-c713ac-kserve-55cb595c9f-4qmnn\" (UID: \"4adaf440-a408-42b2-8cc8-d5ed76e8184a\") " pod="ai-tenant-e2e-shared-a-c713ac/rate-test-model-c713ac-kserve-55cb595c9f-4qmnn" Jul 22 03:52:12.556197 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:12.556045 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4adaf440-a408-42b2-8cc8-d5ed76e8184a-tmp-dir\") pod \"rate-test-model-c713ac-kserve-55cb595c9f-4qmnn\" (UID: \"4adaf440-a408-42b2-8cc8-d5ed76e8184a\") " pod="ai-tenant-e2e-shared-a-c713ac/rate-test-model-c713ac-kserve-55cb595c9f-4qmnn" Jul 22 03:52:12.659557 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:12.657532 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4adaf440-a408-42b2-8cc8-d5ed76e8184a-tls-certs\") pod \"rate-test-model-c713ac-kserve-55cb595c9f-4qmnn\" (UID: \"4adaf440-a408-42b2-8cc8-d5ed76e8184a\") " pod="ai-tenant-e2e-shared-a-c713ac/rate-test-model-c713ac-kserve-55cb595c9f-4qmnn" Jul 22 03:52:12.659557 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:12.657780 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wx6tc\" (UniqueName: \"kubernetes.io/projected/4adaf440-a408-42b2-8cc8-d5ed76e8184a-kube-api-access-wx6tc\") pod \"rate-test-model-c713ac-kserve-55cb595c9f-4qmnn\" (UID: \"4adaf440-a408-42b2-8cc8-d5ed76e8184a\") " pod="ai-tenant-e2e-shared-a-c713ac/rate-test-model-c713ac-kserve-55cb595c9f-4qmnn" Jul 22 03:52:12.659557 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:12.657816 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4adaf440-a408-42b2-8cc8-d5ed76e8184a-dshm\") pod \"rate-test-model-c713ac-kserve-55cb595c9f-4qmnn\" (UID: \"4adaf440-a408-42b2-8cc8-d5ed76e8184a\") " pod="ai-tenant-e2e-shared-a-c713ac/rate-test-model-c713ac-kserve-55cb595c9f-4qmnn" Jul 22 03:52:12.659557 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:12.657848 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4adaf440-a408-42b2-8cc8-d5ed76e8184a-home\") pod \"rate-test-model-c713ac-kserve-55cb595c9f-4qmnn\" (UID: \"4adaf440-a408-42b2-8cc8-d5ed76e8184a\") " pod="ai-tenant-e2e-shared-a-c713ac/rate-test-model-c713ac-kserve-55cb595c9f-4qmnn" Jul 22 03:52:12.659557 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:12.657881 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4adaf440-a408-42b2-8cc8-d5ed76e8184a-model-cache\") pod \"rate-test-model-c713ac-kserve-55cb595c9f-4qmnn\" (UID: \"4adaf440-a408-42b2-8cc8-d5ed76e8184a\") " pod="ai-tenant-e2e-shared-a-c713ac/rate-test-model-c713ac-kserve-55cb595c9f-4qmnn" Jul 22 03:52:12.659557 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:12.658660 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4adaf440-a408-42b2-8cc8-d5ed76e8184a-model-cache\") pod \"rate-test-model-c713ac-kserve-55cb595c9f-4qmnn\" (UID: \"4adaf440-a408-42b2-8cc8-d5ed76e8184a\") " pod="ai-tenant-e2e-shared-a-c713ac/rate-test-model-c713ac-kserve-55cb595c9f-4qmnn" Jul 22 03:52:12.659557 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:12.658791 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4adaf440-a408-42b2-8cc8-d5ed76e8184a-tmp-dir\") pod \"rate-test-model-c713ac-kserve-55cb595c9f-4qmnn\" (UID: \"4adaf440-a408-42b2-8cc8-d5ed76e8184a\") " pod="ai-tenant-e2e-shared-a-c713ac/rate-test-model-c713ac-kserve-55cb595c9f-4qmnn" Jul 22 03:52:12.659557 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:12.659167 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4adaf440-a408-42b2-8cc8-d5ed76e8184a-tmp-dir\") pod \"rate-test-model-c713ac-kserve-55cb595c9f-4qmnn\" (UID: \"4adaf440-a408-42b2-8cc8-d5ed76e8184a\") " pod="ai-tenant-e2e-shared-a-c713ac/rate-test-model-c713ac-kserve-55cb595c9f-4qmnn" Jul 22 03:52:12.659557 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:12.659422 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4adaf440-a408-42b2-8cc8-d5ed76e8184a-home\") pod \"rate-test-model-c713ac-kserve-55cb595c9f-4qmnn\" (UID: \"4adaf440-a408-42b2-8cc8-d5ed76e8184a\") " pod="ai-tenant-e2e-shared-a-c713ac/rate-test-model-c713ac-kserve-55cb595c9f-4qmnn" Jul 22 03:52:12.660765 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:12.660732 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4adaf440-a408-42b2-8cc8-d5ed76e8184a-dshm\") pod \"rate-test-model-c713ac-kserve-55cb595c9f-4qmnn\" (UID: \"4adaf440-a408-42b2-8cc8-d5ed76e8184a\") " pod="ai-tenant-e2e-shared-a-c713ac/rate-test-model-c713ac-kserve-55cb595c9f-4qmnn" Jul 22 03:52:12.661356 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:12.661296 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4adaf440-a408-42b2-8cc8-d5ed76e8184a-tls-certs\") pod \"rate-test-model-c713ac-kserve-55cb595c9f-4qmnn\" (UID: \"4adaf440-a408-42b2-8cc8-d5ed76e8184a\") " pod="ai-tenant-e2e-shared-a-c713ac/rate-test-model-c713ac-kserve-55cb595c9f-4qmnn" Jul 22 03:52:12.667787 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:12.667767 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wx6tc\" (UniqueName: \"kubernetes.io/projected/4adaf440-a408-42b2-8cc8-d5ed76e8184a-kube-api-access-wx6tc\") pod \"rate-test-model-c713ac-kserve-55cb595c9f-4qmnn\" (UID: \"4adaf440-a408-42b2-8cc8-d5ed76e8184a\") " pod="ai-tenant-e2e-shared-a-c713ac/rate-test-model-c713ac-kserve-55cb595c9f-4qmnn" Jul 22 03:52:12.777364 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:12.777285 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c713ac/rate-test-model-c713ac-kserve-55cb595c9f-4qmnn" Jul 22 03:52:12.924190 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:12.924168 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c713ac/rate-test-model-c713ac-kserve-55cb595c9f-4qmnn"] Jul 22 03:52:12.926532 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:52:12.926505 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4adaf440_a408_42b2_8cc8_d5ed76e8184a.slice/crio-a98510b39185135fa8db713f87aa3c39c5d01f08a308335491bf00166a681ac8 WatchSource:0}: Error finding container a98510b39185135fa8db713f87aa3c39c5d01f08a308335491bf00166a681ac8: Status 404 returned error can't find the container with id a98510b39185135fa8db713f87aa3c39c5d01f08a308335491bf00166a681ac8 Jul 22 03:52:13.425053 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:13.425007 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c713ac/rate-test-model-c713ac-kserve-55cb595c9f-4qmnn" event={"ID":"4adaf440-a408-42b2-8cc8-d5ed76e8184a","Type":"ContainerStarted","Data":"f0336567ba3084284a1acaa01860eda6007c6c02eb71d9afcce8b1c7180c726b"} Jul 22 03:52:13.425276 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:13.425062 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c713ac/rate-test-model-c713ac-kserve-55cb595c9f-4qmnn" event={"ID":"4adaf440-a408-42b2-8cc8-d5ed76e8184a","Type":"ContainerStarted","Data":"a98510b39185135fa8db713f87aa3c39c5d01f08a308335491bf00166a681ac8"} Jul 22 03:52:13.445365 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:13.445311 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-c713ac/rate-test-model-c713ac-kserve-55cb595c9f-4qmnn" podStartSLOduration=1.4452952350000001 podStartE2EDuration="1.445295235s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:52:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:52:13.443596973 +0000 UTC m=+2673.892580221" watchObservedRunningTime="2026-07-22 03:52:13.445295235 +0000 UTC m=+2673.894278506" Jul 22 03:52:22.777705 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:22.777671 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-c713ac/rate-test-model-c713ac-kserve-55cb595c9f-4qmnn" Jul 22 03:52:22.777705 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:22.777721 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-c713ac/rate-test-model-c713ac-kserve-55cb595c9f-4qmnn" Jul 22 03:52:22.790514 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:22.790485 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-c713ac/rate-test-model-c713ac-kserve-55cb595c9f-4qmnn" Jul 22 03:52:23.501034 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:23.501008 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-c713ac/rate-test-model-c713ac-kserve-55cb595c9f-4qmnn" Jul 22 03:52:24.432262 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:24.432220 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:52:29.287274 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:29.287244 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:52:29.387730 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:29.387694 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:52:29.558227 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:29.558133 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:52:32.034979 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:32.034893 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:52:40.359558 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:40.359531 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscf8_a01dfba4-f590-412a-8438-1646c51bb1a1/ovn-acl-logging/0.log" Jul 22 03:52:40.364775 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:40.364752 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscf8_a01dfba4-f590-412a-8438-1646c51bb1a1/ovn-acl-logging/0.log" Jul 22 03:52:44.704423 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:44.704383 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:52:44.768773 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:44.768737 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:52:49.063454 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:49.063422 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:52:49.163354 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:49.163318 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:52:57.283578 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:57.283535 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:52:57.400396 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:52:57.400365 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:53:03.552978 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:03.552934 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:53:18.433198 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:18.433166 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:53:24.238219 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:24.238187 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c713ac/rate-test-model-c713ac-kserve-55cb595c9f-4qmnn"] Jul 22 03:53:24.265569 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:24.265539 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:53:25.225985 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:25.225936 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:53:31.104561 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:31.104528 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:53:36.235832 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:36.235792 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:53:39.258221 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:39.258157 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-c713ac/rate-test-model-c713ac-kserve-55cb595c9f-4qmnn" podUID="4adaf440-a408-42b2-8cc8-d5ed76e8184a" containerName="main" containerID="cri-o://f0336567ba3084284a1acaa01860eda6007c6c02eb71d9afcce8b1c7180c726b" gracePeriod=45 Jul 22 03:53:39.506691 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:39.506661 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c713ac/rate-test-model-c713ac-kserve-55cb595c9f-4qmnn" Jul 22 03:53:39.604312 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:39.604285 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4adaf440-a408-42b2-8cc8-d5ed76e8184a-tmp-dir\") pod \"4adaf440-a408-42b2-8cc8-d5ed76e8184a\" (UID: \"4adaf440-a408-42b2-8cc8-d5ed76e8184a\") " Jul 22 03:53:39.604462 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:39.604324 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wx6tc\" (UniqueName: \"kubernetes.io/projected/4adaf440-a408-42b2-8cc8-d5ed76e8184a-kube-api-access-wx6tc\") pod \"4adaf440-a408-42b2-8cc8-d5ed76e8184a\" (UID: \"4adaf440-a408-42b2-8cc8-d5ed76e8184a\") " Jul 22 03:53:39.604462 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:39.604345 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4adaf440-a408-42b2-8cc8-d5ed76e8184a-model-cache\") pod \"4adaf440-a408-42b2-8cc8-d5ed76e8184a\" (UID: \"4adaf440-a408-42b2-8cc8-d5ed76e8184a\") " Jul 22 03:53:39.604462 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:39.604394 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4adaf440-a408-42b2-8cc8-d5ed76e8184a-dshm\") pod \"4adaf440-a408-42b2-8cc8-d5ed76e8184a\" (UID: \"4adaf440-a408-42b2-8cc8-d5ed76e8184a\") " Jul 22 03:53:39.604462 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:39.604454 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4adaf440-a408-42b2-8cc8-d5ed76e8184a-home\") pod \"4adaf440-a408-42b2-8cc8-d5ed76e8184a\" (UID: \"4adaf440-a408-42b2-8cc8-d5ed76e8184a\") " Jul 22 03:53:39.604641 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:39.604538 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4adaf440-a408-42b2-8cc8-d5ed76e8184a-tls-certs\") pod \"4adaf440-a408-42b2-8cc8-d5ed76e8184a\" (UID: \"4adaf440-a408-42b2-8cc8-d5ed76e8184a\") " Jul 22 03:53:39.604641 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:39.604559 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4adaf440-a408-42b2-8cc8-d5ed76e8184a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4adaf440-a408-42b2-8cc8-d5ed76e8184a" (UID: "4adaf440-a408-42b2-8cc8-d5ed76e8184a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:53:39.604750 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:39.604639 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4adaf440-a408-42b2-8cc8-d5ed76e8184a-model-cache" (OuterVolumeSpecName: "model-cache") pod "4adaf440-a408-42b2-8cc8-d5ed76e8184a" (UID: "4adaf440-a408-42b2-8cc8-d5ed76e8184a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:53:39.604750 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:39.604692 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4adaf440-a408-42b2-8cc8-d5ed76e8184a-home" (OuterVolumeSpecName: "home") pod "4adaf440-a408-42b2-8cc8-d5ed76e8184a" (UID: "4adaf440-a408-42b2-8cc8-d5ed76e8184a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:53:39.604868 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:39.604841 2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4adaf440-a408-42b2-8cc8-d5ed76e8184a-home\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:53:39.604868 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:39.604857 2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4adaf440-a408-42b2-8cc8-d5ed76e8184a-tmp-dir\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:53:39.604868 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:39.604866 2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4adaf440-a408-42b2-8cc8-d5ed76e8184a-model-cache\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:53:39.606584 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:39.606558 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4adaf440-a408-42b2-8cc8-d5ed76e8184a-dshm" (OuterVolumeSpecName: "dshm") pod "4adaf440-a408-42b2-8cc8-d5ed76e8184a" (UID: "4adaf440-a408-42b2-8cc8-d5ed76e8184a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:53:39.606695 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:39.606636 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4adaf440-a408-42b2-8cc8-d5ed76e8184a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4adaf440-a408-42b2-8cc8-d5ed76e8184a" (UID: "4adaf440-a408-42b2-8cc8-d5ed76e8184a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:53:39.606695 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:39.606655 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4adaf440-a408-42b2-8cc8-d5ed76e8184a-kube-api-access-wx6tc" (OuterVolumeSpecName: "kube-api-access-wx6tc") pod "4adaf440-a408-42b2-8cc8-d5ed76e8184a" (UID: "4adaf440-a408-42b2-8cc8-d5ed76e8184a"). InnerVolumeSpecName "kube-api-access-wx6tc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:53:39.706333 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:39.706301 2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4adaf440-a408-42b2-8cc8-d5ed76e8184a-tls-certs\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:53:39.706333 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:39.706329 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wx6tc\" (UniqueName: \"kubernetes.io/projected/4adaf440-a408-42b2-8cc8-d5ed76e8184a-kube-api-access-wx6tc\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:53:39.706333 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:39.706338 2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4adaf440-a408-42b2-8cc8-d5ed76e8184a-dshm\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:53:39.935642 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:39.935554 2556 generic.go:356] "Generic (PLEG): container finished" podID="4adaf440-a408-42b2-8cc8-d5ed76e8184a" containerID="f0336567ba3084284a1acaa01860eda6007c6c02eb71d9afcce8b1c7180c726b" exitCode=0 Jul 22 03:53:39.935642 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:39.935596 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c713ac/rate-test-model-c713ac-kserve-55cb595c9f-4qmnn" event={"ID":"4adaf440-a408-42b2-8cc8-d5ed76e8184a","Type":"ContainerDied","Data":"f0336567ba3084284a1acaa01860eda6007c6c02eb71d9afcce8b1c7180c726b"} Jul 22 03:53:39.935642 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:39.935606 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c713ac/rate-test-model-c713ac-kserve-55cb595c9f-4qmnn" Jul 22 03:53:39.935642 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:39.935622 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c713ac/rate-test-model-c713ac-kserve-55cb595c9f-4qmnn" event={"ID":"4adaf440-a408-42b2-8cc8-d5ed76e8184a","Type":"ContainerDied","Data":"a98510b39185135fa8db713f87aa3c39c5d01f08a308335491bf00166a681ac8"} Jul 22 03:53:39.935642 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:39.935637 2556 scope.go:117] "RemoveContainer" containerID="f0336567ba3084284a1acaa01860eda6007c6c02eb71d9afcce8b1c7180c726b" Jul 22 03:53:39.944870 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:39.944855 2556 scope.go:117] "RemoveContainer" containerID="f0336567ba3084284a1acaa01860eda6007c6c02eb71d9afcce8b1c7180c726b" Jul 22 03:53:39.945130 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:53:39.945111 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0336567ba3084284a1acaa01860eda6007c6c02eb71d9afcce8b1c7180c726b\": container with ID starting with f0336567ba3084284a1acaa01860eda6007c6c02eb71d9afcce8b1c7180c726b not found: ID does not exist" containerID="f0336567ba3084284a1acaa01860eda6007c6c02eb71d9afcce8b1c7180c726b" Jul 22 03:53:39.945177 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:39.945139 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0336567ba3084284a1acaa01860eda6007c6c02eb71d9afcce8b1c7180c726b"} err="failed to get container status \"f0336567ba3084284a1acaa01860eda6007c6c02eb71d9afcce8b1c7180c726b\": rpc error: code = NotFound desc = could not find container \"f0336567ba3084284a1acaa01860eda6007c6c02eb71d9afcce8b1c7180c726b\": container with ID starting with f0336567ba3084284a1acaa01860eda6007c6c02eb71d9afcce8b1c7180c726b not found: ID does not exist" Jul 22 03:53:39.956635 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:39.956609 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c713ac/rate-test-model-c713ac-kserve-55cb595c9f-4qmnn"] Jul 22 03:53:39.962793 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:39.962774 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-c713ac/rate-test-model-c713ac-kserve-55cb595c9f-4qmnn"] Jul 22 03:53:40.024378 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:40.024352 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4adaf440-a408-42b2-8cc8-d5ed76e8184a" path="/var/lib/kubelet/pods/4adaf440-a408-42b2-8cc8-d5ed76e8184a/volumes" Jul 22 03:53:44.731629 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:44.731584 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:53:44.831773 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:44.831734 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:53:54.657117 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:54.657081 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:53:58.032660 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:58.032630 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:53:59.266460 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:59.266430 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:53:59.657870 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:53:59.657838 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:54:05.596931 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:05.596885 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:54:05.704297 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:05.704262 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:54:05.835026 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:05.834993 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4"] Jul 22 03:54:05.835319 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:05.835295 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" podUID="b0b0a353-ccc8-4f2f-9f11-a0fe24e16341" containerName="istio-proxy" containerID="cri-o://1b9c4cb43414c4ce5d7a1cb7e3cb0fb46ac07e3af940f906e284312f484d0df9" gracePeriod=30 Jul 22 03:54:05.864993 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:05.864879 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:54:06.099571 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:06.099517 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:54:06.965108 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:06.965069 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:54:07.050094 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:07.050060 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:54:11.101469 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.101405 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:54:11.114849 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.114819 2556 generic.go:356] "Generic (PLEG): container finished" podID="b0b0a353-ccc8-4f2f-9f11-a0fe24e16341" containerID="1b9c4cb43414c4ce5d7a1cb7e3cb0fb46ac07e3af940f906e284312f484d0df9" exitCode=0 Jul 22 03:54:11.115030 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.114878 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" Jul 22 03:54:11.115030 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.114902 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" event={"ID":"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341","Type":"ContainerDied","Data":"1b9c4cb43414c4ce5d7a1cb7e3cb0fb46ac07e3af940f906e284312f484d0df9"} Jul 22 03:54:11.115030 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.114995 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4" event={"ID":"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341","Type":"ContainerDied","Data":"b0de11f8bed158bf2dbb1014a056b2e4fb967b86bfe664c836f3f4d092eb007d"} Jul 22 03:54:11.115030 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.115020 2556 scope.go:117] "RemoveContainer" containerID="1b9c4cb43414c4ce5d7a1cb7e3cb0fb46ac07e3af940f906e284312f484d0df9" Jul 22 03:54:11.125091 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.125071 2556 scope.go:117] "RemoveContainer" containerID="e7bf63f322348fe590f698f35b1d00db28d89bc4b05f97bec9445ac242efbce4" Jul 22 03:54:11.133533 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.133516 2556 scope.go:117] "RemoveContainer" containerID="1b9c4cb43414c4ce5d7a1cb7e3cb0fb46ac07e3af940f906e284312f484d0df9" Jul 22 03:54:11.133805 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:54:11.133782 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b9c4cb43414c4ce5d7a1cb7e3cb0fb46ac07e3af940f906e284312f484d0df9\": container with ID starting with 1b9c4cb43414c4ce5d7a1cb7e3cb0fb46ac07e3af940f906e284312f484d0df9 not found: ID does not exist" containerID="1b9c4cb43414c4ce5d7a1cb7e3cb0fb46ac07e3af940f906e284312f484d0df9" Jul 22 03:54:11.133877 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.133819 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b9c4cb43414c4ce5d7a1cb7e3cb0fb46ac07e3af940f906e284312f484d0df9"} err="failed to get container status \"1b9c4cb43414c4ce5d7a1cb7e3cb0fb46ac07e3af940f906e284312f484d0df9\": rpc error: code = NotFound desc = could not find container \"1b9c4cb43414c4ce5d7a1cb7e3cb0fb46ac07e3af940f906e284312f484d0df9\": container with ID starting with 1b9c4cb43414c4ce5d7a1cb7e3cb0fb46ac07e3af940f906e284312f484d0df9 not found: ID does not exist" Jul 22 03:54:11.133877 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.133844 2556 scope.go:117] "RemoveContainer" containerID="e7bf63f322348fe590f698f35b1d00db28d89bc4b05f97bec9445ac242efbce4" Jul 22 03:54:11.134107 ip-10-0-133-13 kubenswrapper[2556]: E0722 03:54:11.134086 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7bf63f322348fe590f698f35b1d00db28d89bc4b05f97bec9445ac242efbce4\": container with ID starting with e7bf63f322348fe590f698f35b1d00db28d89bc4b05f97bec9445ac242efbce4 not found: ID does not exist" containerID="e7bf63f322348fe590f698f35b1d00db28d89bc4b05f97bec9445ac242efbce4" Jul 22 03:54:11.134169 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.134115 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7bf63f322348fe590f698f35b1d00db28d89bc4b05f97bec9445ac242efbce4"} err="failed to get container status \"e7bf63f322348fe590f698f35b1d00db28d89bc4b05f97bec9445ac242efbce4\": rpc error: code = NotFound desc = could not find container \"e7bf63f322348fe590f698f35b1d00db28d89bc4b05f97bec9445ac242efbce4\": container with ID starting with e7bf63f322348fe590f698f35b1d00db28d89bc4b05f97bec9445ac242efbce4 not found: ID does not exist" Jul 22 03:54:11.197627 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.197587 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-workload-certs\") pod \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\" (UID: \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\") " Jul 22 03:54:11.197799 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.197646 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-istiod-ca-cert\") pod \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\" (UID: \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\") " Jul 22 03:54:11.197799 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.197682 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-workload-socket\") pod \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\" (UID: \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\") " Jul 22 03:54:11.197799 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.197699 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-istio-data\") pod \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\" (UID: \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\") " Jul 22 03:54:11.197799 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.197749 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-istio-token\") pod \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\" (UID: \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\") " Jul 22 03:54:11.197799 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.197782 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-istio-envoy\") pod \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\" (UID: \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\") " Jul 22 03:54:11.198101 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.197837 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lhzzs\" (UniqueName: \"kubernetes.io/projected/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-kube-api-access-lhzzs\") pod \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\" (UID: \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\") " Jul 22 03:54:11.198101 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.197858 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b0b0a353-ccc8-4f2f-9f11-a0fe24e16341" (UID: "b0b0a353-ccc8-4f2f-9f11-a0fe24e16341"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:54:11.198101 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.197870 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-istio-podinfo\") pod \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\" (UID: \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\") " Jul 22 03:54:11.198101 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.197921 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-credential-socket\") pod \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\" (UID: \"b0b0a353-ccc8-4f2f-9f11-a0fe24e16341\") " Jul 22 03:54:11.198101 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.198026 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b0b0a353-ccc8-4f2f-9f11-a0fe24e16341" (UID: "b0b0a353-ccc8-4f2f-9f11-a0fe24e16341"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:54:11.198351 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.198184 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b0b0a353-ccc8-4f2f-9f11-a0fe24e16341" (UID: "b0b0a353-ccc8-4f2f-9f11-a0fe24e16341"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:54:11.198406 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.198389 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b0b0a353-ccc8-4f2f-9f11-a0fe24e16341" (UID: "b0b0a353-ccc8-4f2f-9f11-a0fe24e16341"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:54:11.198462 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.198435 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-workload-certs\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:54:11.198462 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.198457 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-istiod-ca-cert\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:54:11.198558 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.198472 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-workload-socket\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:54:11.198694 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.198669 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-istio-data" (OuterVolumeSpecName: "istio-data") pod "b0b0a353-ccc8-4f2f-9f11-a0fe24e16341" (UID: "b0b0a353-ccc8-4f2f-9f11-a0fe24e16341"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:54:11.199986 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.199936 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-istio-token" (OuterVolumeSpecName: "istio-token") pod "b0b0a353-ccc8-4f2f-9f11-a0fe24e16341" (UID: "b0b0a353-ccc8-4f2f-9f11-a0fe24e16341"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:54:11.200087 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.200058 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-kube-api-access-lhzzs" (OuterVolumeSpecName: "kube-api-access-lhzzs") pod "b0b0a353-ccc8-4f2f-9f11-a0fe24e16341" (UID: "b0b0a353-ccc8-4f2f-9f11-a0fe24e16341"). InnerVolumeSpecName "kube-api-access-lhzzs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:54:11.200348 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.200319 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b0b0a353-ccc8-4f2f-9f11-a0fe24e16341" (UID: "b0b0a353-ccc8-4f2f-9f11-a0fe24e16341"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 03:54:11.200448 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.200404 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b0b0a353-ccc8-4f2f-9f11-a0fe24e16341" (UID: "b0b0a353-ccc8-4f2f-9f11-a0fe24e16341"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:54:11.299806 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.299759 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lhzzs\" (UniqueName: \"kubernetes.io/projected/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-kube-api-access-lhzzs\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:54:11.299806 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.299801 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-istio-podinfo\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:54:11.299806 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.299811 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-credential-socket\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:54:11.299806 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.299820 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-istio-data\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:54:11.300103 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.299829 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-istio-token\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:54:11.300103 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.299837 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341-istio-envoy\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:54:11.438693 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.438659 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4"] Jul 22 03:54:11.443075 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:11.443047 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-c713ac-openshift-default-ff59d78c5-thcr4"] Jul 22 03:54:12.025025 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:12.024980 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0b0a353-ccc8-4f2f-9f11-a0fe24e16341" path="/var/lib/kubelet/pods/b0b0a353-ccc8-4f2f-9f11-a0fe24e16341/volumes" Jul 22 03:54:16.687884 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:16.687841 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:54:19.434403 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:19.434371 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5m587"] Jul 22 03:54:21.280205 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:21.280169 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:54:21.666848 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:21.666813 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:54:26.815274 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:26.815233 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:54:27.793077 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:27.793034 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:54:27.893402 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:27.893373 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:54:28.037034 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:28.036996 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs"] Jul 22 03:54:28.037246 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:28.037215 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" podUID="79ca697d-2b42-4e98-985d-6d9797094db3" containerName="istio-proxy" containerID="cri-o://1fe8d477b39018290ecd420544cff5777e2e6dcc38d12f5d141d1c56a81c4c7e" gracePeriod=30 Jul 22 03:54:28.058312 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:28.058237 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:54:28.277653 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:28.277607 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:54:33.244527 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:33.244494 2556 generic.go:356] "Generic (PLEG): container finished" podID="79ca697d-2b42-4e98-985d-6d9797094db3" containerID="1fe8d477b39018290ecd420544cff5777e2e6dcc38d12f5d141d1c56a81c4c7e" exitCode=0 Jul 22 03:54:33.244856 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:33.244566 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" event={"ID":"79ca697d-2b42-4e98-985d-6d9797094db3","Type":"ContainerDied","Data":"1fe8d477b39018290ecd420544cff5777e2e6dcc38d12f5d141d1c56a81c4c7e"} Jul 22 03:54:33.244856 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:33.244640 2556 scope.go:117] "RemoveContainer" containerID="c55f216e8e65f553a06c47d67ec4cf4484489aa07e477a011cb072489d269dea" Jul 22 03:54:33.289455 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:33.289433 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:54:33.301267 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:33.301225 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79ca697d-2b42-4e98-985d-6d9797094db3-credential-socket\") pod \"79ca697d-2b42-4e98-985d-6d9797094db3\" (UID: \"79ca697d-2b42-4e98-985d-6d9797094db3\") " Jul 22 03:54:33.301267 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:33.301260 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79ca697d-2b42-4e98-985d-6d9797094db3-istio-podinfo\") pod \"79ca697d-2b42-4e98-985d-6d9797094db3\" (UID: \"79ca697d-2b42-4e98-985d-6d9797094db3\") " Jul 22 03:54:33.301408 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:33.301277 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79ca697d-2b42-4e98-985d-6d9797094db3-istio-envoy\") pod \"79ca697d-2b42-4e98-985d-6d9797094db3\" (UID: \"79ca697d-2b42-4e98-985d-6d9797094db3\") " Jul 22 03:54:33.301408 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:33.301301 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79ca697d-2b42-4e98-985d-6d9797094db3-workload-certs\") pod \"79ca697d-2b42-4e98-985d-6d9797094db3\" (UID: \"79ca697d-2b42-4e98-985d-6d9797094db3\") " Jul 22 03:54:33.301408 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:33.301330 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79ca697d-2b42-4e98-985d-6d9797094db3-workload-socket\") pod \"79ca697d-2b42-4e98-985d-6d9797094db3\" (UID: \"79ca697d-2b42-4e98-985d-6d9797094db3\") " Jul 22 03:54:33.301408 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:33.301369 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79ca697d-2b42-4e98-985d-6d9797094db3-istio-data\") pod \"79ca697d-2b42-4e98-985d-6d9797094db3\" (UID: \"79ca697d-2b42-4e98-985d-6d9797094db3\") " Jul 22 03:54:33.301408 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:33.301395 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79ca697d-2b42-4e98-985d-6d9797094db3-istio-token\") pod \"79ca697d-2b42-4e98-985d-6d9797094db3\" (UID: \"79ca697d-2b42-4e98-985d-6d9797094db3\") " Jul 22 03:54:33.301657 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:33.301428 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6kg6z\" (UniqueName: \"kubernetes.io/projected/79ca697d-2b42-4e98-985d-6d9797094db3-kube-api-access-6kg6z\") pod \"79ca697d-2b42-4e98-985d-6d9797094db3\" (UID: \"79ca697d-2b42-4e98-985d-6d9797094db3\") " Jul 22 03:54:33.301657 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:33.301481 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79ca697d-2b42-4e98-985d-6d9797094db3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "79ca697d-2b42-4e98-985d-6d9797094db3" (UID: "79ca697d-2b42-4e98-985d-6d9797094db3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:54:33.301657 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:33.301508 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79ca697d-2b42-4e98-985d-6d9797094db3-istiod-ca-cert\") pod \"79ca697d-2b42-4e98-985d-6d9797094db3\" (UID: \"79ca697d-2b42-4e98-985d-6d9797094db3\") " Jul 22 03:54:33.301821 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:33.301795 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79ca697d-2b42-4e98-985d-6d9797094db3-credential-socket\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:54:33.301821 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:33.301788 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79ca697d-2b42-4e98-985d-6d9797094db3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "79ca697d-2b42-4e98-985d-6d9797094db3" (UID: "79ca697d-2b42-4e98-985d-6d9797094db3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:54:33.302059 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:33.302003 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79ca697d-2b42-4e98-985d-6d9797094db3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "79ca697d-2b42-4e98-985d-6d9797094db3" (UID: "79ca697d-2b42-4e98-985d-6d9797094db3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:54:33.302181 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:33.302159 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79ca697d-2b42-4e98-985d-6d9797094db3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "79ca697d-2b42-4e98-985d-6d9797094db3" (UID: "79ca697d-2b42-4e98-985d-6d9797094db3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:54:33.302312 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:33.302281 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79ca697d-2b42-4e98-985d-6d9797094db3-istio-data" (OuterVolumeSpecName: "istio-data") pod "79ca697d-2b42-4e98-985d-6d9797094db3" (UID: "79ca697d-2b42-4e98-985d-6d9797094db3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:54:33.303578 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:33.303553 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79ca697d-2b42-4e98-985d-6d9797094db3-istio-token" (OuterVolumeSpecName: "istio-token") pod "79ca697d-2b42-4e98-985d-6d9797094db3" (UID: "79ca697d-2b42-4e98-985d-6d9797094db3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:54:33.303658 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:33.303583 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79ca697d-2b42-4e98-985d-6d9797094db3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "79ca697d-2b42-4e98-985d-6d9797094db3" (UID: "79ca697d-2b42-4e98-985d-6d9797094db3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:54:33.303658 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:33.303593 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79ca697d-2b42-4e98-985d-6d9797094db3-kube-api-access-6kg6z" (OuterVolumeSpecName: "kube-api-access-6kg6z") pod "79ca697d-2b42-4e98-985d-6d9797094db3" (UID: "79ca697d-2b42-4e98-985d-6d9797094db3"). InnerVolumeSpecName "kube-api-access-6kg6z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:54:33.303658 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:33.303586 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/79ca697d-2b42-4e98-985d-6d9797094db3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "79ca697d-2b42-4e98-985d-6d9797094db3" (UID: "79ca697d-2b42-4e98-985d-6d9797094db3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 03:54:33.402745 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:33.402714 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79ca697d-2b42-4e98-985d-6d9797094db3-workload-socket\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:54:33.402745 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:33.402739 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79ca697d-2b42-4e98-985d-6d9797094db3-istio-data\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:54:33.402745 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:33.402747 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79ca697d-2b42-4e98-985d-6d9797094db3-istio-token\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:54:33.403009 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:33.402755 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6kg6z\" (UniqueName: \"kubernetes.io/projected/79ca697d-2b42-4e98-985d-6d9797094db3-kube-api-access-6kg6z\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:54:33.403009 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:33.402765 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79ca697d-2b42-4e98-985d-6d9797094db3-istiod-ca-cert\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:54:33.403009 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:33.402773 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79ca697d-2b42-4e98-985d-6d9797094db3-istio-podinfo\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:54:33.403009 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:33.402781 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79ca697d-2b42-4e98-985d-6d9797094db3-istio-envoy\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:54:33.403009 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:33.402788 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79ca697d-2b42-4e98-985d-6d9797094db3-workload-certs\") on node \"ip-10-0-133-13.ec2.internal\" DevicePath \"\"" Jul 22 03:54:33.883657 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:33.883621 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-4prjh_63db064d-1b10-4979-a48c-0e93223b5304/manager/0.log" Jul 22 03:54:33.997052 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:33.997025 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-nnrgm_612328a2-521a-44d1-aa8a-31785bdd3998/manager/0.log" Jul 22 03:54:34.251474 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:34.251396 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" event={"ID":"79ca697d-2b42-4e98-985d-6d9797094db3","Type":"ContainerDied","Data":"64de8a94b37fa2e03fa8a95bcc206e291b13a4e99f878c437f87062e81207ec1"} Jul 22 03:54:34.251474 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:34.251424 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs" Jul 22 03:54:34.251474 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:34.251452 2556 scope.go:117] "RemoveContainer" containerID="1fe8d477b39018290ecd420544cff5777e2e6dcc38d12f5d141d1c56a81c4c7e" Jul 22 03:54:34.272867 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:34.272842 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs"] Jul 22 03:54:34.278072 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:34.278052 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-be39c1-openshift-default-6dcb7df46c-qldfs"] Jul 22 03:54:34.469855 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:34.469831 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5fb7f869bf-gsrx5_4c37b2ca-8721-4147-bf19-eb95bf2d1f4d/copy-manifests/0.log" Jul 22 03:54:34.477736 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:34.477711 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5fb7f869bf-gsrx5_4c37b2ca-8721-4147-bf19-eb95bf2d1f4d/manager/0.log" Jul 22 03:54:36.024157 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:36.024125 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79ca697d-2b42-4e98-985d-6d9797094db3" path="/var/lib/kubelet/pods/79ca697d-2b42-4e98-985d-6d9797094db3/volumes" Jul 22 03:54:36.174822 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:36.174792 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fhnsh_80d312a3-2e8b-47ee-9a27-bab66ba1a670/util/0.log" Jul 22 03:54:36.181291 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:36.181275 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fhnsh_80d312a3-2e8b-47ee-9a27-bab66ba1a670/pull/0.log" Jul 22 03:54:36.187312 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:36.187292 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fhnsh_80d312a3-2e8b-47ee-9a27-bab66ba1a670/extract/0.log" Jul 22 03:54:36.298078 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:36.298018 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lgx6j_eee6f75e-6170-48c2-ac4f-8e340679dde2/util/0.log" Jul 22 03:54:36.304737 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:36.304719 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lgx6j_eee6f75e-6170-48c2-ac4f-8e340679dde2/pull/0.log" Jul 22 03:54:36.310754 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:36.310734 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lgx6j_eee6f75e-6170-48c2-ac4f-8e340679dde2/extract/0.log" Jul 22 03:54:36.417388 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:36.417359 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q6hgs_49081396-f915-404a-a95a-1625bba5456b/util/0.log" Jul 22 03:54:36.423627 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:36.423603 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q6hgs_49081396-f915-404a-a95a-1625bba5456b/pull/0.log" Jul 22 03:54:36.430385 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:36.430369 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q6hgs_49081396-f915-404a-a95a-1625bba5456b/extract/0.log" Jul 22 03:54:36.540350 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:36.540324 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rwqdw_4d7095a2-1171-4232-8838-b98a0848a484/util/0.log" Jul 22 03:54:36.546138 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:36.546120 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rwqdw_4d7095a2-1171-4232-8838-b98a0848a484/pull/0.log" Jul 22 03:54:36.552213 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:36.552157 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rwqdw_4d7095a2-1171-4232-8838-b98a0848a484/extract/0.log" Jul 22 03:54:36.807886 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:36.807786 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-g58pn_c7d619e0-6915-49f1-b700-515c176c680a/manager/0.log" Jul 22 03:54:36.917543 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:36.917513 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-jkmck_df73b24e-a59c-4074-a0ff-afe832f72d32/manager/0.log" Jul 22 03:54:37.131654 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:37.131600 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-wmxqz_8094bb22-572a-46e3-8c00-2e3ee71387e4/registry-server/0.log" Jul 22 03:54:37.248647 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:37.248593 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-zvqzj_50ce2458-b52d-4a77-bdab-da22cc84dafd/manager/0.log" Jul 22 03:54:37.360067 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:37.360039 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-5m587_4e4f6830-a3fd-4bd8-bd05-b5827da572f3/limitador/0.log" Jul 22 03:54:37.473709 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:37.473635 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-flnml_abbcc799-0450-425a-be2d-f8f2324e6a23/manager/0.log" Jul 22 03:54:37.814795 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:37.814716 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf45mh7n_7e03f9ba-5161-404d-998b-c0dd42f50790/istio-proxy/0.log" Jul 22 03:54:38.038816 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:38.038789 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59676447d6-27d27_4b59b506-ebea-4869-8530-ff57fdee114b/kube-auth-proxy/0.log" Jul 22 03:54:38.260175 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:38.260147 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-4c9nn_221f388a-ed5d-4c97-a320-1339d242c437/istio-proxy/1.log" Jul 22 03:54:39.070078 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:39.070040 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-rx4vr_f17974a8-c3f1-42ed-bc11-eec56e94281d/main/0.log" Jul 22 03:54:39.404072 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:39.403975 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-vzhkz_131ea682-fde2-41ab-9593-490f403e63b8/main/0.log" Jul 22 03:54:44.794899 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:44.794866 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8cvpq/must-gather-q874s"] Jul 22 03:54:44.795310 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:44.795260 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0b0a353-ccc8-4f2f-9f11-a0fe24e16341" containerName="istio-proxy" Jul 22 03:54:44.795310 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:44.795272 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0b0a353-ccc8-4f2f-9f11-a0fe24e16341" containerName="istio-proxy" Jul 22 03:54:44.795310 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:44.795284 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4adaf440-a408-42b2-8cc8-d5ed76e8184a" containerName="main" Jul 22 03:54:44.795310 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:44.795290 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="4adaf440-a408-42b2-8cc8-d5ed76e8184a" containerName="main" Jul 22 03:54:44.795310 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:44.795302 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79ca697d-2b42-4e98-985d-6d9797094db3" containerName="istio-proxy" Jul 22 03:54:44.795310 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:44.795308 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="79ca697d-2b42-4e98-985d-6d9797094db3" containerName="istio-proxy" Jul 22 03:54:44.795553 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:44.795316 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79ca697d-2b42-4e98-985d-6d9797094db3" containerName="istio-proxy" Jul 22 03:54:44.795553 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:44.795321 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="79ca697d-2b42-4e98-985d-6d9797094db3" containerName="istio-proxy" Jul 22 03:54:44.795553 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:44.795394 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="4adaf440-a408-42b2-8cc8-d5ed76e8184a" containerName="main" Jul 22 03:54:44.795553 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:44.795402 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0b0a353-ccc8-4f2f-9f11-a0fe24e16341" containerName="istio-proxy" Jul 22 03:54:44.795553 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:44.795409 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="79ca697d-2b42-4e98-985d-6d9797094db3" containerName="istio-proxy" Jul 22 03:54:44.795553 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:44.795416 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0b0a353-ccc8-4f2f-9f11-a0fe24e16341" containerName="istio-proxy" Jul 22 03:54:44.795553 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:44.795471 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0b0a353-ccc8-4f2f-9f11-a0fe24e16341" containerName="istio-proxy" Jul 22 03:54:44.795553 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:44.795477 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0b0a353-ccc8-4f2f-9f11-a0fe24e16341" containerName="istio-proxy" Jul 22 03:54:44.795553 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:44.795535 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="79ca697d-2b42-4e98-985d-6d9797094db3" containerName="istio-proxy" Jul 22 03:54:44.800270 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:44.800250 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8cvpq/must-gather-q874s" Jul 22 03:54:44.802903 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:44.802880 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8cvpq\"/\"openshift-service-ca.crt\"" Jul 22 03:54:44.803039 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:44.802917 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8cvpq\"/\"default-dockercfg-c29st\"" Jul 22 03:54:44.804107 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:44.804091 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8cvpq\"/\"kube-root-ca.crt\"" Jul 22 03:54:44.809098 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:44.809071 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:54:44.811318 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:44.811295 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8cvpq/must-gather-q874s"] Jul 22 03:54:44.901563 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:44.901530 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" limit=1024 Jul 22 03:54:44.905797 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:44.905774 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/31ef69c4-960b-467e-83f6-c654e42c6ca5-must-gather-output\") pod \"must-gather-q874s\" (UID: \"31ef69c4-960b-467e-83f6-c654e42c6ca5\") " pod="openshift-must-gather-8cvpq/must-gather-q874s" Jul 22 03:54:44.905881 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:44.905819 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7jln\" (UniqueName: \"kubernetes.io/projected/31ef69c4-960b-467e-83f6-c654e42c6ca5-kube-api-access-f7jln\") pod \"must-gather-q874s\" (UID: \"31ef69c4-960b-467e-83f6-c654e42c6ca5\") " pod="openshift-must-gather-8cvpq/must-gather-q874s" Jul 22 03:54:45.006874 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:45.006843 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/31ef69c4-960b-467e-83f6-c654e42c6ca5-must-gather-output\") pod \"must-gather-q874s\" (UID: \"31ef69c4-960b-467e-83f6-c654e42c6ca5\") " pod="openshift-must-gather-8cvpq/must-gather-q874s" Jul 22 03:54:45.007071 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:45.006890 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7jln\" (UniqueName: \"kubernetes.io/projected/31ef69c4-960b-467e-83f6-c654e42c6ca5-kube-api-access-f7jln\") pod \"must-gather-q874s\" (UID: \"31ef69c4-960b-467e-83f6-c654e42c6ca5\") " pod="openshift-must-gather-8cvpq/must-gather-q874s" Jul 22 03:54:45.007191 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:45.007174 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/31ef69c4-960b-467e-83f6-c654e42c6ca5-must-gather-output\") pod \"must-gather-q874s\" (UID: \"31ef69c4-960b-467e-83f6-c654e42c6ca5\") " pod="openshift-must-gather-8cvpq/must-gather-q874s" Jul 22 03:54:45.015484 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:45.015462 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7jln\" (UniqueName: \"kubernetes.io/projected/31ef69c4-960b-467e-83f6-c654e42c6ca5-kube-api-access-f7jln\") pod \"must-gather-q874s\" (UID: \"31ef69c4-960b-467e-83f6-c654e42c6ca5\") " pod="openshift-must-gather-8cvpq/must-gather-q874s" Jul 22 03:54:45.110414 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:45.110380 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8cvpq/must-gather-q874s" Jul 22 03:54:45.252666 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:45.252639 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8cvpq/must-gather-q874s"] Jul 22 03:54:45.254450 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:54:45.254424 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod31ef69c4_960b_467e_83f6_c654e42c6ca5.slice/crio-640a892947efe600d292689af61e5bd628108a3d164ebee96c75d791485ba2ad WatchSource:0}: Error finding container 640a892947efe600d292689af61e5bd628108a3d164ebee96c75d791485ba2ad: Status 404 returned error can't find the container with id 640a892947efe600d292689af61e5bd628108a3d164ebee96c75d791485ba2ad Jul 22 03:54:45.256401 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:45.256385 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 03:54:45.309694 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:45.309657 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8cvpq/must-gather-q874s" event={"ID":"31ef69c4-960b-467e-83f6-c654e42c6ca5","Type":"ContainerStarted","Data":"640a892947efe600d292689af61e5bd628108a3d164ebee96c75d791485ba2ad"} Jul 22 03:54:46.319912 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:46.319877 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8cvpq/must-gather-q874s" event={"ID":"31ef69c4-960b-467e-83f6-c654e42c6ca5","Type":"ContainerStarted","Data":"14fd593ceb16fcc51d2a2396f72d929c03528d24f5104d2c90746bd30a6b7f33"} Jul 22 03:54:47.330448 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:47.330415 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8cvpq/must-gather-q874s" event={"ID":"31ef69c4-960b-467e-83f6-c654e42c6ca5","Type":"ContainerStarted","Data":"927060e06e2624149d4ef6e90260c7cae26773abbc781c8c5b9103f0dfd76924"} Jul 22 03:54:47.350115 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:47.350051 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8cvpq/must-gather-q874s" podStartSLOduration=2.392642296 podStartE2EDuration="3.350029532s" totalImagesPullingTime="957.387236ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:54:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:54:47.349591407 +0000 UTC m=+2827.798574660" watchObservedRunningTime="2026-07-22 03:54:47.350029532 +0000 UTC m=+2827.799012780" Jul 22 03:54:47.820413 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:47.820388 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-qhg45_1297a647-da14-4c7b-bf94-6f4f4e679c31/global-pull-secret-syncer/0.log" Jul 22 03:54:47.917820 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:47.917778 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-68xqw_db34ad96-aec1-457b-8c55-61fea9c7f14a/konnectivity-agent/0.log" Jul 22 03:54:48.048447 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:48.048417 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-13.ec2.internal_d5cf9dba7131053469ef1265ca066507/haproxy/0.log" Jul 22 03:54:52.293726 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:52.293667 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fhnsh_80d312a3-2e8b-47ee-9a27-bab66ba1a670/extract/0.log" Jul 22 03:54:52.321983 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:52.321939 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fhnsh_80d312a3-2e8b-47ee-9a27-bab66ba1a670/util/0.log" Jul 22 03:54:52.350033 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:52.349979 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fhnsh_80d312a3-2e8b-47ee-9a27-bab66ba1a670/pull/0.log" Jul 22 03:54:52.390328 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:52.390228 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lgx6j_eee6f75e-6170-48c2-ac4f-8e340679dde2/extract/0.log" Jul 22 03:54:52.421092 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:52.421054 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lgx6j_eee6f75e-6170-48c2-ac4f-8e340679dde2/util/0.log" Jul 22 03:54:52.451590 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:52.451561 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lgx6j_eee6f75e-6170-48c2-ac4f-8e340679dde2/pull/0.log" Jul 22 03:54:52.484102 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:52.484066 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q6hgs_49081396-f915-404a-a95a-1625bba5456b/extract/0.log" Jul 22 03:54:52.510776 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:52.510720 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q6hgs_49081396-f915-404a-a95a-1625bba5456b/util/0.log" Jul 22 03:54:52.538083 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:52.538050 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q6hgs_49081396-f915-404a-a95a-1625bba5456b/pull/0.log" Jul 22 03:54:52.584863 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:52.584831 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rwqdw_4d7095a2-1171-4232-8838-b98a0848a484/extract/0.log" Jul 22 03:54:52.608757 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:52.608727 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rwqdw_4d7095a2-1171-4232-8838-b98a0848a484/util/0.log" Jul 22 03:54:52.638696 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:52.638662 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rwqdw_4d7095a2-1171-4232-8838-b98a0848a484/pull/0.log" Jul 22 03:54:53.307492 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:53.307460 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-g58pn_c7d619e0-6915-49f1-b700-515c176c680a/manager/0.log" Jul 22 03:54:53.333837 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:53.333804 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-jkmck_df73b24e-a59c-4074-a0ff-afe832f72d32/manager/0.log" Jul 22 03:54:53.408329 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:53.408123 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-wmxqz_8094bb22-572a-46e3-8c00-2e3ee71387e4/registry-server/0.log" Jul 22 03:54:53.609028 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:53.607807 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-zvqzj_50ce2458-b52d-4a77-bdab-da22cc84dafd/manager/0.log" Jul 22 03:54:53.634570 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:53.634537 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-5m587_4e4f6830-a3fd-4bd8-bd05-b5827da572f3/limitador/0.log" Jul 22 03:54:53.720312 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:53.720275 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-flnml_abbcc799-0450-425a-be2d-f8f2324e6a23/manager/0.log" Jul 22 03:54:55.472611 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:55.472580 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-gh6jj_168e0b81-ebe6-402a-99c6-e7881d11718a/kube-state-metrics/0.log" Jul 22 03:54:55.494513 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:55.494483 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-gh6jj_168e0b81-ebe6-402a-99c6-e7881d11718a/kube-rbac-proxy-main/0.log" Jul 22 03:54:55.524153 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:55.524126 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-gh6jj_168e0b81-ebe6-402a-99c6-e7881d11718a/kube-rbac-proxy-self/0.log" Jul 22 03:54:55.580364 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:55.580321 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-mpzw6_48041ebc-ec47-4b82-86b1-50b154e9b453/monitoring-plugin/0.log" Jul 22 03:54:55.837190 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:55.837161 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-blt8n_a8f83db4-079b-44ff-a400-0762ba4a1dfd/node-exporter/0.log" Jul 22 03:54:55.901263 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:55.901234 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-blt8n_a8f83db4-079b-44ff-a400-0762ba4a1dfd/kube-rbac-proxy/0.log" Jul 22 03:54:55.944214 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:55.944188 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-blt8n_a8f83db4-079b-44ff-a400-0762ba4a1dfd/init-textfile/0.log" Jul 22 03:54:56.230658 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:56.230580 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-jf5kv_465a5cf5-d8c4-46bb-905e-7992138abf82/prometheus-operator/0.log" Jul 22 03:54:56.253424 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:56.253397 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-jf5kv_465a5cf5-d8c4-46bb-905e-7992138abf82/kube-rbac-proxy/0.log" Jul 22 03:54:56.281719 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:56.281691 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-ktf9k_7b4c464a-27f8-4ed7-bf38-1ef06032ac5a/prometheus-operator-admission-webhook/0.log" Jul 22 03:54:57.486537 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:57.486496 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8cvpq/perf-node-gather-daemonset-s2kjn"] Jul 22 03:54:57.492207 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:57.492173 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8cvpq/perf-node-gather-daemonset-s2kjn" Jul 22 03:54:57.499256 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:57.499224 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8cvpq/perf-node-gather-daemonset-s2kjn"] Jul 22 03:54:57.645077 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:57.645041 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5792409e-b818-4107-a4fa-791ed6395b90-sys\") pod \"perf-node-gather-daemonset-s2kjn\" (UID: \"5792409e-b818-4107-a4fa-791ed6395b90\") " pod="openshift-must-gather-8cvpq/perf-node-gather-daemonset-s2kjn" Jul 22 03:54:57.645236 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:57.645100 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcd9w\" (UniqueName: \"kubernetes.io/projected/5792409e-b818-4107-a4fa-791ed6395b90-kube-api-access-vcd9w\") pod \"perf-node-gather-daemonset-s2kjn\" (UID: \"5792409e-b818-4107-a4fa-791ed6395b90\") " pod="openshift-must-gather-8cvpq/perf-node-gather-daemonset-s2kjn" Jul 22 03:54:57.645236 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:57.645188 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5792409e-b818-4107-a4fa-791ed6395b90-lib-modules\") pod \"perf-node-gather-daemonset-s2kjn\" (UID: \"5792409e-b818-4107-a4fa-791ed6395b90\") " pod="openshift-must-gather-8cvpq/perf-node-gather-daemonset-s2kjn" Jul 22 03:54:57.645314 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:57.645237 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5792409e-b818-4107-a4fa-791ed6395b90-proc\") pod \"perf-node-gather-daemonset-s2kjn\" (UID: \"5792409e-b818-4107-a4fa-791ed6395b90\") " pod="openshift-must-gather-8cvpq/perf-node-gather-daemonset-s2kjn" Jul 22 03:54:57.645314 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:57.645267 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5792409e-b818-4107-a4fa-791ed6395b90-podres\") pod \"perf-node-gather-daemonset-s2kjn\" (UID: \"5792409e-b818-4107-a4fa-791ed6395b90\") " pod="openshift-must-gather-8cvpq/perf-node-gather-daemonset-s2kjn" Jul 22 03:54:57.746803 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:57.746715 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5792409e-b818-4107-a4fa-791ed6395b90-sys\") pod \"perf-node-gather-daemonset-s2kjn\" (UID: \"5792409e-b818-4107-a4fa-791ed6395b90\") " pod="openshift-must-gather-8cvpq/perf-node-gather-daemonset-s2kjn" Jul 22 03:54:57.747098 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:57.747076 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcd9w\" (UniqueName: \"kubernetes.io/projected/5792409e-b818-4107-a4fa-791ed6395b90-kube-api-access-vcd9w\") pod \"perf-node-gather-daemonset-s2kjn\" (UID: \"5792409e-b818-4107-a4fa-791ed6395b90\") " pod="openshift-must-gather-8cvpq/perf-node-gather-daemonset-s2kjn" Jul 22 03:54:57.747327 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:57.747310 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5792409e-b818-4107-a4fa-791ed6395b90-lib-modules\") pod \"perf-node-gather-daemonset-s2kjn\" (UID: \"5792409e-b818-4107-a4fa-791ed6395b90\") " pod="openshift-must-gather-8cvpq/perf-node-gather-daemonset-s2kjn" Jul 22 03:54:57.747646 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:57.747625 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5792409e-b818-4107-a4fa-791ed6395b90-proc\") pod \"perf-node-gather-daemonset-s2kjn\" (UID: \"5792409e-b818-4107-a4fa-791ed6395b90\") " pod="openshift-must-gather-8cvpq/perf-node-gather-daemonset-s2kjn" Jul 22 03:54:57.747848 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:57.747832 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5792409e-b818-4107-a4fa-791ed6395b90-podres\") pod \"perf-node-gather-daemonset-s2kjn\" (UID: \"5792409e-b818-4107-a4fa-791ed6395b90\") " pod="openshift-must-gather-8cvpq/perf-node-gather-daemonset-s2kjn" Jul 22 03:54:57.748153 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:57.746924 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5792409e-b818-4107-a4fa-791ed6395b90-sys\") pod \"perf-node-gather-daemonset-s2kjn\" (UID: \"5792409e-b818-4107-a4fa-791ed6395b90\") " pod="openshift-must-gather-8cvpq/perf-node-gather-daemonset-s2kjn" Jul 22 03:54:57.748297 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:57.747780 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5792409e-b818-4107-a4fa-791ed6395b90-proc\") pod \"perf-node-gather-daemonset-s2kjn\" (UID: \"5792409e-b818-4107-a4fa-791ed6395b90\") " pod="openshift-must-gather-8cvpq/perf-node-gather-daemonset-s2kjn" Jul 22 03:54:57.748412 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:57.747559 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5792409e-b818-4107-a4fa-791ed6395b90-lib-modules\") pod \"perf-node-gather-daemonset-s2kjn\" (UID: \"5792409e-b818-4107-a4fa-791ed6395b90\") " pod="openshift-must-gather-8cvpq/perf-node-gather-daemonset-s2kjn" Jul 22 03:54:57.748503 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:57.748042 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5792409e-b818-4107-a4fa-791ed6395b90-podres\") pod \"perf-node-gather-daemonset-s2kjn\" (UID: \"5792409e-b818-4107-a4fa-791ed6395b90\") " pod="openshift-must-gather-8cvpq/perf-node-gather-daemonset-s2kjn" Jul 22 03:54:57.755124 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:57.755091 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcd9w\" (UniqueName: \"kubernetes.io/projected/5792409e-b818-4107-a4fa-791ed6395b90-kube-api-access-vcd9w\") pod \"perf-node-gather-daemonset-s2kjn\" (UID: \"5792409e-b818-4107-a4fa-791ed6395b90\") " pod="openshift-must-gather-8cvpq/perf-node-gather-daemonset-s2kjn" Jul 22 03:54:57.786512 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:57.786486 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-mm92k_faa204cb-ed7e-4afe-9115-af504e1c48b6/networking-console-plugin/0.log" Jul 22 03:54:57.808086 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:57.808056 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8cvpq/perf-node-gather-daemonset-s2kjn" Jul 22 03:54:58.010342 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:58.010318 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8cvpq/perf-node-gather-daemonset-s2kjn"] Jul 22 03:54:58.013032 ip-10-0-133-13 kubenswrapper[2556]: W0722 03:54:58.013002 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5792409e_b818_4107_a4fa_791ed6395b90.slice/crio-4e6da83be7510551ccdb9b6332f369e3e8842ea8c61c53c4c1ad85203884a495 WatchSource:0}: Error finding container 4e6da83be7510551ccdb9b6332f369e3e8842ea8c61c53c4c1ad85203884a495: Status 404 returned error can't find the container with id 4e6da83be7510551ccdb9b6332f369e3e8842ea8c61c53c4c1ad85203884a495 Jul 22 03:54:58.434139 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:58.434096 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8cvpq/perf-node-gather-daemonset-s2kjn" event={"ID":"5792409e-b818-4107-a4fa-791ed6395b90","Type":"ContainerStarted","Data":"4fce8e63e1e199b0a926af5ed7e59ca0606144682800cc34d32a1ba7d8d71f58"} Jul 22 03:54:58.434342 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:58.434167 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8cvpq/perf-node-gather-daemonset-s2kjn" event={"ID":"5792409e-b818-4107-a4fa-791ed6395b90","Type":"ContainerStarted","Data":"4e6da83be7510551ccdb9b6332f369e3e8842ea8c61c53c4c1ad85203884a495"} Jul 22 03:54:58.434342 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:58.434223 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8cvpq/perf-node-gather-daemonset-s2kjn" Jul 22 03:54:58.453674 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:58.453613 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8cvpq/perf-node-gather-daemonset-s2kjn" podStartSLOduration=1.453594397 podStartE2EDuration="1.453594397s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:54:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:54:58.449558867 +0000 UTC m=+2838.898542117" watchObservedRunningTime="2026-07-22 03:54:58.453594397 +0000 UTC m=+2838.902577646" Jul 22 03:54:58.922781 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:54:58.922750 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-767c8f5bf5-w88vb_85100ccf-bd8f-4ec5-bd5f-ce0cf7dbd519/console/0.log" Jul 22 03:55:00.560588 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:55:00.560553 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-d8rr4_01af83ea-a187-4abb-ba83-7d45027fb098/dns/0.log" Jul 22 03:55:00.583396 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:55:00.583341 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-d8rr4_01af83ea-a187-4abb-ba83-7d45027fb098/kube-rbac-proxy/0.log" Jul 22 03:55:00.651729 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:55:00.651700 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kkcg5_497ea3ce-7ad8-42f9-a52e-8094ec646cc9/dns-node-resolver/0.log" Jul 22 03:55:01.245032 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:55:01.245004 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-8458747fff-b6gnb_b402f8e9-6401-4160-a5c2-baa9dfbc7cd8/registry/0.log" Jul 22 03:55:01.318409 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:55:01.318377 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rv678_590f0095-ecb1-43e4-9c1e-1c62a87a3b8a/node-ca/0.log" Jul 22 03:55:02.282739 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:55:02.282710 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf45mh7n_7e03f9ba-5161-404d-998b-c0dd42f50790/istio-proxy/0.log" Jul 22 03:55:02.471645 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:55:02.471607 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59676447d6-27d27_4b59b506-ebea-4869-8530-ff57fdee114b/kube-auth-proxy/0.log" Jul 22 03:55:02.558720 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:55:02.558644 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-4c9nn_221f388a-ed5d-4c97-a320-1339d242c437/istio-proxy/0.log" Jul 22 03:55:02.589711 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:55:02.589678 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-4c9nn_221f388a-ed5d-4c97-a320-1339d242c437/istio-proxy/1.log" Jul 22 03:55:03.497499 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:55:03.497470 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qb5ws_57b3d756-0f91-4adb-bfcf-92561b58d767/serve-healthcheck-canary/0.log" Jul 22 03:55:04.203353 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:55:04.203301 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zs5nq_c5dc9285-733c-4cf4-abad-90ec9da9674d/kube-rbac-proxy/0.log" Jul 22 03:55:04.238177 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:55:04.238155 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zs5nq_c5dc9285-733c-4cf4-abad-90ec9da9674d/exporter/0.log" Jul 22 03:55:04.265249 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:55:04.265203 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zs5nq_c5dc9285-733c-4cf4-abad-90ec9da9674d/extractor/0.log" Jul 22 03:55:04.454219 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:55:04.454140 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8cvpq/perf-node-gather-daemonset-s2kjn" Jul 22 03:55:06.704680 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:55:06.704583 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-4prjh_63db064d-1b10-4979-a48c-0e93223b5304/manager/0.log" Jul 22 03:55:06.933224 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:55:06.933188 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-nnrgm_612328a2-521a-44d1-aa8a-31785bdd3998/manager/0.log" Jul 22 03:55:07.230022 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:55:07.229991 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5fb7f869bf-gsrx5_4c37b2ca-8721-4147-bf19-eb95bf2d1f4d/manager/0.log" Jul 22 03:55:07.289712 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:55:07.289688 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5fb7f869bf-gsrx5_4c37b2ca-8721-4147-bf19-eb95bf2d1f4d/copy-manifests/0.log" Jul 22 03:55:07.827292 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:55:07.827255 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf45mh7n_7e03f9ba-5161-404d-998b-c0dd42f50790/istio-proxy/0.log" Jul 22 03:55:07.955536 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:55:07.955500 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59676447d6-27d27_4b59b506-ebea-4869-8530-ff57fdee114b/kube-auth-proxy/0.log" Jul 22 03:55:07.994278 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:55:07.994253 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-4c9nn_221f388a-ed5d-4c97-a320-1339d242c437/istio-proxy/0.log" Jul 22 03:55:08.031690 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:55:08.031662 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-4c9nn_221f388a-ed5d-4c97-a320-1339d242c437/istio-proxy/1.log" Jul 22 03:55:08.758758 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:55:08.758729 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-2nzz6_3c165226-b386-4f76-a5c7-4c9c2204daae/openshift-lws-operator/0.log" Jul 22 03:55:09.208287 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:55:09.208260 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf45mh7n_7e03f9ba-5161-404d-998b-c0dd42f50790/istio-proxy/0.log" Jul 22 03:55:09.346604 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:55:09.346575 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59676447d6-27d27_4b59b506-ebea-4869-8530-ff57fdee114b/kube-auth-proxy/0.log" Jul 22 03:55:09.384806 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:55:09.384772 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-4c9nn_221f388a-ed5d-4c97-a320-1339d242c437/istio-proxy/0.log" Jul 22 03:55:09.424431 ip-10-0-133-13 kubenswrapper[2556]: I0722 03:55:09.424389 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-4c9nn_221f388a-ed5d-4c97-a320-1339d242c437/istio-proxy/1.log"