Jul 13 09:08:35.993487 ip-10-0-129-99 systemd[1]: Starting Kubernetes Kubelet... Jul 13 09:08:36.433279 ip-10-0-129-99 kubenswrapper[2552]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 09:08:36.433279 ip-10-0-129-99 kubenswrapper[2552]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 13 09:08:36.433279 ip-10-0-129-99 kubenswrapper[2552]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 09:08:36.433279 ip-10-0-129-99 kubenswrapper[2552]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 13 09:08:36.433279 ip-10-0-129-99 kubenswrapper[2552]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 09:08:36.435439 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.435338 2552 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 13 09:08:36.441729 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441715 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 09:08:36.441729 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441729 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 09:08:36.441785 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441734 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 09:08:36.441785 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441737 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 09:08:36.441785 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441740 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 09:08:36.441785 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441742 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 09:08:36.441785 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441745 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 09:08:36.441785 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441748 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 09:08:36.441785 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441755 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 09:08:36.441785 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441758 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 09:08:36.441785 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441761 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 09:08:36.441785 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441763 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 09:08:36.441785 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441766 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 09:08:36.441785 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441769 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 09:08:36.441785 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441771 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 09:08:36.441785 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441774 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 09:08:36.441785 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441776 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 09:08:36.441785 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441781 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 09:08:36.441785 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441784 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 09:08:36.441785 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441787 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 09:08:36.441785 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441791 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 09:08:36.442262 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441794 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 09:08:36.442262 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441797 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 09:08:36.442262 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441801 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 09:08:36.442262 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441805 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 09:08:36.442262 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441807 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 09:08:36.442262 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441810 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 09:08:36.442262 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441813 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 09:08:36.442262 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441815 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 09:08:36.442262 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441818 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 09:08:36.442262 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441820 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 09:08:36.442262 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441823 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 09:08:36.442262 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441825 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 09:08:36.442262 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441828 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 09:08:36.442262 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441836 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 09:08:36.442262 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441839 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 09:08:36.442262 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441841 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 09:08:36.442262 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441844 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 09:08:36.442262 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441847 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 09:08:36.442262 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441851 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 09:08:36.442703 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441854 2552 feature_gate.go:328] unrecognized feature gate: Example Jul 13 09:08:36.442703 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441857 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 09:08:36.442703 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441860 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 09:08:36.442703 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441862 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 09:08:36.442703 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441865 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 09:08:36.442703 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441868 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 09:08:36.442703 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441871 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 09:08:36.442703 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441873 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 09:08:36.442703 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441876 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 09:08:36.442703 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441879 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 09:08:36.442703 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441881 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 09:08:36.442703 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441883 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 09:08:36.442703 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441886 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 09:08:36.442703 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441889 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 09:08:36.442703 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441891 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 09:08:36.442703 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441894 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 09:08:36.442703 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441896 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 09:08:36.442703 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441899 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 09:08:36.442703 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441902 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 09:08:36.443166 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441904 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 09:08:36.443166 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441908 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 09:08:36.443166 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441910 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 09:08:36.443166 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441913 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 09:08:36.443166 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441915 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 09:08:36.443166 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441918 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 09:08:36.443166 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441920 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 09:08:36.443166 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441922 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 09:08:36.443166 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441925 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 09:08:36.443166 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441927 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 09:08:36.443166 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441930 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 09:08:36.443166 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441933 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 09:08:36.443166 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441935 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 09:08:36.443166 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441938 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 09:08:36.443166 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441940 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 09:08:36.443166 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441942 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 09:08:36.443166 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441945 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 09:08:36.443166 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441947 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 09:08:36.443166 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441950 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 09:08:36.443166 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441954 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 09:08:36.443653 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441957 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 09:08:36.443653 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441959 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 09:08:36.443653 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441962 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 09:08:36.443653 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441964 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 09:08:36.443653 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441967 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 09:08:36.443653 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441969 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 09:08:36.443653 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441972 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 09:08:36.443653 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441974 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 09:08:36.443653 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441976 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 09:08:36.443653 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441979 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 09:08:36.443653 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441981 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 09:08:36.443653 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441984 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 09:08:36.443653 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441986 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 09:08:36.443653 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441989 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 09:08:36.443653 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441991 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 09:08:36.443653 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441994 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 09:08:36.443653 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441996 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 09:08:36.443653 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.441999 2552 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 09:08:36.443653 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442001 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 09:08:36.443653 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442004 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 09:08:36.444127 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442676 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 09:08:36.444127 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442682 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 09:08:36.444127 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442685 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 09:08:36.444127 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442689 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 09:08:36.444127 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442692 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 09:08:36.444127 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442694 2552 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 09:08:36.444127 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442697 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 09:08:36.444127 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442700 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 09:08:36.444127 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442703 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 09:08:36.444127 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442706 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 09:08:36.444127 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442709 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 09:08:36.444127 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442712 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 09:08:36.444127 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442714 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 09:08:36.444127 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442717 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 09:08:36.444127 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442720 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 09:08:36.444127 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442722 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 09:08:36.444127 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442725 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 09:08:36.444127 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442728 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 09:08:36.444127 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442730 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 09:08:36.444127 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442733 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 09:08:36.444633 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442735 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 09:08:36.444633 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442738 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 09:08:36.444633 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442740 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 09:08:36.444633 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442743 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 09:08:36.444633 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442746 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 09:08:36.444633 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442749 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 09:08:36.444633 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442752 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 09:08:36.444633 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442755 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 09:08:36.444633 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442758 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 09:08:36.444633 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442760 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 09:08:36.444633 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442763 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 09:08:36.444633 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442766 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 09:08:36.444633 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442769 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 09:08:36.444633 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442771 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 09:08:36.444633 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442774 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 09:08:36.444633 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442776 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 09:08:36.444633 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442779 2552 feature_gate.go:328] unrecognized feature gate: Example Jul 13 09:08:36.444633 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442781 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 09:08:36.444633 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442784 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 09:08:36.445102 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442787 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 09:08:36.445102 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442790 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 09:08:36.445102 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442793 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 09:08:36.445102 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442796 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 09:08:36.445102 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442799 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 09:08:36.445102 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442801 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 09:08:36.445102 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442804 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 09:08:36.445102 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442806 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 09:08:36.445102 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442809 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 09:08:36.445102 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442811 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 09:08:36.445102 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442814 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 09:08:36.445102 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442816 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 09:08:36.445102 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442818 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 09:08:36.445102 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442821 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 09:08:36.445102 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442824 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 09:08:36.445102 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442826 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 09:08:36.445102 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442828 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 09:08:36.445102 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442831 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 09:08:36.445102 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442835 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 09:08:36.445572 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442838 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 09:08:36.445572 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442841 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 09:08:36.445572 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442844 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 09:08:36.445572 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442846 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 09:08:36.445572 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442849 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 09:08:36.445572 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442851 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 09:08:36.445572 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442854 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 09:08:36.445572 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442856 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 09:08:36.445572 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442859 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 09:08:36.445572 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442861 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 09:08:36.445572 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442864 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 09:08:36.445572 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442866 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 09:08:36.445572 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442869 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 09:08:36.445572 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442872 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 09:08:36.445572 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442875 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 09:08:36.445572 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442878 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 09:08:36.445572 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442880 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 09:08:36.445572 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442883 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 09:08:36.445572 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442888 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 09:08:36.445572 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442891 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 09:08:36.446057 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442894 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 09:08:36.446057 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442897 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 09:08:36.446057 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442899 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 09:08:36.446057 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442902 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 09:08:36.446057 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442904 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 09:08:36.446057 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442909 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 09:08:36.446057 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442912 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 09:08:36.446057 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442914 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 09:08:36.446057 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442917 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 09:08:36.446057 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442919 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 09:08:36.446057 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442922 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 09:08:36.446057 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442924 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 09:08:36.446057 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442927 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 09:08:36.446057 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442929 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 09:08:36.446057 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442932 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 09:08:36.446057 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442934 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 09:08:36.446057 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442937 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 09:08:36.446057 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442939 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 09:08:36.446057 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442942 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 09:08:36.446057 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442944 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 09:08:36.446563 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.442947 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 09:08:36.446563 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443357 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 09:08:36.446563 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443363 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 09:08:36.446563 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443366 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 09:08:36.446563 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443369 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 09:08:36.446563 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443372 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 09:08:36.446563 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443376 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 09:08:36.446563 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443379 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 09:08:36.446563 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443381 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 09:08:36.446563 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443385 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 09:08:36.446563 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443389 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 09:08:36.446563 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443392 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 09:08:36.446563 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443395 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 09:08:36.446563 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443397 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 09:08:36.446563 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443400 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 09:08:36.446563 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443403 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 09:08:36.446563 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443405 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 09:08:36.446563 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443412 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 09:08:36.446563 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443415 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 09:08:36.447098 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443417 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 09:08:36.447098 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443420 2552 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 09:08:36.447098 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443422 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 09:08:36.447098 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443425 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 09:08:36.447098 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443427 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 09:08:36.447098 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443430 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 09:08:36.447098 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443433 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 09:08:36.447098 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443436 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 09:08:36.447098 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443440 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 09:08:36.447098 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443443 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 09:08:36.447098 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443445 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 09:08:36.447098 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443448 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 09:08:36.447098 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443450 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 09:08:36.447098 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443453 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 09:08:36.447098 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443456 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 09:08:36.447098 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443459 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 09:08:36.447098 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443461 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 09:08:36.447098 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443464 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 09:08:36.447098 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443466 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 09:08:36.447098 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443468 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 09:08:36.447596 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443471 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 09:08:36.447596 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443474 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 09:08:36.447596 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443476 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 09:08:36.447596 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443479 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 09:08:36.447596 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443482 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 09:08:36.447596 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443484 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 09:08:36.447596 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443487 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 09:08:36.447596 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443489 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 09:08:36.447596 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443492 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 09:08:36.447596 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443494 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 09:08:36.447596 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443498 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 09:08:36.447596 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443500 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 09:08:36.447596 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443503 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 09:08:36.447596 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443506 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 09:08:36.447596 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443508 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 09:08:36.447596 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443510 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 09:08:36.447596 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443513 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 09:08:36.447596 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443515 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 09:08:36.447596 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443518 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 09:08:36.447596 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443521 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 09:08:36.448111 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443523 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 09:08:36.448111 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443525 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 09:08:36.448111 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443528 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 09:08:36.448111 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443531 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 09:08:36.448111 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443533 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 09:08:36.448111 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443536 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 09:08:36.448111 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443538 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 09:08:36.448111 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443541 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 09:08:36.448111 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443543 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 09:08:36.448111 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443546 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 09:08:36.448111 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443548 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 09:08:36.448111 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443551 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 09:08:36.448111 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443553 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 09:08:36.448111 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443556 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 09:08:36.448111 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443558 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 09:08:36.448111 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443561 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 09:08:36.448111 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443563 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 09:08:36.448111 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443566 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 09:08:36.448111 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443568 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 09:08:36.448111 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443571 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 09:08:36.448640 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443573 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 09:08:36.448640 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443576 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 09:08:36.448640 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443579 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 09:08:36.448640 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443583 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 09:08:36.448640 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443585 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 09:08:36.448640 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443587 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 09:08:36.448640 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443590 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 09:08:36.448640 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443593 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 09:08:36.448640 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443597 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 09:08:36.448640 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443600 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 09:08:36.448640 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443603 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 09:08:36.448640 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443606 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 09:08:36.448640 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443609 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 09:08:36.448640 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443612 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 09:08:36.448640 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443614 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 09:08:36.448640 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443617 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 09:08:36.448640 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443619 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 09:08:36.448640 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443622 2552 feature_gate.go:328] unrecognized feature gate: Example Jul 13 09:08:36.448640 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443624 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 09:08:36.449101 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443627 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 09:08:36.449101 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.443629 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 09:08:36.449101 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443692 2552 flags.go:64] FLAG: --address="0.0.0.0" Jul 13 09:08:36.449101 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443699 2552 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 13 09:08:36.449101 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443705 2552 flags.go:64] FLAG: --anonymous-auth="true" Jul 13 09:08:36.449101 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443709 2552 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 13 09:08:36.449101 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443713 2552 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 13 09:08:36.449101 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443716 2552 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 13 09:08:36.449101 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443720 2552 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 13 09:08:36.449101 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443724 2552 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 13 09:08:36.449101 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443728 2552 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 13 09:08:36.449101 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443730 2552 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 13 09:08:36.449101 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443734 2552 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 13 09:08:36.449101 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443737 2552 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 13 09:08:36.449101 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443740 2552 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 13 09:08:36.449101 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443744 2552 flags.go:64] FLAG: --cgroup-root="" Jul 13 09:08:36.449101 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443747 2552 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 13 09:08:36.449101 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443750 2552 flags.go:64] FLAG: --client-ca-file="" Jul 13 09:08:36.449101 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443753 2552 flags.go:64] FLAG: --cloud-provider="external" Jul 13 09:08:36.449101 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443756 2552 flags.go:64] FLAG: --cluster-dns="[]" Jul 13 09:08:36.449101 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443760 2552 flags.go:64] FLAG: --cluster-domain="" Jul 13 09:08:36.449101 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443762 2552 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 13 09:08:36.449101 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443767 2552 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 13 09:08:36.449101 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443771 2552 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 13 09:08:36.449714 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443774 2552 flags.go:64] FLAG: --container-log-max-files="5" Jul 13 09:08:36.449714 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443777 2552 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 13 09:08:36.449714 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443781 2552 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 13 09:08:36.449714 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443784 2552 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 13 09:08:36.449714 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443787 2552 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 13 09:08:36.449714 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443789 2552 flags.go:64] FLAG: --contention-profiling="false" Jul 13 09:08:36.449714 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443792 2552 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 13 09:08:36.449714 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443796 2552 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 13 09:08:36.449714 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443799 2552 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 13 09:08:36.449714 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443801 2552 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 13 09:08:36.449714 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443805 2552 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 13 09:08:36.449714 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443808 2552 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 13 09:08:36.449714 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443811 2552 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 13 09:08:36.449714 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443814 2552 flags.go:64] FLAG: --enable-load-reader="false" Jul 13 09:08:36.449714 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443817 2552 flags.go:64] FLAG: --enable-server="true" Jul 13 09:08:36.449714 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443820 2552 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 13 09:08:36.449714 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443824 2552 flags.go:64] FLAG: --event-burst="100" Jul 13 09:08:36.449714 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443827 2552 flags.go:64] FLAG: --event-qps="50" Jul 13 09:08:36.449714 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443830 2552 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 13 09:08:36.449714 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443833 2552 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 13 09:08:36.449714 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443836 2552 flags.go:64] FLAG: --eviction-hard="" Jul 13 09:08:36.449714 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443839 2552 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 13 09:08:36.449714 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443842 2552 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 13 09:08:36.449714 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443845 2552 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 13 09:08:36.449714 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443848 2552 flags.go:64] FLAG: --eviction-soft="" Jul 13 09:08:36.450433 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443851 2552 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 13 09:08:36.450433 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443853 2552 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 13 09:08:36.450433 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443856 2552 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 13 09:08:36.450433 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443859 2552 flags.go:64] FLAG: --experimental-mounter-path="" Jul 13 09:08:36.450433 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443862 2552 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 13 09:08:36.450433 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443866 2552 flags.go:64] FLAG: --fail-swap-on="true" Jul 13 09:08:36.450433 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443869 2552 flags.go:64] FLAG: --feature-gates="" Jul 13 09:08:36.450433 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443872 2552 flags.go:64] FLAG: --file-check-frequency="20s" Jul 13 09:08:36.450433 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443875 2552 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 13 09:08:36.450433 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443878 2552 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 13 09:08:36.450433 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443881 2552 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 13 09:08:36.450433 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443884 2552 flags.go:64] FLAG: --healthz-port="10248" Jul 13 09:08:36.450433 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443887 2552 flags.go:64] FLAG: --help="false" Jul 13 09:08:36.450433 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443890 2552 flags.go:64] FLAG: --hostname-override="ip-10-0-129-99.ec2.internal" Jul 13 09:08:36.450433 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443893 2552 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 13 09:08:36.450433 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443896 2552 flags.go:64] FLAG: --http-check-frequency="20s" Jul 13 09:08:36.450433 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443899 2552 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 13 09:08:36.450433 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443902 2552 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 13 09:08:36.450433 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443906 2552 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 13 09:08:36.450433 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443909 2552 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 13 09:08:36.450433 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443912 2552 flags.go:64] FLAG: --image-service-endpoint="" Jul 13 09:08:36.450433 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443915 2552 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 13 09:08:36.450433 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443918 2552 flags.go:64] FLAG: --kube-api-burst="100" Jul 13 09:08:36.451122 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443921 2552 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 13 09:08:36.451122 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443924 2552 flags.go:64] FLAG: --kube-api-qps="50" Jul 13 09:08:36.451122 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443927 2552 flags.go:64] FLAG: --kube-reserved="" Jul 13 09:08:36.451122 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443930 2552 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 13 09:08:36.451122 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443932 2552 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 13 09:08:36.451122 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443935 2552 flags.go:64] FLAG: --kubelet-cgroups="" Jul 13 09:08:36.451122 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443938 2552 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 13 09:08:36.451122 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443941 2552 flags.go:64] FLAG: --lock-file="" Jul 13 09:08:36.451122 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443944 2552 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 13 09:08:36.451122 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443946 2552 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 13 09:08:36.451122 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443949 2552 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 13 09:08:36.451122 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443955 2552 flags.go:64] FLAG: --log-json-split-stream="false" Jul 13 09:08:36.451122 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443958 2552 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 13 09:08:36.451122 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443960 2552 flags.go:64] FLAG: --log-text-split-stream="false" Jul 13 09:08:36.451122 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443965 2552 flags.go:64] FLAG: --logging-format="text" Jul 13 09:08:36.451122 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443968 2552 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 13 09:08:36.451122 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443971 2552 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 13 09:08:36.451122 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443974 2552 flags.go:64] FLAG: --manifest-url="" Jul 13 09:08:36.451122 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443977 2552 flags.go:64] FLAG: --manifest-url-header="" Jul 13 09:08:36.451122 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443987 2552 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 13 09:08:36.451122 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443991 2552 flags.go:64] FLAG: --max-open-files="1000000" Jul 13 09:08:36.451122 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443994 2552 flags.go:64] FLAG: --max-pods="110" Jul 13 09:08:36.451122 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.443997 2552 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 13 09:08:36.451122 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444001 2552 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 13 09:08:36.451122 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444003 2552 flags.go:64] FLAG: --memory-manager-policy="None" Jul 13 09:08:36.452047 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444006 2552 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 13 09:08:36.452047 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444009 2552 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 13 09:08:36.452047 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444012 2552 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 13 09:08:36.452047 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444014 2552 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 13 09:08:36.452047 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444022 2552 flags.go:64] FLAG: --node-status-max-images="50" Jul 13 09:08:36.452047 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444025 2552 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 13 09:08:36.452047 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444027 2552 flags.go:64] FLAG: --oom-score-adj="-999" Jul 13 09:08:36.452047 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444039 2552 flags.go:64] FLAG: --pod-cidr="" Jul 13 09:08:36.452047 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444041 2552 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 13 09:08:36.452047 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444047 2552 flags.go:64] FLAG: --pod-manifest-path="" Jul 13 09:08:36.452047 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444050 2552 flags.go:64] FLAG: --pod-max-pids="-1" Jul 13 09:08:36.452047 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444053 2552 flags.go:64] FLAG: --pods-per-core="0" Jul 13 09:08:36.452047 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444056 2552 flags.go:64] FLAG: --port="10250" Jul 13 09:08:36.452047 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444059 2552 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 13 09:08:36.452047 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444062 2552 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ee17ee5d5172b897" Jul 13 09:08:36.452047 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444065 2552 flags.go:64] FLAG: --qos-reserved="" Jul 13 09:08:36.452047 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444068 2552 flags.go:64] FLAG: --read-only-port="10255" Jul 13 09:08:36.452047 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444071 2552 flags.go:64] FLAG: --register-node="true" Jul 13 09:08:36.452047 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444074 2552 flags.go:64] FLAG: --register-with-taints="" Jul 13 09:08:36.452047 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444077 2552 flags.go:64] FLAG: --registry-burst="10" Jul 13 09:08:36.452047 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444080 2552 flags.go:64] FLAG: --registry-qps="5" Jul 13 09:08:36.452047 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444083 2552 flags.go:64] FLAG: --reserved-cpus="" Jul 13 09:08:36.452047 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444089 2552 flags.go:64] FLAG: --reserved-memory="" Jul 13 09:08:36.452047 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444093 2552 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 13 09:08:36.452047 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444096 2552 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 13 09:08:36.452815 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444098 2552 flags.go:64] FLAG: --rotate-certificates="false" Jul 13 09:08:36.452815 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444101 2552 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 13 09:08:36.452815 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444104 2552 flags.go:64] FLAG: --runonce="false" Jul 13 09:08:36.452815 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444107 2552 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 13 09:08:36.452815 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444110 2552 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 13 09:08:36.452815 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444113 2552 flags.go:64] FLAG: --seccomp-default="false" Jul 13 09:08:36.452815 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444115 2552 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 13 09:08:36.452815 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444118 2552 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 13 09:08:36.452815 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444121 2552 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 13 09:08:36.452815 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444124 2552 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 13 09:08:36.452815 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444127 2552 flags.go:64] FLAG: --storage-driver-password="root" Jul 13 09:08:36.452815 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444130 2552 flags.go:64] FLAG: --storage-driver-secure="false" Jul 13 09:08:36.452815 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444133 2552 flags.go:64] FLAG: --storage-driver-table="stats" Jul 13 09:08:36.452815 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444136 2552 flags.go:64] FLAG: --storage-driver-user="root" Jul 13 09:08:36.452815 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444139 2552 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 13 09:08:36.452815 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444142 2552 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 13 09:08:36.452815 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444144 2552 flags.go:64] FLAG: --system-cgroups="" Jul 13 09:08:36.452815 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444148 2552 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 13 09:08:36.452815 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444153 2552 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 13 09:08:36.452815 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444156 2552 flags.go:64] FLAG: --tls-cert-file="" Jul 13 09:08:36.452815 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444158 2552 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 13 09:08:36.452815 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444163 2552 flags.go:64] FLAG: --tls-min-version="" Jul 13 09:08:36.452815 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444165 2552 flags.go:64] FLAG: --tls-private-key-file="" Jul 13 09:08:36.452815 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444168 2552 flags.go:64] FLAG: --topology-manager-policy="none" Jul 13 09:08:36.452815 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444171 2552 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 13 09:08:36.453464 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444174 2552 flags.go:64] FLAG: --topology-manager-scope="container" Jul 13 09:08:36.453464 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444176 2552 flags.go:64] FLAG: --v="2" Jul 13 09:08:36.453464 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444180 2552 flags.go:64] FLAG: --version="false" Jul 13 09:08:36.453464 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444183 2552 flags.go:64] FLAG: --vmodule="" Jul 13 09:08:36.453464 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444190 2552 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 13 09:08:36.453464 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444194 2552 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 13 09:08:36.453464 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444317 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 09:08:36.453464 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444322 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 09:08:36.453464 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444325 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 09:08:36.453464 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444328 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 09:08:36.453464 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444331 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 09:08:36.453464 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444333 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 09:08:36.453464 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444336 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 09:08:36.453464 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444338 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 09:08:36.453464 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444341 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 09:08:36.453464 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444343 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 09:08:36.453464 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444346 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 09:08:36.453464 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444348 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 09:08:36.453464 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444351 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 09:08:36.453464 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444353 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 09:08:36.453464 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444356 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 09:08:36.453995 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444358 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 09:08:36.453995 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444361 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 09:08:36.453995 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444363 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 09:08:36.453995 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444366 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 09:08:36.453995 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444369 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 09:08:36.453995 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444371 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 09:08:36.453995 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444374 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 09:08:36.453995 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444376 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 09:08:36.453995 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444379 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 09:08:36.453995 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444381 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 09:08:36.453995 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444383 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 09:08:36.453995 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444386 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 09:08:36.453995 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444388 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 09:08:36.453995 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444391 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 09:08:36.453995 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444394 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 09:08:36.453995 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444398 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 09:08:36.453995 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444401 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 09:08:36.453995 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444404 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 09:08:36.453995 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444406 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 09:08:36.453995 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444409 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 09:08:36.454534 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444411 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 09:08:36.454534 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444414 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 09:08:36.454534 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444416 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 09:08:36.454534 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444419 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 09:08:36.454534 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444422 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 09:08:36.454534 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444424 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 09:08:36.454534 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444426 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 09:08:36.454534 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444429 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 09:08:36.454534 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444432 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 09:08:36.454534 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444434 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 09:08:36.454534 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444437 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 09:08:36.454534 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444440 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 09:08:36.454534 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444442 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 09:08:36.454534 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444445 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 09:08:36.454534 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444447 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 09:08:36.454534 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444450 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 09:08:36.454534 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444452 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 09:08:36.454534 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444458 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 09:08:36.454534 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444461 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 09:08:36.454534 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444463 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 09:08:36.455014 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444466 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 09:08:36.455014 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444468 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 09:08:36.455014 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444471 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 09:08:36.455014 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444474 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 09:08:36.455014 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444476 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 09:08:36.455014 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444478 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 09:08:36.455014 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444481 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 09:08:36.455014 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444487 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 09:08:36.455014 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444489 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 09:08:36.455014 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444492 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 09:08:36.455014 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444494 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 09:08:36.455014 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444497 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 09:08:36.455014 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444500 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 09:08:36.455014 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444504 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 09:08:36.455014 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444507 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 09:08:36.455014 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444510 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 09:08:36.455014 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444512 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 09:08:36.455014 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444515 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 09:08:36.455014 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444517 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 09:08:36.455509 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444520 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 09:08:36.455509 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444522 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 09:08:36.455509 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444525 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 09:08:36.455509 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444527 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 09:08:36.455509 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444530 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 09:08:36.455509 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444532 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 09:08:36.455509 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444535 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 09:08:36.455509 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444537 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 09:08:36.455509 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444540 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 09:08:36.455509 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444544 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 09:08:36.455509 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444547 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 09:08:36.455509 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444550 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 09:08:36.455509 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444554 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 09:08:36.455509 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444556 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 09:08:36.455509 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444559 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 09:08:36.455509 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444562 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 09:08:36.455509 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444564 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 09:08:36.455509 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444567 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 09:08:36.455509 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444571 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 09:08:36.456031 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444574 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 09:08:36.456031 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444579 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 09:08:36.456031 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444582 2552 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 09:08:36.456031 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444585 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 09:08:36.456031 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444588 2552 feature_gate.go:328] unrecognized feature gate: Example Jul 13 09:08:36.456031 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.444590 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 09:08:36.456031 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.444596 2552 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 09:08:36.456031 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.453807 2552 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 13 09:08:36.456031 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.453821 2552 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 13 09:08:36.456031 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.453840 2552 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 13 09:08:36.456031 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.453846 2552 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 13 09:08:36.456031 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.453901 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 09:08:36.456031 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.453906 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 09:08:36.456031 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.453909 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 09:08:36.456031 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.453912 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 09:08:36.456520 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.453915 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 09:08:36.456520 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.453918 2552 feature_gate.go:328] unrecognized feature gate: Example Jul 13 09:08:36.456520 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.453920 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 09:08:36.456520 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.453923 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 09:08:36.456520 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.453925 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 09:08:36.456520 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.453928 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 09:08:36.456520 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.453931 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 09:08:36.456520 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.453933 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 09:08:36.456520 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.453937 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 09:08:36.456520 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.453942 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 09:08:36.456520 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.453945 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 09:08:36.456520 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.453948 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 09:08:36.456520 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.453951 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 09:08:36.456520 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.453953 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 09:08:36.456520 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.453956 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 09:08:36.456520 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.453959 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 09:08:36.456520 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.453961 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 09:08:36.456520 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.453964 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 09:08:36.456520 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.453968 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 09:08:36.456970 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.453971 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 09:08:36.456970 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.453974 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 09:08:36.456970 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.453977 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 09:08:36.456970 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.453979 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 09:08:36.456970 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.453982 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 09:08:36.456970 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.453985 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 09:08:36.456970 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.453987 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 09:08:36.456970 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.453990 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 09:08:36.456970 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.453992 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 09:08:36.456970 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.453996 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 09:08:36.456970 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454000 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 09:08:36.456970 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454003 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 09:08:36.456970 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454006 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 09:08:36.456970 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454008 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 09:08:36.456970 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454011 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 09:08:36.456970 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454013 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 09:08:36.456970 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454016 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 09:08:36.456970 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454018 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 09:08:36.456970 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454021 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 09:08:36.457452 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454023 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 09:08:36.457452 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454026 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 09:08:36.457452 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454028 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 09:08:36.457452 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454031 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 09:08:36.457452 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454033 2552 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 09:08:36.457452 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454036 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 09:08:36.457452 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454039 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 09:08:36.457452 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454041 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 09:08:36.457452 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454044 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 09:08:36.457452 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454046 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 09:08:36.457452 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454049 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 09:08:36.457452 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454052 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 09:08:36.457452 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454054 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 09:08:36.457452 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454059 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 09:08:36.457452 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454062 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 09:08:36.457452 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454065 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 09:08:36.457452 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454068 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 09:08:36.457452 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454070 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 09:08:36.457452 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454073 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 09:08:36.457452 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454075 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 09:08:36.457947 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454077 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 09:08:36.457947 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454080 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 09:08:36.457947 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454083 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 09:08:36.457947 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454085 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 09:08:36.457947 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454087 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 09:08:36.457947 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454090 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 09:08:36.457947 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454093 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 09:08:36.457947 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454095 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 09:08:36.457947 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454097 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 09:08:36.457947 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454100 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 09:08:36.457947 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454102 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 09:08:36.457947 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454105 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 09:08:36.457947 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454107 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 09:08:36.457947 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454110 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 09:08:36.457947 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454112 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 09:08:36.457947 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454115 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 09:08:36.457947 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454118 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 09:08:36.457947 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454121 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 09:08:36.457947 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454123 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 09:08:36.457947 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454126 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 09:08:36.458472 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454128 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 09:08:36.458472 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454131 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 09:08:36.458472 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454134 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 09:08:36.458472 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454136 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 09:08:36.458472 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454139 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 09:08:36.458472 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454141 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 09:08:36.458472 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454144 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 09:08:36.458472 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454147 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 09:08:36.458472 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454149 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 09:08:36.458472 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454152 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 09:08:36.458472 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454154 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 09:08:36.458472 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454156 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 09:08:36.458472 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454159 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 09:08:36.458472 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454161 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 09:08:36.458472 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454164 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 09:08:36.458472 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454166 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 09:08:36.458472 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454169 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 09:08:36.458885 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.454174 2552 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 09:08:36.458885 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454319 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 09:08:36.458885 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454325 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 09:08:36.458885 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454329 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 09:08:36.458885 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454332 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 09:08:36.458885 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454335 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 09:08:36.458885 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454338 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 09:08:36.458885 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454341 2552 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 09:08:36.458885 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454344 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 09:08:36.458885 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454347 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 09:08:36.458885 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454349 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 09:08:36.458885 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454352 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 09:08:36.458885 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454355 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 09:08:36.458885 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454358 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 09:08:36.458885 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454361 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 09:08:36.459282 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454364 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 09:08:36.459282 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454368 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 09:08:36.459282 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454372 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 09:08:36.459282 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454375 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 09:08:36.459282 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454378 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 09:08:36.459282 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454381 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 09:08:36.459282 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454383 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 09:08:36.459282 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454386 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 09:08:36.459282 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454388 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 09:08:36.459282 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454391 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 09:08:36.459282 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454393 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 09:08:36.459282 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454396 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 09:08:36.459282 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454399 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 09:08:36.459282 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454401 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 09:08:36.459282 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454405 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 09:08:36.459282 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454408 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 09:08:36.459282 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454411 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 09:08:36.459282 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454415 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 09:08:36.459282 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454417 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 09:08:36.459749 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454420 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 09:08:36.459749 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454422 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 09:08:36.459749 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454425 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 09:08:36.459749 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454428 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 09:08:36.459749 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454430 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 09:08:36.459749 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454433 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 09:08:36.459749 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454435 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 09:08:36.459749 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454438 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 09:08:36.459749 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454440 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 09:08:36.459749 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454443 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 09:08:36.459749 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454445 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 09:08:36.459749 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454448 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 09:08:36.459749 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454451 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 09:08:36.459749 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454454 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 09:08:36.459749 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454456 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 09:08:36.459749 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454459 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 09:08:36.459749 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454462 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 09:08:36.459749 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454464 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 09:08:36.459749 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454467 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 09:08:36.460314 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454470 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 09:08:36.460314 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454472 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 09:08:36.460314 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454474 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 09:08:36.460314 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454477 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 09:08:36.460314 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454480 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 09:08:36.460314 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454482 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 09:08:36.460314 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454484 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 09:08:36.460314 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454487 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 09:08:36.460314 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454489 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 09:08:36.460314 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454492 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 09:08:36.460314 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454495 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 09:08:36.460314 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454497 2552 feature_gate.go:328] unrecognized feature gate: Example Jul 13 09:08:36.460314 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454500 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 09:08:36.460314 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454502 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 09:08:36.460314 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454505 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 09:08:36.460314 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454507 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 09:08:36.460314 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454510 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 09:08:36.460314 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454512 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 09:08:36.460314 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454515 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 09:08:36.460314 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454517 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 09:08:36.460781 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454519 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 09:08:36.460781 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454522 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 09:08:36.460781 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454524 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 09:08:36.460781 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454526 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 09:08:36.460781 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454529 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 09:08:36.460781 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454531 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 09:08:36.460781 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454535 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 09:08:36.460781 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454537 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 09:08:36.460781 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454540 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 09:08:36.460781 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454543 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 09:08:36.460781 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454548 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 09:08:36.460781 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454550 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 09:08:36.460781 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454553 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 09:08:36.460781 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454555 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 09:08:36.460781 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454558 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 09:08:36.460781 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454560 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 09:08:36.460781 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454563 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 09:08:36.460781 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454565 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 09:08:36.460781 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454568 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 09:08:36.461252 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454571 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 09:08:36.461252 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454575 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 09:08:36.461252 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454577 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 09:08:36.461252 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454580 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 09:08:36.461252 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454583 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 09:08:36.461252 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454585 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 09:08:36.461252 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454588 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 09:08:36.461252 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:36.454590 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 09:08:36.461252 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.454595 2552 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 09:08:36.461252 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.454694 2552 server.go:962] "Client rotation is on, will bootstrap in background" Jul 13 09:08:36.461252 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.458020 2552 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 13 09:08:36.461252 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.459733 2552 server.go:1019] "Starting client certificate rotation" Jul 13 09:08:36.461252 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.459837 2552 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 13 09:08:36.461252 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.459873 2552 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 13 09:08:36.462525 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.462511 2552 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 13 09:08:36.463662 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.463643 2552 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 13 09:08:36.475137 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.475119 2552 log.go:25] "Validated CRI v1 runtime API" Jul 13 09:08:36.477470 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.477455 2552 log.go:25] "Validated CRI v1 image API" Jul 13 09:08:36.478381 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.478363 2552 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 13 09:08:36.483526 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.483508 2552 fs.go:135] Filesystem UUIDs: map[62597637-76a9-4fbf-a8e6-014bd563cee2:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 bb35315f-f62e-41fd-ad21-00ef5c6bbd5c:/dev/nvme0n1p3] Jul 13 09:08:36.483592 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.483525 2552 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 13 09:08:36.488920 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.488897 2552 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 13 09:08:36.490556 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.490447 2552 manager.go:216] Machine: {Timestamp:2026-07-13 09:08:36.489135487 +0000 UTC m=+0.369090744 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099344 MemoryCapacity:33164505088 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec208a683b6071d7a84f1ecea8123764 SystemUUID:ec208a68-3b60-71d7-a84f-1ecea8123764 BootID:6dd64f0f-bc5c-48de-8d5a-7653de6b1a02 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582254592 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:2e:71:87:d7:45 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:2e:71:87:d7:45 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f6:fc:2d:4d:86:57 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164505088 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 13 09:08:36.490556 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.490549 2552 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 13 09:08:36.490699 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.490624 2552 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 13 09:08:36.492583 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.492560 2552 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 13 09:08:36.492735 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.492586 2552 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-99.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 13 09:08:36.494280 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.494268 2552 topology_manager.go:138] "Creating topology manager with none policy" Jul 13 09:08:36.494329 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.494283 2552 container_manager_linux.go:309] "Creating device plugin manager" Jul 13 09:08:36.494329 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.494290 2552 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 13 09:08:36.495159 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.495148 2552 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 13 09:08:36.497127 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.497117 2552 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 13 09:08:36.501637 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.501624 2552 state_mem.go:36] "Initialized new in-memory state store" Jul 13 09:08:36.501749 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.501738 2552 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 13 09:08:36.501802 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.501767 2552 kubelet.go:486] "Attempting to sync node with API server" Jul 13 09:08:36.501802 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.501790 2552 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 13 09:08:36.501916 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.501821 2552 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 13 09:08:36.501916 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.501832 2552 kubelet.go:398] "Adding apiserver pod source" Jul 13 09:08:36.501916 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.501842 2552 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 13 09:08:36.502170 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.502158 2552 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 13 09:08:36.502226 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.502177 2552 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 13 09:08:36.504570 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.504546 2552 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 13 09:08:36.507014 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.506998 2552 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 13 09:08:36.507085 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.507037 2552 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 13 09:08:36.507242 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.507231 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 13 09:08:36.507278 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.507250 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 13 09:08:36.507278 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.507260 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 13 09:08:36.507278 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.507267 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 13 09:08:36.507278 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.507278 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 13 09:08:36.507419 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.507287 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 13 09:08:36.507935 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.507925 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 13 09:08:36.507964 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.507937 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 13 09:08:36.507964 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.507946 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 13 09:08:36.507964 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.507954 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 13 09:08:36.508035 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.507965 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 13 09:08:36.508035 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.507975 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 13 09:08:36.508783 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.508769 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 13 09:08:36.508827 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.508786 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 13 09:08:36.510296 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:36.510275 2552 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 13 09:08:36.510476 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.510450 2552 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-99.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 13 09:08:36.510476 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:36.510461 2552 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-99.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 13 09:08:36.511948 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.511931 2552 server.go:1268] "Started kubelet" Jul 13 09:08:36.512048 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.512001 2552 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 13 09:08:36.512172 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.512012 2552 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 13 09:08:36.512300 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.512285 2552 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 13 09:08:36.512736 ip-10-0-129-99 systemd[1]: Started Kubernetes Kubelet. Jul 13 09:08:36.513851 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.513775 2552 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 13 09:08:36.514073 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.514049 2552 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cbjsv" Jul 13 09:08:36.514859 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.514842 2552 server.go:310] "Adding debug handlers to kubelet server" Jul 13 09:08:36.516920 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:36.516886 2552 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 13 09:08:36.521598 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.521582 2552 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cbjsv" Jul 13 09:08:36.522754 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.522741 2552 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 13 09:08:36.523423 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.523408 2552 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 13 09:08:36.523648 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:36.523632 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-99.ec2.internal\" not found" Jul 13 09:08:36.523826 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:36.522632 2552 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-99.ec2.internal.18c1ce437388cdbc default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-99.ec2.internal,UID:ip-10-0-129-99.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-99.ec2.internal,},FirstTimestamp:2026-07-13 09:08:36.511903164 +0000 UTC m=+0.391858423,LastTimestamp:2026-07-13 09:08:36.511903164 +0000 UTC m=+0.391858423,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-99.ec2.internal,}" Jul 13 09:08:36.524334 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.524295 2552 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 13 09:08:36.524334 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.524337 2552 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 13 09:08:36.524468 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.524456 2552 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 13 09:08:36.524688 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.524488 2552 reconstruct.go:97] "Volume reconstruction finished" Jul 13 09:08:36.524910 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.524900 2552 reconciler.go:29] "Reconciler: start to sync state" Jul 13 09:08:36.525602 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.525584 2552 factory.go:55] Registering systemd factory Jul 13 09:08:36.525693 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.525642 2552 factory.go:223] Registration of the systemd container factory successfully Jul 13 09:08:36.527462 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.527441 2552 factory.go:153] Registering CRI-O factory Jul 13 09:08:36.527536 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.527464 2552 factory.go:223] Registration of the crio container factory successfully Jul 13 09:08:36.527536 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.527512 2552 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 13 09:08:36.527536 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.527535 2552 factory.go:103] Registering Raw factory Jul 13 09:08:36.527670 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.527560 2552 manager.go:1196] Started watching for new ooms in manager Jul 13 09:08:36.528225 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.528193 2552 manager.go:349] Starting recovery of all containers Jul 13 09:08:36.536148 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.536128 2552 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 09:08:36.539078 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:36.539058 2552 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-129-99.ec2.internal\" not found" node="ip-10-0-129-99.ec2.internal" Jul 13 09:08:36.540029 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.540014 2552 manager.go:354] Recovery completed Jul 13 09:08:36.543793 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.543698 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 09:08:36.545490 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.545474 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasSufficientMemory" Jul 13 09:08:36.545566 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.545505 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 09:08:36.545566 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.545519 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasSufficientPID" Jul 13 09:08:36.545996 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.545981 2552 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 13 09:08:36.545996 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.545992 2552 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 13 09:08:36.546094 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.546006 2552 state_mem.go:36] "Initialized new in-memory state store" Jul 13 09:08:36.546094 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.546048 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 09:08:36.546720 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.546703 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasSufficientMemory" Jul 13 09:08:36.546788 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.546729 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 09:08:36.546788 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.546739 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasSufficientPID" Jul 13 09:08:36.547990 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.547976 2552 policy_none.go:49] "None policy: Start" Jul 13 09:08:36.547990 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.547993 2552 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 13 09:08:36.548079 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.548003 2552 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 13 09:08:36.549962 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.549952 2552 policy_none.go:47] "Start" Jul 13 09:08:36.577888 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.577873 2552 manager.go:337] "Starting Device Plugin manager" Jul 13 09:08:36.577982 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:36.577908 2552 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 13 09:08:36.577982 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.577917 2552 server.go:87] "Starting device plugin registration server" Jul 13 09:08:36.578165 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.578152 2552 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 13 09:08:36.578233 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.578170 2552 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 13 09:08:36.578365 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.578347 2552 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 13 09:08:36.578672 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.578420 2552 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 13 09:08:36.578672 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.578432 2552 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 13 09:08:36.578904 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:36.578789 2552 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 13 09:08:36.578904 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:36.578826 2552 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-99.ec2.internal\" not found" Jul 13 09:08:36.658365 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.658311 2552 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 13 09:08:36.660489 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.659377 2552 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 13 09:08:36.660489 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.659418 2552 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 13 09:08:36.660489 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.659441 2552 kubelet.go:2448] "Starting kubelet main sync loop" Jul 13 09:08:36.660489 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:36.659472 2552 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 13 09:08:36.661873 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.661852 2552 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 09:08:36.678565 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.678551 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 09:08:36.679306 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.679285 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasSufficientMemory" Jul 13 09:08:36.679370 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.679311 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 09:08:36.679370 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.679320 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasSufficientPID" Jul 13 09:08:36.679370 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.679341 2552 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-99.ec2.internal" Jul 13 09:08:36.687613 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.687599 2552 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-99.ec2.internal" Jul 13 09:08:36.687683 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:36.687619 2552 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-99.ec2.internal\": node \"ip-10-0-129-99.ec2.internal\" not found" Jul 13 09:08:36.703396 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:36.703381 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-99.ec2.internal\" not found" Jul 13 09:08:36.759588 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.759560 2552 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-99.ec2.internal"] Jul 13 09:08:36.759642 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.759632 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 09:08:36.761831 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.761818 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasSufficientMemory" Jul 13 09:08:36.761896 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.761844 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 09:08:36.761896 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.761864 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasSufficientPID" Jul 13 09:08:36.764200 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.764188 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 09:08:36.764339 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.764315 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal" Jul 13 09:08:36.764375 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.764353 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 09:08:36.764908 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.764882 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasSufficientMemory" Jul 13 09:08:36.764908 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.764898 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasSufficientMemory" Jul 13 09:08:36.764908 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.764907 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 09:08:36.765067 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.764918 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 09:08:36.765067 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.764922 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasSufficientPID" Jul 13 09:08:36.765067 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.764933 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasSufficientPID" Jul 13 09:08:36.767180 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.767166 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-99.ec2.internal" Jul 13 09:08:36.767245 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.767193 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 09:08:36.767867 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.767853 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasSufficientMemory" Jul 13 09:08:36.767947 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.767877 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 09:08:36.767947 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.767888 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasSufficientPID" Jul 13 09:08:36.784700 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:36.784684 2552 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-99.ec2.internal\" not found" node="ip-10-0-129-99.ec2.internal" Jul 13 09:08:36.788163 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:36.788148 2552 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-99.ec2.internal\" not found" node="ip-10-0-129-99.ec2.internal" Jul 13 09:08:36.803602 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:36.803583 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-99.ec2.internal\" not found" Jul 13 09:08:36.826965 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.826944 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1c25182e5a9db0ffb01a93fffea4671a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal\" (UID: \"1c25182e5a9db0ffb01a93fffea4671a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal" Jul 13 09:08:36.827040 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.826967 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c25182e5a9db0ffb01a93fffea4671a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal\" (UID: \"1c25182e5a9db0ffb01a93fffea4671a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal" Jul 13 09:08:36.827040 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.826982 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bbafb723d47a725ee02db2c015defda8-config\") pod \"kube-apiserver-proxy-ip-10-0-129-99.ec2.internal\" (UID: \"bbafb723d47a725ee02db2c015defda8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-99.ec2.internal" Jul 13 09:08:36.904656 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:36.904636 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-99.ec2.internal\" not found" Jul 13 09:08:36.928141 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.928088 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1c25182e5a9db0ffb01a93fffea4671a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal\" (UID: \"1c25182e5a9db0ffb01a93fffea4671a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal" Jul 13 09:08:36.928141 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.928121 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c25182e5a9db0ffb01a93fffea4671a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal\" (UID: \"1c25182e5a9db0ffb01a93fffea4671a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal" Jul 13 09:08:36.928299 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.928151 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bbafb723d47a725ee02db2c015defda8-config\") pod \"kube-apiserver-proxy-ip-10-0-129-99.ec2.internal\" (UID: \"bbafb723d47a725ee02db2c015defda8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-99.ec2.internal" Jul 13 09:08:36.928299 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.928223 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1c25182e5a9db0ffb01a93fffea4671a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal\" (UID: \"1c25182e5a9db0ffb01a93fffea4671a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal" Jul 13 09:08:36.928299 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.928263 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bbafb723d47a725ee02db2c015defda8-config\") pod \"kube-apiserver-proxy-ip-10-0-129-99.ec2.internal\" (UID: \"bbafb723d47a725ee02db2c015defda8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-99.ec2.internal" Jul 13 09:08:36.928299 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:36.928297 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c25182e5a9db0ffb01a93fffea4671a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal\" (UID: \"1c25182e5a9db0ffb01a93fffea4671a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal" Jul 13 09:08:37.005462 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:37.005443 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-99.ec2.internal\" not found" Jul 13 09:08:37.090994 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:37.090959 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal" Jul 13 09:08:37.091087 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:37.090974 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-99.ec2.internal" Jul 13 09:08:37.105955 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:37.105936 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-99.ec2.internal\" not found" Jul 13 09:08:37.210033 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:37.209982 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-99.ec2.internal\" not found" Jul 13 09:08:37.310756 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:37.310731 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-99.ec2.internal\" not found" Jul 13 09:08:37.411578 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:37.411545 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-99.ec2.internal\" not found" Jul 13 09:08:37.459225 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:37.459176 2552 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 13 09:08:37.459748 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:37.459334 2552 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 13 09:08:37.459748 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:37.459358 2552 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 13 09:08:37.511796 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:37.511734 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-99.ec2.internal\" not found" Jul 13 09:08:37.533268 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:37.533234 2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-12 09:03:37 +0000 UTC" deadline="2027-12-17 14:41:37.946612024 +0000 UTC" Jul 13 09:08:37.533268 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:37.533264 2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12533h33m0.41335186s" Jul 13 09:08:37.612742 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:37.612716 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-99.ec2.internal\" not found" Jul 13 09:08:37.713742 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:37.713715 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-99.ec2.internal\" not found" Jul 13 09:08:37.731282 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:37.731249 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c25182e5a9db0ffb01a93fffea4671a.slice/crio-dd5adf8ef5d4650bb3e7621067d3e76442ff811d836ea8223a6362b1778c5fd1 WatchSource:0}: Error finding container dd5adf8ef5d4650bb3e7621067d3e76442ff811d836ea8223a6362b1778c5fd1: Status 404 returned error can't find the container with id dd5adf8ef5d4650bb3e7621067d3e76442ff811d836ea8223a6362b1778c5fd1 Jul 13 09:08:37.735277 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:37.735262 2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 09:08:37.743733 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:37.743714 2552 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 09:08:37.812838 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:37.812813 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbafb723d47a725ee02db2c015defda8.slice/crio-e0a91c6b579ad1d09be235da23acc1b44bc30e91dc43574d8bb0164fcc11b940 WatchSource:0}: Error finding container e0a91c6b579ad1d09be235da23acc1b44bc30e91dc43574d8bb0164fcc11b940: Status 404 returned error can't find the container with id e0a91c6b579ad1d09be235da23acc1b44bc30e91dc43574d8bb0164fcc11b940 Jul 13 09:08:37.824363 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:37.824347 2552 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal" Jul 13 09:08:37.836464 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:37.836445 2552 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 13 09:08:37.838305 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:37.838291 2552 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-99.ec2.internal" Jul 13 09:08:37.848617 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:37.848598 2552 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 13 09:08:37.946828 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:37.946808 2552 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 09:08:38.397161 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.397130 2552 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 09:08:38.503379 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.503350 2552 apiserver.go:52] "Watching apiserver" Jul 13 09:08:38.512362 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.512336 2552 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 13 09:08:38.512920 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.512891 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-wxkd7","kube-system/konnectivity-agent-c6q7b","kube-system/kube-apiserver-proxy-ip-10-0-129-99.ec2.internal","openshift-dns/node-resolver-d9vsl","openshift-image-registry/node-ca-s8xqc","openshift-multus/multus-additional-cni-plugins-bggk8","openshift-multus/multus-lssfs","openshift-multus/network-metrics-daemon-nsd2z","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6gxn","openshift-cluster-node-tuning-operator/tuned-pqrf7","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal","openshift-network-diagnostics/network-check-target-4p88d","openshift-network-operator/iptables-alerter-lj9ds","openshift-ovn-kubernetes/ovnkube-node-gp6mv"] Jul 13 09:08:38.515646 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.515624 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.518079 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.517935 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lj9ds" Jul 13 09:08:38.519120 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.518896 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 13 09:08:38.519120 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.518912 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 13 09:08:38.519120 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.518912 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 13 09:08:38.519120 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.518973 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-85m76\"" Jul 13 09:08:38.519423 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.519381 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 13 09:08:38.520788 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.520304 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 13 09:08:38.520788 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.520345 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.520788 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.520415 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 13 09:08:38.520788 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.520734 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 13 09:08:38.521364 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.521343 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-pkblv\"" Jul 13 09:08:38.522760 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.522733 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-d9vsl" Jul 13 09:08:38.524506 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.524028 2552 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 13 09:08:38.524506 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.524365 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 13 09:08:38.524506 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.524497 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 13 09:08:38.524845 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.524670 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5tb9p\"" Jul 13 09:08:38.524845 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.524835 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 13 09:08:38.524968 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.524946 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 13 09:08:38.527120 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.525588 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 13 09:08:38.527120 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.525647 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 13 09:08:38.527120 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.525795 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 13 09:08:38.527120 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.526046 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 13 09:08:38.527120 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.526282 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-b55tw\"" Jul 13 09:08:38.527970 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.527927 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-s8xqc" Jul 13 09:08:38.530845 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.530576 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bggk8" Jul 13 09:08:38.530845 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.530601 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 13 09:08:38.530845 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.530713 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 13 09:08:38.531201 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.531182 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 13 09:08:38.531380 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.531360 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vj5q2\"" Jul 13 09:08:38.533243 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.533225 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6gxn" Jul 13 09:08:38.533460 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.533443 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 13 09:08:38.533593 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.533477 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nhf8v\"" Jul 13 09:08:38.533864 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.533844 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 13 09:08:38.536049 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.536027 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wxkd7" Jul 13 09:08:38.536158 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:38.536131 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wxkd7" podUID="708d0abc-11a7-403e-add0-08c1d93e33f8" Jul 13 09:08:38.536428 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.536408 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 13 09:08:38.536515 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.536439 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 13 09:08:38.536515 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.536442 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-w6qzs\"" Jul 13 09:08:38.536633 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.536560 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 13 09:08:38.537839 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.537795 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a1d3bed-1a9d-42ce-b6b5-454d4aaac7da-tmp-dir\") pod \"node-resolver-d9vsl\" (UID: \"1a1d3bed-1a9d-42ce-b6b5-454d4aaac7da\") " pod="openshift-dns/node-resolver-d9vsl" Jul 13 09:08:38.537993 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.537854 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/777385c6-f18f-4560-902e-67b4b1885fcd-host\") pod \"node-ca-s8xqc\" (UID: \"777385c6-f18f-4560-902e-67b4b1885fcd\") " pod="openshift-image-registry/node-ca-s8xqc" Jul 13 09:08:38.537993 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.537879 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-etc-openvswitch\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.537993 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.537930 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/777385c6-f18f-4560-902e-67b4b1885fcd-serviceca\") pod \"node-ca-s8xqc\" (UID: \"777385c6-f18f-4560-902e-67b4b1885fcd\") " pod="openshift-image-registry/node-ca-s8xqc" Jul 13 09:08:38.537993 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.537969 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-host-run-k8s-cni-cncf-io\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.538204 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.537997 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-multus-conf-dir\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.538204 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.538022 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6900b633-aa4e-448a-9cca-4647d10ec24a-iptables-alerter-script\") pod \"iptables-alerter-lj9ds\" (UID: \"6900b633-aa4e-448a-9cca-4647d10ec24a\") " pod="openshift-network-operator/iptables-alerter-lj9ds" Jul 13 09:08:38.538204 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.538047 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6900b633-aa4e-448a-9cca-4647d10ec24a-host-slash\") pod \"iptables-alerter-lj9ds\" (UID: \"6900b633-aa4e-448a-9cca-4647d10ec24a\") " pod="openshift-network-operator/iptables-alerter-lj9ds" Jul 13 09:08:38.538204 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.538077 2552 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 13 09:08:38.538204 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.538091 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-run-ovn\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.538204 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.538116 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-host-cni-netd\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.538204 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.538141 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/47250d22-56b6-4503-af3f-36a9e79691b2-env-overrides\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.538204 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.538165 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/47250d22-56b6-4503-af3f-36a9e79691b2-ovn-node-metrics-cert\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.538204 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.538192 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vccz\" (UniqueName: \"kubernetes.io/projected/47250d22-56b6-4503-af3f-36a9e79691b2-kube-api-access-5vccz\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.538673 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.538220 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z54fq\" (UniqueName: \"kubernetes.io/projected/777385c6-f18f-4560-902e-67b4b1885fcd-kube-api-access-z54fq\") pod \"node-ca-s8xqc\" (UID: \"777385c6-f18f-4560-902e-67b4b1885fcd\") " pod="openshift-image-registry/node-ca-s8xqc" Jul 13 09:08:38.538673 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.538264 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d3ff727f-48d2-4148-a379-0698694f0a0e-multus-daemon-config\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.538673 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.538289 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/47250d22-56b6-4503-af3f-36a9e79691b2-ovnkube-script-lib\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.538673 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.538428 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-host-kubelet\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.538673 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.538465 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-systemd-units\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.538673 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.538490 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-host-run-netns\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.538673 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.538514 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-run-openvswitch\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.538673 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.538537 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-host-run-ovn-kubernetes\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.538673 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.538578 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-host-run-multus-certs\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.538673 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.538613 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-run-systemd\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.539155 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.538706 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-multus-cni-dir\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.539155 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.538783 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-os-release\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.539155 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.538830 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-host-var-lib-cni-multus\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.539155 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.538900 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-node-log\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.539155 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.538985 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d3ff727f-48d2-4148-a379-0698694f0a0e-cni-binary-copy\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.539155 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.539069 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-host-run-netns\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.539155 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.539116 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-host-var-lib-cni-bin\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.539155 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.539146 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-hostroot\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.539594 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.539181 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwm4s\" (UniqueName: \"kubernetes.io/projected/d3ff727f-48d2-4148-a379-0698694f0a0e-kube-api-access-xwm4s\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.539594 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.539205 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.539594 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.539250 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8mdh\" (UniqueName: \"kubernetes.io/projected/1a1d3bed-1a9d-42ce-b6b5-454d4aaac7da-kube-api-access-g8mdh\") pod \"node-resolver-d9vsl\" (UID: \"1a1d3bed-1a9d-42ce-b6b5-454d4aaac7da\") " pod="openshift-dns/node-resolver-d9vsl" Jul 13 09:08:38.539594 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.539274 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-system-cni-dir\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.539594 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.539333 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-cnibin\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.539594 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.539360 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-host-var-lib-kubelet\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.539594 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.539430 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-etc-kubernetes\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.539594 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.539471 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-var-lib-openvswitch\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.539594 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.539509 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/47250d22-56b6-4503-af3f-36a9e79691b2-ovnkube-config\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.539594 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.539536 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-multus-socket-dir-parent\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.539594 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.539552 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-c6q7b" Jul 13 09:08:38.539594 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.539566 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwsl9\" (UniqueName: \"kubernetes.io/projected/6900b633-aa4e-448a-9cca-4647d10ec24a-kube-api-access-dwsl9\") pod \"iptables-alerter-lj9ds\" (UID: \"6900b633-aa4e-448a-9cca-4647d10ec24a\") " pod="openshift-network-operator/iptables-alerter-lj9ds" Jul 13 09:08:38.539594 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.539595 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-host-slash\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.540280 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.539626 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-log-socket\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.540280 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.539649 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-host-cni-bin\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.540280 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.539670 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1a1d3bed-1a9d-42ce-b6b5-454d4aaac7da-hosts-file\") pod \"node-resolver-d9vsl\" (UID: \"1a1d3bed-1a9d-42ce-b6b5-454d4aaac7da\") " pod="openshift-dns/node-resolver-d9vsl" Jul 13 09:08:38.541481 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.541458 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.542327 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.542310 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 13 09:08:38.542440 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.542415 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 13 09:08:38.542556 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.542539 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-ntsqp\"" Jul 13 09:08:38.544030 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.543990 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsd2z" Jul 13 09:08:38.544104 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:38.544059 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nsd2z" podUID="a24643eb-b486-4118-9d57-77c6b07ef957" Jul 13 09:08:38.544370 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.544349 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7sxmv\"" Jul 13 09:08:38.544463 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.544359 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 13 09:08:38.544528 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.544473 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 13 09:08:38.546568 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.546549 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4p88d" Jul 13 09:08:38.546687 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:38.546607 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4p88d" podUID="d6624d2e-c7a5-44e5-9ace-f88ecb9ff201" Jul 13 09:08:38.563488 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.563459 2552 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2ght6" Jul 13 09:08:38.571185 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.571164 2552 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2ght6" Jul 13 09:08:38.625721 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.625699 2552 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 13 09:08:38.640569 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.640545 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-os-release\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.640678 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.640576 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5nnt\" (UniqueName: \"kubernetes.io/projected/d6624d2e-c7a5-44e5-9ace-f88ecb9ff201-kube-api-access-r5nnt\") pod \"network-check-target-4p88d\" (UID: \"d6624d2e-c7a5-44e5-9ace-f88ecb9ff201\") " pod="openshift-network-diagnostics/network-check-target-4p88d" Jul 13 09:08:38.640678 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.640593 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-node-log\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.640678 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.640608 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-hostroot\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.640678 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.640624 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwm4s\" (UniqueName: \"kubernetes.io/projected/d3ff727f-48d2-4148-a379-0698694f0a0e-kube-api-access-xwm4s\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.640678 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.640647 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/53caa5bd-dca1-4654-a407-0fc895185601-cnibin\") pod \"multus-additional-cni-plugins-bggk8\" (UID: \"53caa5bd-dca1-4654-a407-0fc895185601\") " pod="openshift-multus/multus-additional-cni-plugins-bggk8" Jul 13 09:08:38.640678 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.640671 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.640970 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.640689 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d-agent-certs\") pod \"konnectivity-agent-c6q7b\" (UID: \"a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d\") " pod="kube-system/konnectivity-agent-c6q7b" Jul 13 09:08:38.640970 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.640698 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-node-log\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.640970 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.640716 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-os-release\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.640970 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.640704 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3675cf57-c496-4ad4-8a40-c1759c0363b2-etc-modprobe-d\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.640970 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.640753 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-hostroot\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.640970 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.640754 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.640970 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.640768 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a24643eb-b486-4118-9d57-77c6b07ef957-metrics-certs\") pod \"network-metrics-daemon-nsd2z\" (UID: \"a24643eb-b486-4118-9d57-77c6b07ef957\") " pod="openshift-multus/network-metrics-daemon-nsd2z" Jul 13 09:08:38.640970 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.640790 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/47250d22-56b6-4503-af3f-36a9e79691b2-ovnkube-config\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.640970 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.640834 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwsl9\" (UniqueName: \"kubernetes.io/projected/6900b633-aa4e-448a-9cca-4647d10ec24a-kube-api-access-dwsl9\") pod \"iptables-alerter-lj9ds\" (UID: \"6900b633-aa4e-448a-9cca-4647d10ec24a\") " pod="openshift-network-operator/iptables-alerter-lj9ds" Jul 13 09:08:38.640970 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.640862 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/53caa5bd-dca1-4654-a407-0fc895185601-system-cni-dir\") pod \"multus-additional-cni-plugins-bggk8\" (UID: \"53caa5bd-dca1-4654-a407-0fc895185601\") " pod="openshift-multus/multus-additional-cni-plugins-bggk8" Jul 13 09:08:38.640970 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.640889 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/53caa5bd-dca1-4654-a407-0fc895185601-os-release\") pod \"multus-additional-cni-plugins-bggk8\" (UID: \"53caa5bd-dca1-4654-a407-0fc895185601\") " pod="openshift-multus/multus-additional-cni-plugins-bggk8" Jul 13 09:08:38.640970 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.640931 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/53caa5bd-dca1-4654-a407-0fc895185601-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bggk8\" (UID: \"53caa5bd-dca1-4654-a407-0fc895185601\") " pod="openshift-multus/multus-additional-cni-plugins-bggk8" Jul 13 09:08:38.640970 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.640965 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/708d0abc-11a7-403e-add0-08c1d93e33f8-dbus\") pod \"global-pull-secret-syncer-wxkd7\" (UID: \"708d0abc-11a7-403e-add0-08c1d93e33f8\") " pod="kube-system/global-pull-secret-syncer-wxkd7" Jul 13 09:08:38.641675 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.640996 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-host-slash\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.641675 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641026 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-host-cni-bin\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.641675 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641035 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-host-slash\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.641675 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641052 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/777385c6-f18f-4560-902e-67b4b1885fcd-host\") pod \"node-ca-s8xqc\" (UID: \"777385c6-f18f-4560-902e-67b4b1885fcd\") " pod="openshift-image-registry/node-ca-s8xqc" Jul 13 09:08:38.641675 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641075 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/777385c6-f18f-4560-902e-67b4b1885fcd-serviceca\") pod \"node-ca-s8xqc\" (UID: \"777385c6-f18f-4560-902e-67b4b1885fcd\") " pod="openshift-image-registry/node-ca-s8xqc" Jul 13 09:08:38.641675 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641090 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-host-cni-bin\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.641675 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641099 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3675cf57-c496-4ad4-8a40-c1759c0363b2-etc-sysctl-conf\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.641675 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641124 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3675cf57-c496-4ad4-8a40-c1759c0363b2-etc-tuned\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.641675 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641124 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/777385c6-f18f-4560-902e-67b4b1885fcd-host\") pod \"node-ca-s8xqc\" (UID: \"777385c6-f18f-4560-902e-67b4b1885fcd\") " pod="openshift-image-registry/node-ca-s8xqc" Jul 13 09:08:38.641675 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641149 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-etc-openvswitch\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.641675 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641174 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-host-run-k8s-cni-cncf-io\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.641675 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641198 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-multus-conf-dir\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.641675 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641221 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6900b633-aa4e-448a-9cca-4647d10ec24a-host-slash\") pod \"iptables-alerter-lj9ds\" (UID: \"6900b633-aa4e-448a-9cca-4647d10ec24a\") " pod="openshift-network-operator/iptables-alerter-lj9ds" Jul 13 09:08:38.641675 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641206 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-etc-openvswitch\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.641675 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641257 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3675cf57-c496-4ad4-8a40-c1759c0363b2-etc-sysconfig\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.641675 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641283 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3675cf57-c496-4ad4-8a40-c1759c0363b2-tmp\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.641675 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641293 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6900b633-aa4e-448a-9cca-4647d10ec24a-host-slash\") pod \"iptables-alerter-lj9ds\" (UID: \"6900b633-aa4e-448a-9cca-4647d10ec24a\") " pod="openshift-network-operator/iptables-alerter-lj9ds" Jul 13 09:08:38.641675 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641290 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-host-run-k8s-cni-cncf-io\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.642543 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641313 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djkpp\" (UniqueName: \"kubernetes.io/projected/a24643eb-b486-4118-9d57-77c6b07ef957-kube-api-access-djkpp\") pod \"network-metrics-daemon-nsd2z\" (UID: \"a24643eb-b486-4118-9d57-77c6b07ef957\") " pod="openshift-multus/network-metrics-daemon-nsd2z" Jul 13 09:08:38.642543 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641342 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-run-ovn\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.642543 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641367 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-host-cni-netd\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.642543 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641377 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-run-ovn\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.642543 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641345 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-multus-conf-dir\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.642543 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641394 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/708d0abc-11a7-403e-add0-08c1d93e33f8-kubelet-config\") pod \"global-pull-secret-syncer-wxkd7\" (UID: \"708d0abc-11a7-403e-add0-08c1d93e33f8\") " pod="kube-system/global-pull-secret-syncer-wxkd7" Jul 13 09:08:38.642543 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641417 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-host-cni-netd\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.642543 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641420 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d-konnectivity-ca\") pod \"konnectivity-agent-c6q7b\" (UID: \"a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d\") " pod="kube-system/konnectivity-agent-c6q7b" Jul 13 09:08:38.642543 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641428 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/47250d22-56b6-4503-af3f-36a9e79691b2-ovnkube-config\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.642543 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641442 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-host-kubelet\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.642543 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641459 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-systemd-units\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.642543 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641474 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-host-run-netns\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.642543 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641493 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-run-openvswitch\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.642543 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641510 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-host-kubelet\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.642543 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641518 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-host-run-ovn-kubernetes\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.642543 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641548 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-host-run-multus-certs\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.642543 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641551 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-run-openvswitch\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.643437 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641560 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-host-run-ovn-kubernetes\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.643437 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641562 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/777385c6-f18f-4560-902e-67b4b1885fcd-serviceca\") pod \"node-ca-s8xqc\" (UID: \"777385c6-f18f-4560-902e-67b4b1885fcd\") " pod="openshift-image-registry/node-ca-s8xqc" Jul 13 09:08:38.643437 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641562 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-host-run-netns\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.643437 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641522 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-systemd-units\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.643437 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641574 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a412e01c-34c9-4949-9c0e-9a83142be38a-socket-dir\") pod \"aws-ebs-csi-driver-node-c6gxn\" (UID: \"a412e01c-34c9-4949-9c0e-9a83142be38a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6gxn" Jul 13 09:08:38.643437 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641602 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-host-run-multus-certs\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.643437 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641626 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3675cf57-c496-4ad4-8a40-c1759c0363b2-etc-kubernetes\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.643437 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641651 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-host-var-lib-cni-multus\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.643437 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641671 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-host-var-lib-cni-multus\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.643437 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641678 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/708d0abc-11a7-403e-add0-08c1d93e33f8-original-pull-secret\") pod \"global-pull-secret-syncer-wxkd7\" (UID: \"708d0abc-11a7-403e-add0-08c1d93e33f8\") " pod="kube-system/global-pull-secret-syncer-wxkd7" Jul 13 09:08:38.643437 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641706 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3675cf57-c496-4ad4-8a40-c1759c0363b2-etc-sysctl-d\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.643437 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641730 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3675cf57-c496-4ad4-8a40-c1759c0363b2-sys\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.643437 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641746 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3675cf57-c496-4ad4-8a40-c1759c0363b2-host\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.643437 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641763 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d3ff727f-48d2-4148-a379-0698694f0a0e-cni-binary-copy\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.643437 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641781 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-host-run-netns\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.643437 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641810 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-host-var-lib-cni-bin\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.643437 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641823 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-host-run-netns\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.644132 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641894 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a412e01c-34c9-4949-9c0e-9a83142be38a-etc-selinux\") pod \"aws-ebs-csi-driver-node-c6gxn\" (UID: \"a412e01c-34c9-4949-9c0e-9a83142be38a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6gxn" Jul 13 09:08:38.644132 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641920 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3675cf57-c496-4ad4-8a40-c1759c0363b2-var-lib-kubelet\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.644132 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641929 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-host-var-lib-cni-bin\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.644132 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641946 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8mdh\" (UniqueName: \"kubernetes.io/projected/1a1d3bed-1a9d-42ce-b6b5-454d4aaac7da-kube-api-access-g8mdh\") pod \"node-resolver-d9vsl\" (UID: \"1a1d3bed-1a9d-42ce-b6b5-454d4aaac7da\") " pod="openshift-dns/node-resolver-d9vsl" Jul 13 09:08:38.644132 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641971 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-system-cni-dir\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.644132 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.641994 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-cnibin\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.644132 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.642012 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-system-cni-dir\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.644132 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.642017 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-host-var-lib-kubelet\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.644132 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.642045 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-etc-kubernetes\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.644132 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.642057 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-host-var-lib-kubelet\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.644132 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.642077 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a412e01c-34c9-4949-9c0e-9a83142be38a-registration-dir\") pod \"aws-ebs-csi-driver-node-c6gxn\" (UID: \"a412e01c-34c9-4949-9c0e-9a83142be38a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6gxn" Jul 13 09:08:38.644132 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.642090 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-etc-kubernetes\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.644132 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.642104 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27lk6\" (UniqueName: \"kubernetes.io/projected/a412e01c-34c9-4949-9c0e-9a83142be38a-kube-api-access-27lk6\") pod \"aws-ebs-csi-driver-node-c6gxn\" (UID: \"a412e01c-34c9-4949-9c0e-9a83142be38a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6gxn" Jul 13 09:08:38.644132 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.642101 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-cnibin\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.644132 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.642135 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-var-lib-openvswitch\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.644132 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.642169 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-var-lib-openvswitch\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.644132 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.642185 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-multus-socket-dir-parent\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.645132 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.642232 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a412e01c-34c9-4949-9c0e-9a83142be38a-device-dir\") pod \"aws-ebs-csi-driver-node-c6gxn\" (UID: \"a412e01c-34c9-4949-9c0e-9a83142be38a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6gxn" Jul 13 09:08:38.645132 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.642258 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3675cf57-c496-4ad4-8a40-c1759c0363b2-etc-systemd\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.645132 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.642284 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3675cf57-c496-4ad4-8a40-c1759c0363b2-run\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.645132 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.642300 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-multus-socket-dir-parent\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.645132 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.642312 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-log-socket\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.645132 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.642328 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d3ff727f-48d2-4148-a379-0698694f0a0e-cni-binary-copy\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.645132 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.642344 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1a1d3bed-1a9d-42ce-b6b5-454d4aaac7da-hosts-file\") pod \"node-resolver-d9vsl\" (UID: \"1a1d3bed-1a9d-42ce-b6b5-454d4aaac7da\") " pod="openshift-dns/node-resolver-d9vsl" Jul 13 09:08:38.645132 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.642354 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-log-socket\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.645132 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.642391 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a1d3bed-1a9d-42ce-b6b5-454d4aaac7da-tmp-dir\") pod \"node-resolver-d9vsl\" (UID: \"1a1d3bed-1a9d-42ce-b6b5-454d4aaac7da\") " pod="openshift-dns/node-resolver-d9vsl" Jul 13 09:08:38.645132 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.642415 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6900b633-aa4e-448a-9cca-4647d10ec24a-iptables-alerter-script\") pod \"iptables-alerter-lj9ds\" (UID: \"6900b633-aa4e-448a-9cca-4647d10ec24a\") " pod="openshift-network-operator/iptables-alerter-lj9ds" Jul 13 09:08:38.645132 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.642441 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/53caa5bd-dca1-4654-a407-0fc895185601-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bggk8\" (UID: \"53caa5bd-dca1-4654-a407-0fc895185601\") " pod="openshift-multus/multus-additional-cni-plugins-bggk8" Jul 13 09:08:38.645132 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.642541 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/53caa5bd-dca1-4654-a407-0fc895185601-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bggk8\" (UID: \"53caa5bd-dca1-4654-a407-0fc895185601\") " pod="openshift-multus/multus-additional-cni-plugins-bggk8" Jul 13 09:08:38.645132 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.642569 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2vnt\" (UniqueName: \"kubernetes.io/projected/53caa5bd-dca1-4654-a407-0fc895185601-kube-api-access-g2vnt\") pod \"multus-additional-cni-plugins-bggk8\" (UID: \"53caa5bd-dca1-4654-a407-0fc895185601\") " pod="openshift-multus/multus-additional-cni-plugins-bggk8" Jul 13 09:08:38.645132 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.642583 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1a1d3bed-1a9d-42ce-b6b5-454d4aaac7da-hosts-file\") pod \"node-resolver-d9vsl\" (UID: \"1a1d3bed-1a9d-42ce-b6b5-454d4aaac7da\") " pod="openshift-dns/node-resolver-d9vsl" Jul 13 09:08:38.645132 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.642588 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a412e01c-34c9-4949-9c0e-9a83142be38a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c6gxn\" (UID: \"a412e01c-34c9-4949-9c0e-9a83142be38a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6gxn" Jul 13 09:08:38.645132 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.642634 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/47250d22-56b6-4503-af3f-36a9e79691b2-env-overrides\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.645132 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.642662 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/47250d22-56b6-4503-af3f-36a9e79691b2-ovn-node-metrics-cert\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.645876 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.642689 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vccz\" (UniqueName: \"kubernetes.io/projected/47250d22-56b6-4503-af3f-36a9e79691b2-kube-api-access-5vccz\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.645876 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.642715 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z54fq\" (UniqueName: \"kubernetes.io/projected/777385c6-f18f-4560-902e-67b4b1885fcd-kube-api-access-z54fq\") pod \"node-ca-s8xqc\" (UID: \"777385c6-f18f-4560-902e-67b4b1885fcd\") " pod="openshift-image-registry/node-ca-s8xqc" Jul 13 09:08:38.645876 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.642744 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d3ff727f-48d2-4148-a379-0698694f0a0e-multus-daemon-config\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.645876 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.642772 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a412e01c-34c9-4949-9c0e-9a83142be38a-sys-fs\") pod \"aws-ebs-csi-driver-node-c6gxn\" (UID: \"a412e01c-34c9-4949-9c0e-9a83142be38a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6gxn" Jul 13 09:08:38.645876 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.642800 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3675cf57-c496-4ad4-8a40-c1759c0363b2-lib-modules\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.645876 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.642845 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a1d3bed-1a9d-42ce-b6b5-454d4aaac7da-tmp-dir\") pod \"node-resolver-d9vsl\" (UID: \"1a1d3bed-1a9d-42ce-b6b5-454d4aaac7da\") " pod="openshift-dns/node-resolver-d9vsl" Jul 13 09:08:38.645876 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.642921 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6900b633-aa4e-448a-9cca-4647d10ec24a-iptables-alerter-script\") pod \"iptables-alerter-lj9ds\" (UID: \"6900b633-aa4e-448a-9cca-4647d10ec24a\") " pod="openshift-network-operator/iptables-alerter-lj9ds" Jul 13 09:08:38.645876 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.643011 2552 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 13 09:08:38.645876 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.643049 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/47250d22-56b6-4503-af3f-36a9e79691b2-ovnkube-script-lib\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.645876 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.643075 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/53caa5bd-dca1-4654-a407-0fc895185601-cni-binary-copy\") pod \"multus-additional-cni-plugins-bggk8\" (UID: \"53caa5bd-dca1-4654-a407-0fc895185601\") " pod="openshift-multus/multus-additional-cni-plugins-bggk8" Jul 13 09:08:38.645876 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.643089 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/47250d22-56b6-4503-af3f-36a9e79691b2-env-overrides\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.645876 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.643092 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vh5n9\" (UniqueName: \"kubernetes.io/projected/3675cf57-c496-4ad4-8a40-c1759c0363b2-kube-api-access-vh5n9\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.645876 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.643148 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-run-systemd\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.645876 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.643193 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/47250d22-56b6-4503-af3f-36a9e79691b2-run-systemd\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.645876 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.643237 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-multus-cni-dir\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.645876 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.643376 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3ff727f-48d2-4148-a379-0698694f0a0e-multus-cni-dir\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.645876 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.643482 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d3ff727f-48d2-4148-a379-0698694f0a0e-multus-daemon-config\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.645876 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.643567 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/47250d22-56b6-4503-af3f-36a9e79691b2-ovnkube-script-lib\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.646728 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.646710 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/47250d22-56b6-4503-af3f-36a9e79691b2-ovn-node-metrics-cert\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.650025 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.649970 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwsl9\" (UniqueName: \"kubernetes.io/projected/6900b633-aa4e-448a-9cca-4647d10ec24a-kube-api-access-dwsl9\") pod \"iptables-alerter-lj9ds\" (UID: \"6900b633-aa4e-448a-9cca-4647d10ec24a\") " pod="openshift-network-operator/iptables-alerter-lj9ds" Jul 13 09:08:38.650231 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.650162 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwm4s\" (UniqueName: \"kubernetes.io/projected/d3ff727f-48d2-4148-a379-0698694f0a0e-kube-api-access-xwm4s\") pod \"multus-lssfs\" (UID: \"d3ff727f-48d2-4148-a379-0698694f0a0e\") " pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.650477 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.650456 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8mdh\" (UniqueName: \"kubernetes.io/projected/1a1d3bed-1a9d-42ce-b6b5-454d4aaac7da-kube-api-access-g8mdh\") pod \"node-resolver-d9vsl\" (UID: \"1a1d3bed-1a9d-42ce-b6b5-454d4aaac7da\") " pod="openshift-dns/node-resolver-d9vsl" Jul 13 09:08:38.651532 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.651505 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vccz\" (UniqueName: \"kubernetes.io/projected/47250d22-56b6-4503-af3f-36a9e79691b2-kube-api-access-5vccz\") pod \"ovnkube-node-gp6mv\" (UID: \"47250d22-56b6-4503-af3f-36a9e79691b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.651761 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.651742 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z54fq\" (UniqueName: \"kubernetes.io/projected/777385c6-f18f-4560-902e-67b4b1885fcd-kube-api-access-z54fq\") pod \"node-ca-s8xqc\" (UID: \"777385c6-f18f-4560-902e-67b4b1885fcd\") " pod="openshift-image-registry/node-ca-s8xqc" Jul 13 09:08:38.665320 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.665269 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-99.ec2.internal" event={"ID":"bbafb723d47a725ee02db2c015defda8","Type":"ContainerStarted","Data":"e0a91c6b579ad1d09be235da23acc1b44bc30e91dc43574d8bb0164fcc11b940"} Jul 13 09:08:38.666463 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.666444 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal" event={"ID":"1c25182e5a9db0ffb01a93fffea4671a","Type":"ContainerStarted","Data":"dd5adf8ef5d4650bb3e7621067d3e76442ff811d836ea8223a6362b1778c5fd1"} Jul 13 09:08:38.740473 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.740449 2552 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 09:08:38.743415 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.743396 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/53caa5bd-dca1-4654-a407-0fc895185601-system-cni-dir\") pod \"multus-additional-cni-plugins-bggk8\" (UID: \"53caa5bd-dca1-4654-a407-0fc895185601\") " pod="openshift-multus/multus-additional-cni-plugins-bggk8" Jul 13 09:08:38.743522 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.743430 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/53caa5bd-dca1-4654-a407-0fc895185601-os-release\") pod \"multus-additional-cni-plugins-bggk8\" (UID: \"53caa5bd-dca1-4654-a407-0fc895185601\") " pod="openshift-multus/multus-additional-cni-plugins-bggk8" Jul 13 09:08:38.743522 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.743456 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/53caa5bd-dca1-4654-a407-0fc895185601-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bggk8\" (UID: \"53caa5bd-dca1-4654-a407-0fc895185601\") " pod="openshift-multus/multus-additional-cni-plugins-bggk8" Jul 13 09:08:38.743522 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.743483 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/708d0abc-11a7-403e-add0-08c1d93e33f8-dbus\") pod \"global-pull-secret-syncer-wxkd7\" (UID: \"708d0abc-11a7-403e-add0-08c1d93e33f8\") " pod="kube-system/global-pull-secret-syncer-wxkd7" Jul 13 09:08:38.743522 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.743510 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3675cf57-c496-4ad4-8a40-c1759c0363b2-etc-sysctl-conf\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.743736 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.743568 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/53caa5bd-dca1-4654-a407-0fc895185601-system-cni-dir\") pod \"multus-additional-cni-plugins-bggk8\" (UID: \"53caa5bd-dca1-4654-a407-0fc895185601\") " pod="openshift-multus/multus-additional-cni-plugins-bggk8" Jul 13 09:08:38.743736 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.743585 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/53caa5bd-dca1-4654-a407-0fc895185601-os-release\") pod \"multus-additional-cni-plugins-bggk8\" (UID: \"53caa5bd-dca1-4654-a407-0fc895185601\") " pod="openshift-multus/multus-additional-cni-plugins-bggk8" Jul 13 09:08:38.743736 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.743610 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3675cf57-c496-4ad4-8a40-c1759c0363b2-etc-tuned\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.743736 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.743636 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3675cf57-c496-4ad4-8a40-c1759c0363b2-etc-sysconfig\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.743736 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.743646 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3675cf57-c496-4ad4-8a40-c1759c0363b2-etc-sysctl-conf\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.743736 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.743652 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3675cf57-c496-4ad4-8a40-c1759c0363b2-tmp\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.744042 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.743777 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/708d0abc-11a7-403e-add0-08c1d93e33f8-dbus\") pod \"global-pull-secret-syncer-wxkd7\" (UID: \"708d0abc-11a7-403e-add0-08c1d93e33f8\") " pod="kube-system/global-pull-secret-syncer-wxkd7" Jul 13 09:08:38.744042 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.743920 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3675cf57-c496-4ad4-8a40-c1759c0363b2-etc-sysconfig\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.744042 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.744006 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djkpp\" (UniqueName: \"kubernetes.io/projected/a24643eb-b486-4118-9d57-77c6b07ef957-kube-api-access-djkpp\") pod \"network-metrics-daemon-nsd2z\" (UID: \"a24643eb-b486-4118-9d57-77c6b07ef957\") " pod="openshift-multus/network-metrics-daemon-nsd2z" Jul 13 09:08:38.744201 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.744061 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/708d0abc-11a7-403e-add0-08c1d93e33f8-kubelet-config\") pod \"global-pull-secret-syncer-wxkd7\" (UID: \"708d0abc-11a7-403e-add0-08c1d93e33f8\") " pod="kube-system/global-pull-secret-syncer-wxkd7" Jul 13 09:08:38.744201 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.744088 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d-konnectivity-ca\") pod \"konnectivity-agent-c6q7b\" (UID: \"a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d\") " pod="kube-system/konnectivity-agent-c6q7b" Jul 13 09:08:38.744201 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.744121 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a412e01c-34c9-4949-9c0e-9a83142be38a-socket-dir\") pod \"aws-ebs-csi-driver-node-c6gxn\" (UID: \"a412e01c-34c9-4949-9c0e-9a83142be38a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6gxn" Jul 13 09:08:38.744201 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.744131 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/53caa5bd-dca1-4654-a407-0fc895185601-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bggk8\" (UID: \"53caa5bd-dca1-4654-a407-0fc895185601\") " pod="openshift-multus/multus-additional-cni-plugins-bggk8" Jul 13 09:08:38.744201 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.744147 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3675cf57-c496-4ad4-8a40-c1759c0363b2-etc-kubernetes\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.744201 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.744189 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/708d0abc-11a7-403e-add0-08c1d93e33f8-original-pull-secret\") pod \"global-pull-secret-syncer-wxkd7\" (UID: \"708d0abc-11a7-403e-add0-08c1d93e33f8\") " pod="kube-system/global-pull-secret-syncer-wxkd7" Jul 13 09:08:38.745149 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.744209 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3675cf57-c496-4ad4-8a40-c1759c0363b2-etc-kubernetes\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.745149 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.744223 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3675cf57-c496-4ad4-8a40-c1759c0363b2-etc-sysctl-d\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.745149 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.744251 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3675cf57-c496-4ad4-8a40-c1759c0363b2-sys\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.745149 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.744277 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/708d0abc-11a7-403e-add0-08c1d93e33f8-kubelet-config\") pod \"global-pull-secret-syncer-wxkd7\" (UID: \"708d0abc-11a7-403e-add0-08c1d93e33f8\") " pod="kube-system/global-pull-secret-syncer-wxkd7" Jul 13 09:08:38.745149 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.744298 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3675cf57-c496-4ad4-8a40-c1759c0363b2-host\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.745149 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.744328 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a412e01c-34c9-4949-9c0e-9a83142be38a-etc-selinux\") pod \"aws-ebs-csi-driver-node-c6gxn\" (UID: \"a412e01c-34c9-4949-9c0e-9a83142be38a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6gxn" Jul 13 09:08:38.745149 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.744353 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3675cf57-c496-4ad4-8a40-c1759c0363b2-var-lib-kubelet\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.745149 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.744381 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a412e01c-34c9-4949-9c0e-9a83142be38a-registration-dir\") pod \"aws-ebs-csi-driver-node-c6gxn\" (UID: \"a412e01c-34c9-4949-9c0e-9a83142be38a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6gxn" Jul 13 09:08:38.745149 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.744407 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27lk6\" (UniqueName: \"kubernetes.io/projected/a412e01c-34c9-4949-9c0e-9a83142be38a-kube-api-access-27lk6\") pod \"aws-ebs-csi-driver-node-c6gxn\" (UID: \"a412e01c-34c9-4949-9c0e-9a83142be38a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6gxn" Jul 13 09:08:38.745149 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.744412 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a412e01c-34c9-4949-9c0e-9a83142be38a-socket-dir\") pod \"aws-ebs-csi-driver-node-c6gxn\" (UID: \"a412e01c-34c9-4949-9c0e-9a83142be38a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6gxn" Jul 13 09:08:38.745149 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.744424 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3675cf57-c496-4ad4-8a40-c1759c0363b2-etc-sysctl-d\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.745149 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.744434 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a412e01c-34c9-4949-9c0e-9a83142be38a-device-dir\") pod \"aws-ebs-csi-driver-node-c6gxn\" (UID: \"a412e01c-34c9-4949-9c0e-9a83142be38a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6gxn" Jul 13 09:08:38.745149 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.744466 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3675cf57-c496-4ad4-8a40-c1759c0363b2-etc-systemd\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.745149 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.744492 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3675cf57-c496-4ad4-8a40-c1759c0363b2-run\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.745149 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.744506 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a412e01c-34c9-4949-9c0e-9a83142be38a-etc-selinux\") pod \"aws-ebs-csi-driver-node-c6gxn\" (UID: \"a412e01c-34c9-4949-9c0e-9a83142be38a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6gxn" Jul 13 09:08:38.745149 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:38.744531 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 09:08:38.745149 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.744546 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3675cf57-c496-4ad4-8a40-c1759c0363b2-etc-systemd\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.745149 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.744601 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3675cf57-c496-4ad4-8a40-c1759c0363b2-run\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.746003 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:38.744617 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/708d0abc-11a7-403e-add0-08c1d93e33f8-original-pull-secret podName:708d0abc-11a7-403e-add0-08c1d93e33f8 nodeName:}" failed. No retries permitted until 2026-07-13 09:08:39.244574185 +0000 UTC m=+3.124529436 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/708d0abc-11a7-403e-add0-08c1d93e33f8-original-pull-secret") pod "global-pull-secret-syncer-wxkd7" (UID: "708d0abc-11a7-403e-add0-08c1d93e33f8") : object "kube-system"/"original-pull-secret" not registered Jul 13 09:08:38.746003 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.744645 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a412e01c-34c9-4949-9c0e-9a83142be38a-registration-dir\") pod \"aws-ebs-csi-driver-node-c6gxn\" (UID: \"a412e01c-34c9-4949-9c0e-9a83142be38a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6gxn" Jul 13 09:08:38.746003 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.744683 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/53caa5bd-dca1-4654-a407-0fc895185601-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bggk8\" (UID: \"53caa5bd-dca1-4654-a407-0fc895185601\") " pod="openshift-multus/multus-additional-cni-plugins-bggk8" Jul 13 09:08:38.746003 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.744711 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/53caa5bd-dca1-4654-a407-0fc895185601-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bggk8\" (UID: \"53caa5bd-dca1-4654-a407-0fc895185601\") " pod="openshift-multus/multus-additional-cni-plugins-bggk8" Jul 13 09:08:38.746003 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.744727 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3675cf57-c496-4ad4-8a40-c1759c0363b2-host\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.746003 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.745137 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d-konnectivity-ca\") pod \"konnectivity-agent-c6q7b\" (UID: \"a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d\") " pod="kube-system/konnectivity-agent-c6q7b" Jul 13 09:08:38.746003 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.745277 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3675cf57-c496-4ad4-8a40-c1759c0363b2-var-lib-kubelet\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.746003 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.744782 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a412e01c-34c9-4949-9c0e-9a83142be38a-device-dir\") pod \"aws-ebs-csi-driver-node-c6gxn\" (UID: \"a412e01c-34c9-4949-9c0e-9a83142be38a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6gxn" Jul 13 09:08:38.746003 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.745502 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/53caa5bd-dca1-4654-a407-0fc895185601-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bggk8\" (UID: \"53caa5bd-dca1-4654-a407-0fc895185601\") " pod="openshift-multus/multus-additional-cni-plugins-bggk8" Jul 13 09:08:38.746003 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.745583 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2vnt\" (UniqueName: \"kubernetes.io/projected/53caa5bd-dca1-4654-a407-0fc895185601-kube-api-access-g2vnt\") pod \"multus-additional-cni-plugins-bggk8\" (UID: \"53caa5bd-dca1-4654-a407-0fc895185601\") " pod="openshift-multus/multus-additional-cni-plugins-bggk8" Jul 13 09:08:38.746003 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.745625 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3675cf57-c496-4ad4-8a40-c1759c0363b2-sys\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.746003 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.745644 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a412e01c-34c9-4949-9c0e-9a83142be38a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c6gxn\" (UID: \"a412e01c-34c9-4949-9c0e-9a83142be38a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6gxn" Jul 13 09:08:38.746003 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.745755 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a412e01c-34c9-4949-9c0e-9a83142be38a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c6gxn\" (UID: \"a412e01c-34c9-4949-9c0e-9a83142be38a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6gxn" Jul 13 09:08:38.746003 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.745811 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a412e01c-34c9-4949-9c0e-9a83142be38a-sys-fs\") pod \"aws-ebs-csi-driver-node-c6gxn\" (UID: \"a412e01c-34c9-4949-9c0e-9a83142be38a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6gxn" Jul 13 09:08:38.746003 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.745864 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3675cf57-c496-4ad4-8a40-c1759c0363b2-lib-modules\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.746003 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.745934 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/53caa5bd-dca1-4654-a407-0fc895185601-cni-binary-copy\") pod \"multus-additional-cni-plugins-bggk8\" (UID: \"53caa5bd-dca1-4654-a407-0fc895185601\") " pod="openshift-multus/multus-additional-cni-plugins-bggk8" Jul 13 09:08:38.746777 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.745985 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vh5n9\" (UniqueName: \"kubernetes.io/projected/3675cf57-c496-4ad4-8a40-c1759c0363b2-kube-api-access-vh5n9\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.746777 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.746025 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5nnt\" (UniqueName: \"kubernetes.io/projected/d6624d2e-c7a5-44e5-9ace-f88ecb9ff201-kube-api-access-r5nnt\") pod \"network-check-target-4p88d\" (UID: \"d6624d2e-c7a5-44e5-9ace-f88ecb9ff201\") " pod="openshift-network-diagnostics/network-check-target-4p88d" Jul 13 09:08:38.746777 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.746035 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3675cf57-c496-4ad4-8a40-c1759c0363b2-lib-modules\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.746777 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.746067 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/53caa5bd-dca1-4654-a407-0fc895185601-cnibin\") pod \"multus-additional-cni-plugins-bggk8\" (UID: \"53caa5bd-dca1-4654-a407-0fc895185601\") " pod="openshift-multus/multus-additional-cni-plugins-bggk8" Jul 13 09:08:38.746777 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.746076 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a412e01c-34c9-4949-9c0e-9a83142be38a-sys-fs\") pod \"aws-ebs-csi-driver-node-c6gxn\" (UID: \"a412e01c-34c9-4949-9c0e-9a83142be38a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6gxn" Jul 13 09:08:38.746777 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.746101 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d-agent-certs\") pod \"konnectivity-agent-c6q7b\" (UID: \"a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d\") " pod="kube-system/konnectivity-agent-c6q7b" Jul 13 09:08:38.746777 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.746122 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3675cf57-c496-4ad4-8a40-c1759c0363b2-etc-modprobe-d\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.746777 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.746134 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/53caa5bd-dca1-4654-a407-0fc895185601-cnibin\") pod \"multus-additional-cni-plugins-bggk8\" (UID: \"53caa5bd-dca1-4654-a407-0fc895185601\") " pod="openshift-multus/multus-additional-cni-plugins-bggk8" Jul 13 09:08:38.746777 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.746140 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a24643eb-b486-4118-9d57-77c6b07ef957-metrics-certs\") pod \"network-metrics-daemon-nsd2z\" (UID: \"a24643eb-b486-4118-9d57-77c6b07ef957\") " pod="openshift-multus/network-metrics-daemon-nsd2z" Jul 13 09:08:38.746777 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.746578 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/53caa5bd-dca1-4654-a407-0fc895185601-cni-binary-copy\") pod \"multus-additional-cni-plugins-bggk8\" (UID: \"53caa5bd-dca1-4654-a407-0fc895185601\") " pod="openshift-multus/multus-additional-cni-plugins-bggk8" Jul 13 09:08:38.746777 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.746656 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3675cf57-c496-4ad4-8a40-c1759c0363b2-etc-tuned\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.746777 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:38.746781 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 09:08:38.747380 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.746796 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3675cf57-c496-4ad4-8a40-c1759c0363b2-etc-modprobe-d\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.747380 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:38.746934 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a24643eb-b486-4118-9d57-77c6b07ef957-metrics-certs podName:a24643eb-b486-4118-9d57-77c6b07ef957 nodeName:}" failed. No retries permitted until 2026-07-13 09:08:39.246912881 +0000 UTC m=+3.126868176 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a24643eb-b486-4118-9d57-77c6b07ef957-metrics-certs") pod "network-metrics-daemon-nsd2z" (UID: "a24643eb-b486-4118-9d57-77c6b07ef957") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 09:08:38.747380 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.747308 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/53caa5bd-dca1-4654-a407-0fc895185601-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bggk8\" (UID: \"53caa5bd-dca1-4654-a407-0fc895185601\") " pod="openshift-multus/multus-additional-cni-plugins-bggk8" Jul 13 09:08:38.748983 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.748961 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3675cf57-c496-4ad4-8a40-c1759c0363b2-tmp\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.750231 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.750180 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d-agent-certs\") pod \"konnectivity-agent-c6q7b\" (UID: \"a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d\") " pod="kube-system/konnectivity-agent-c6q7b" Jul 13 09:08:38.752059 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:38.751914 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 09:08:38.752059 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:38.751951 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 09:08:38.752059 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:38.751966 2552 projected.go:196] Error preparing data for projected volume kube-api-access-r5nnt for pod openshift-network-diagnostics/network-check-target-4p88d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 09:08:38.752059 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:38.752031 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d6624d2e-c7a5-44e5-9ace-f88ecb9ff201-kube-api-access-r5nnt podName:d6624d2e-c7a5-44e5-9ace-f88ecb9ff201 nodeName:}" failed. No retries permitted until 2026-07-13 09:08:39.252014265 +0000 UTC m=+3.131969516 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-r5nnt" (UniqueName: "kubernetes.io/projected/d6624d2e-c7a5-44e5-9ace-f88ecb9ff201-kube-api-access-r5nnt") pod "network-check-target-4p88d" (UID: "d6624d2e-c7a5-44e5-9ace-f88ecb9ff201") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 09:08:38.753659 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.753626 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27lk6\" (UniqueName: \"kubernetes.io/projected/a412e01c-34c9-4949-9c0e-9a83142be38a-kube-api-access-27lk6\") pod \"aws-ebs-csi-driver-node-c6gxn\" (UID: \"a412e01c-34c9-4949-9c0e-9a83142be38a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6gxn" Jul 13 09:08:38.754059 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.754034 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djkpp\" (UniqueName: \"kubernetes.io/projected/a24643eb-b486-4118-9d57-77c6b07ef957-kube-api-access-djkpp\") pod \"network-metrics-daemon-nsd2z\" (UID: \"a24643eb-b486-4118-9d57-77c6b07ef957\") " pod="openshift-multus/network-metrics-daemon-nsd2z" Jul 13 09:08:38.754783 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.754760 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vh5n9\" (UniqueName: \"kubernetes.io/projected/3675cf57-c496-4ad4-8a40-c1759c0363b2-kube-api-access-vh5n9\") pod \"tuned-pqrf7\" (UID: \"3675cf57-c496-4ad4-8a40-c1759c0363b2\") " pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:38.754860 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.754831 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2vnt\" (UniqueName: \"kubernetes.io/projected/53caa5bd-dca1-4654-a407-0fc895185601-kube-api-access-g2vnt\") pod \"multus-additional-cni-plugins-bggk8\" (UID: \"53caa5bd-dca1-4654-a407-0fc895185601\") " pod="openshift-multus/multus-additional-cni-plugins-bggk8" Jul 13 09:08:38.829908 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.829876 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lssfs" Jul 13 09:08:38.840374 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.840346 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lj9ds" Jul 13 09:08:38.850487 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.850467 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:08:38.855339 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.855317 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-d9vsl" Jul 13 09:08:38.864195 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.864174 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-s8xqc" Jul 13 09:08:38.871969 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.871942 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bggk8" Jul 13 09:08:38.888922 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.888904 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6gxn" Jul 13 09:08:38.895874 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.895854 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-c6q7b" Jul 13 09:08:38.903802 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:38.903755 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" Jul 13 09:08:39.250536 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:39.250449 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a24643eb-b486-4118-9d57-77c6b07ef957-metrics-certs\") pod \"network-metrics-daemon-nsd2z\" (UID: \"a24643eb-b486-4118-9d57-77c6b07ef957\") " pod="openshift-multus/network-metrics-daemon-nsd2z" Jul 13 09:08:39.250536 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:39.250507 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/708d0abc-11a7-403e-add0-08c1d93e33f8-original-pull-secret\") pod \"global-pull-secret-syncer-wxkd7\" (UID: \"708d0abc-11a7-403e-add0-08c1d93e33f8\") " pod="kube-system/global-pull-secret-syncer-wxkd7" Jul 13 09:08:39.250757 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:39.250601 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 09:08:39.250757 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:39.250608 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 09:08:39.250757 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:39.250663 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/708d0abc-11a7-403e-add0-08c1d93e33f8-original-pull-secret podName:708d0abc-11a7-403e-add0-08c1d93e33f8 nodeName:}" failed. No retries permitted until 2026-07-13 09:08:40.25064632 +0000 UTC m=+4.130601567 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/708d0abc-11a7-403e-add0-08c1d93e33f8-original-pull-secret") pod "global-pull-secret-syncer-wxkd7" (UID: "708d0abc-11a7-403e-add0-08c1d93e33f8") : object "kube-system"/"original-pull-secret" not registered Jul 13 09:08:39.250757 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:39.250686 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a24643eb-b486-4118-9d57-77c6b07ef957-metrics-certs podName:a24643eb-b486-4118-9d57-77c6b07ef957 nodeName:}" failed. No retries permitted until 2026-07-13 09:08:40.250674467 +0000 UTC m=+4.130629715 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a24643eb-b486-4118-9d57-77c6b07ef957-metrics-certs") pod "network-metrics-daemon-nsd2z" (UID: "a24643eb-b486-4118-9d57-77c6b07ef957") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 09:08:39.351841 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:39.351804 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5nnt\" (UniqueName: \"kubernetes.io/projected/d6624d2e-c7a5-44e5-9ace-f88ecb9ff201-kube-api-access-r5nnt\") pod \"network-check-target-4p88d\" (UID: \"d6624d2e-c7a5-44e5-9ace-f88ecb9ff201\") " pod="openshift-network-diagnostics/network-check-target-4p88d" Jul 13 09:08:39.352008 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:39.351980 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 09:08:39.352008 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:39.352000 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 09:08:39.352080 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:39.352011 2552 projected.go:196] Error preparing data for projected volume kube-api-access-r5nnt for pod openshift-network-diagnostics/network-check-target-4p88d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 09:08:39.352080 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:39.352066 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d6624d2e-c7a5-44e5-9ace-f88ecb9ff201-kube-api-access-r5nnt podName:d6624d2e-c7a5-44e5-9ace-f88ecb9ff201 nodeName:}" failed. No retries permitted until 2026-07-13 09:08:40.352050887 +0000 UTC m=+4.232006133 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-r5nnt" (UniqueName: "kubernetes.io/projected/d6624d2e-c7a5-44e5-9ace-f88ecb9ff201-kube-api-access-r5nnt") pod "network-check-target-4p88d" (UID: "d6624d2e-c7a5-44e5-9ace-f88ecb9ff201") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 09:08:39.467960 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:39.467920 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3ff727f_48d2_4148_a379_0698694f0a0e.slice/crio-85a1edd7dbf38250089b3f1e5fb755455f637051a4aaf374531a76adcd4dc8bf WatchSource:0}: Error finding container 85a1edd7dbf38250089b3f1e5fb755455f637051a4aaf374531a76adcd4dc8bf: Status 404 returned error can't find the container with id 85a1edd7dbf38250089b3f1e5fb755455f637051a4aaf374531a76adcd4dc8bf Jul 13 09:08:39.474666 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:39.474643 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3675cf57_c496_4ad4_8a40_c1759c0363b2.slice/crio-a4ed021a37a9f44370542407af485a5a19eff287d3c8ca90063b2746f2e0257a WatchSource:0}: Error finding container a4ed021a37a9f44370542407af485a5a19eff287d3c8ca90063b2746f2e0257a: Status 404 returned error can't find the container with id a4ed021a37a9f44370542407af485a5a19eff287d3c8ca90063b2746f2e0257a Jul 13 09:08:39.497594 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:39.497559 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6900b633_aa4e_448a_9cca_4647d10ec24a.slice/crio-20527e9bf613bd3d7864b1c7f2e7a4f6312b2010d0fdee1ad8828ba85826bff1 WatchSource:0}: Error finding container 20527e9bf613bd3d7864b1c7f2e7a4f6312b2010d0fdee1ad8828ba85826bff1: Status 404 returned error can't find the container with id 20527e9bf613bd3d7864b1c7f2e7a4f6312b2010d0fdee1ad8828ba85826bff1 Jul 13 09:08:39.497934 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:39.497909 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda57ce5c3_5f12_4858_9a2c_be4bcc36dd9d.slice/crio-183666573aea7548e9f3efd8179d540fc3b16abadaf4a718fe698788897192f6 WatchSource:0}: Error finding container 183666573aea7548e9f3efd8179d540fc3b16abadaf4a718fe698788897192f6: Status 404 returned error can't find the container with id 183666573aea7548e9f3efd8179d540fc3b16abadaf4a718fe698788897192f6 Jul 13 09:08:39.498298 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:39.498281 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47250d22_56b6_4503_af3f_36a9e79691b2.slice/crio-371c68a36e210154d65545a8482f90d0d13b4fbb8b4e57e92135429134b66d62 WatchSource:0}: Error finding container 371c68a36e210154d65545a8482f90d0d13b4fbb8b4e57e92135429134b66d62: Status 404 returned error can't find the container with id 371c68a36e210154d65545a8482f90d0d13b4fbb8b4e57e92135429134b66d62 Jul 13 09:08:39.499893 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:39.499746 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod777385c6_f18f_4560_902e_67b4b1885fcd.slice/crio-728ac0cfeafb1a5f5331a78971bec933b83327eeae19c5801eec24258296425e WatchSource:0}: Error finding container 728ac0cfeafb1a5f5331a78971bec933b83327eeae19c5801eec24258296425e: Status 404 returned error can't find the container with id 728ac0cfeafb1a5f5331a78971bec933b83327eeae19c5801eec24258296425e Jul 13 09:08:39.500679 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:39.500617 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda412e01c_34c9_4949_9c0e_9a83142be38a.slice/crio-0f8a3394e86bf0a3c5005183c3c70acf2685e70438b537732c4d61b4fc2c7a6f WatchSource:0}: Error finding container 0f8a3394e86bf0a3c5005183c3c70acf2685e70438b537732c4d61b4fc2c7a6f: Status 404 returned error can't find the container with id 0f8a3394e86bf0a3c5005183c3c70acf2685e70438b537732c4d61b4fc2c7a6f Jul 13 09:08:39.501595 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:39.501501 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a1d3bed_1a9d_42ce_b6b5_454d4aaac7da.slice/crio-ff4ff3fc63cc7cf9ed98f60091a4e1b12325ab711332ceeb7c02f02e0db3a694 WatchSource:0}: Error finding container ff4ff3fc63cc7cf9ed98f60091a4e1b12325ab711332ceeb7c02f02e0db3a694: Status 404 returned error can't find the container with id ff4ff3fc63cc7cf9ed98f60091a4e1b12325ab711332ceeb7c02f02e0db3a694 Jul 13 09:08:39.502989 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:08:39.502953 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53caa5bd_dca1_4654_a407_0fc895185601.slice/crio-a919aeb50c0a68a4e395f4c6c475b4bb69beae43b409d0a5f890f8c71103e199 WatchSource:0}: Error finding container a919aeb50c0a68a4e395f4c6c475b4bb69beae43b409d0a5f890f8c71103e199: Status 404 returned error can't find the container with id a919aeb50c0a68a4e395f4c6c475b4bb69beae43b409d0a5f890f8c71103e199 Jul 13 09:08:39.575557 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:39.575510 2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 09:03:39 +0000 UTC" deadline="2028-04-27 09:36:05.252075804 +0000 UTC" Jul 13 09:08:39.575557 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:39.575555 2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15696h27m25.676523868s" Jul 13 09:08:39.660500 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:39.660471 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4p88d" Jul 13 09:08:39.660633 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:39.660596 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4p88d" podUID="d6624d2e-c7a5-44e5-9ace-f88ecb9ff201" Jul 13 09:08:39.670255 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:39.670200 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-d9vsl" event={"ID":"1a1d3bed-1a9d-42ce-b6b5-454d4aaac7da","Type":"ContainerStarted","Data":"ff4ff3fc63cc7cf9ed98f60091a4e1b12325ab711332ceeb7c02f02e0db3a694"} Jul 13 09:08:39.671496 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:39.671470 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6gxn" event={"ID":"a412e01c-34c9-4949-9c0e-9a83142be38a","Type":"ContainerStarted","Data":"0f8a3394e86bf0a3c5005183c3c70acf2685e70438b537732c4d61b4fc2c7a6f"} Jul 13 09:08:39.672750 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:39.672731 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-s8xqc" event={"ID":"777385c6-f18f-4560-902e-67b4b1885fcd","Type":"ContainerStarted","Data":"728ac0cfeafb1a5f5331a78971bec933b83327eeae19c5801eec24258296425e"} Jul 13 09:08:39.674077 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:39.674044 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-c6q7b" event={"ID":"a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d","Type":"ContainerStarted","Data":"183666573aea7548e9f3efd8179d540fc3b16abadaf4a718fe698788897192f6"} Jul 13 09:08:39.675299 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:39.675275 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" event={"ID":"3675cf57-c496-4ad4-8a40-c1759c0363b2","Type":"ContainerStarted","Data":"a4ed021a37a9f44370542407af485a5a19eff287d3c8ca90063b2746f2e0257a"} Jul 13 09:08:39.677482 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:39.677457 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-99.ec2.internal" event={"ID":"bbafb723d47a725ee02db2c015defda8","Type":"ContainerStarted","Data":"3419ff2cf0984aeb44692dcb18c31ffa65284635be00c485439a8f858cfc613a"} Jul 13 09:08:39.678611 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:39.678585 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bggk8" event={"ID":"53caa5bd-dca1-4654-a407-0fc895185601","Type":"ContainerStarted","Data":"a919aeb50c0a68a4e395f4c6c475b4bb69beae43b409d0a5f890f8c71103e199"} Jul 13 09:08:39.680051 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:39.680026 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" event={"ID":"47250d22-56b6-4503-af3f-36a9e79691b2","Type":"ContainerStarted","Data":"371c68a36e210154d65545a8482f90d0d13b4fbb8b4e57e92135429134b66d62"} Jul 13 09:08:39.681377 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:39.681352 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lj9ds" event={"ID":"6900b633-aa4e-448a-9cca-4647d10ec24a","Type":"ContainerStarted","Data":"20527e9bf613bd3d7864b1c7f2e7a4f6312b2010d0fdee1ad8828ba85826bff1"} Jul 13 09:08:39.682747 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:39.682726 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lssfs" event={"ID":"d3ff727f-48d2-4148-a379-0698694f0a0e","Type":"ContainerStarted","Data":"85a1edd7dbf38250089b3f1e5fb755455f637051a4aaf374531a76adcd4dc8bf"} Jul 13 09:08:40.260057 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:40.260017 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/708d0abc-11a7-403e-add0-08c1d93e33f8-original-pull-secret\") pod \"global-pull-secret-syncer-wxkd7\" (UID: \"708d0abc-11a7-403e-add0-08c1d93e33f8\") " pod="kube-system/global-pull-secret-syncer-wxkd7" Jul 13 09:08:40.260238 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:40.260117 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a24643eb-b486-4118-9d57-77c6b07ef957-metrics-certs\") pod \"network-metrics-daemon-nsd2z\" (UID: \"a24643eb-b486-4118-9d57-77c6b07ef957\") " pod="openshift-multus/network-metrics-daemon-nsd2z" Jul 13 09:08:40.260350 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:40.260276 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 09:08:40.260350 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:40.260341 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a24643eb-b486-4118-9d57-77c6b07ef957-metrics-certs podName:a24643eb-b486-4118-9d57-77c6b07ef957 nodeName:}" failed. No retries permitted until 2026-07-13 09:08:42.260320525 +0000 UTC m=+6.140275777 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a24643eb-b486-4118-9d57-77c6b07ef957-metrics-certs") pod "network-metrics-daemon-nsd2z" (UID: "a24643eb-b486-4118-9d57-77c6b07ef957") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 09:08:40.260774 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:40.260752 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 09:08:40.260855 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:40.260813 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/708d0abc-11a7-403e-add0-08c1d93e33f8-original-pull-secret podName:708d0abc-11a7-403e-add0-08c1d93e33f8 nodeName:}" failed. No retries permitted until 2026-07-13 09:08:42.260796071 +0000 UTC m=+6.140751321 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/708d0abc-11a7-403e-add0-08c1d93e33f8-original-pull-secret") pod "global-pull-secret-syncer-wxkd7" (UID: "708d0abc-11a7-403e-add0-08c1d93e33f8") : object "kube-system"/"original-pull-secret" not registered Jul 13 09:08:40.361236 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:40.361193 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5nnt\" (UniqueName: \"kubernetes.io/projected/d6624d2e-c7a5-44e5-9ace-f88ecb9ff201-kube-api-access-r5nnt\") pod \"network-check-target-4p88d\" (UID: \"d6624d2e-c7a5-44e5-9ace-f88ecb9ff201\") " pod="openshift-network-diagnostics/network-check-target-4p88d" Jul 13 09:08:40.361439 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:40.361402 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 09:08:40.361439 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:40.361429 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 09:08:40.361563 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:40.361442 2552 projected.go:196] Error preparing data for projected volume kube-api-access-r5nnt for pod openshift-network-diagnostics/network-check-target-4p88d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 09:08:40.361563 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:40.361500 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d6624d2e-c7a5-44e5-9ace-f88ecb9ff201-kube-api-access-r5nnt podName:d6624d2e-c7a5-44e5-9ace-f88ecb9ff201 nodeName:}" failed. No retries permitted until 2026-07-13 09:08:42.361482902 +0000 UTC m=+6.241438165 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-r5nnt" (UniqueName: "kubernetes.io/projected/d6624d2e-c7a5-44e5-9ace-f88ecb9ff201-kube-api-access-r5nnt") pod "network-check-target-4p88d" (UID: "d6624d2e-c7a5-44e5-9ace-f88ecb9ff201") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 09:08:40.575855 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:40.575816 2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 09:03:39 +0000 UTC" deadline="2028-02-25 16:06:36.893505461 +0000 UTC" Jul 13 09:08:40.576236 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:40.575855 2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14214h57m56.317655197s" Jul 13 09:08:40.660595 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:40.660562 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wxkd7" Jul 13 09:08:40.660752 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:40.660707 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wxkd7" podUID="708d0abc-11a7-403e-add0-08c1d93e33f8" Jul 13 09:08:40.660829 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:40.660782 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsd2z" Jul 13 09:08:40.660944 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:40.660917 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nsd2z" podUID="a24643eb-b486-4118-9d57-77c6b07ef957" Jul 13 09:08:41.660467 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:41.660387 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4p88d" Jul 13 09:08:41.660928 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:41.660530 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4p88d" podUID="d6624d2e-c7a5-44e5-9ace-f88ecb9ff201" Jul 13 09:08:42.280751 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:42.280671 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/708d0abc-11a7-403e-add0-08c1d93e33f8-original-pull-secret\") pod \"global-pull-secret-syncer-wxkd7\" (UID: \"708d0abc-11a7-403e-add0-08c1d93e33f8\") " pod="kube-system/global-pull-secret-syncer-wxkd7" Jul 13 09:08:42.280918 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:42.280758 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a24643eb-b486-4118-9d57-77c6b07ef957-metrics-certs\") pod \"network-metrics-daemon-nsd2z\" (UID: \"a24643eb-b486-4118-9d57-77c6b07ef957\") " pod="openshift-multus/network-metrics-daemon-nsd2z" Jul 13 09:08:42.280918 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:42.280859 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 09:08:42.281037 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:42.280920 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a24643eb-b486-4118-9d57-77c6b07ef957-metrics-certs podName:a24643eb-b486-4118-9d57-77c6b07ef957 nodeName:}" failed. No retries permitted until 2026-07-13 09:08:46.280901097 +0000 UTC m=+10.160856350 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a24643eb-b486-4118-9d57-77c6b07ef957-metrics-certs") pod "network-metrics-daemon-nsd2z" (UID: "a24643eb-b486-4118-9d57-77c6b07ef957") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 09:08:42.281037 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:42.280859 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 09:08:42.281037 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:42.280967 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/708d0abc-11a7-403e-add0-08c1d93e33f8-original-pull-secret podName:708d0abc-11a7-403e-add0-08c1d93e33f8 nodeName:}" failed. No retries permitted until 2026-07-13 09:08:46.280956695 +0000 UTC m=+10.160911945 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/708d0abc-11a7-403e-add0-08c1d93e33f8-original-pull-secret") pod "global-pull-secret-syncer-wxkd7" (UID: "708d0abc-11a7-403e-add0-08c1d93e33f8") : object "kube-system"/"original-pull-secret" not registered Jul 13 09:08:42.382111 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:42.382073 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5nnt\" (UniqueName: \"kubernetes.io/projected/d6624d2e-c7a5-44e5-9ace-f88ecb9ff201-kube-api-access-r5nnt\") pod \"network-check-target-4p88d\" (UID: \"d6624d2e-c7a5-44e5-9ace-f88ecb9ff201\") " pod="openshift-network-diagnostics/network-check-target-4p88d" Jul 13 09:08:42.382360 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:42.382281 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 09:08:42.382360 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:42.382311 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 09:08:42.382360 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:42.382325 2552 projected.go:196] Error preparing data for projected volume kube-api-access-r5nnt for pod openshift-network-diagnostics/network-check-target-4p88d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 09:08:42.382546 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:42.382382 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d6624d2e-c7a5-44e5-9ace-f88ecb9ff201-kube-api-access-r5nnt podName:d6624d2e-c7a5-44e5-9ace-f88ecb9ff201 nodeName:}" failed. No retries permitted until 2026-07-13 09:08:46.382362073 +0000 UTC m=+10.262317327 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-r5nnt" (UniqueName: "kubernetes.io/projected/d6624d2e-c7a5-44e5-9ace-f88ecb9ff201-kube-api-access-r5nnt") pod "network-check-target-4p88d" (UID: "d6624d2e-c7a5-44e5-9ace-f88ecb9ff201") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 09:08:42.660719 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:42.660301 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wxkd7" Jul 13 09:08:42.660719 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:42.660457 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wxkd7" podUID="708d0abc-11a7-403e-add0-08c1d93e33f8" Jul 13 09:08:42.660719 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:42.660512 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsd2z" Jul 13 09:08:42.660719 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:42.660647 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nsd2z" podUID="a24643eb-b486-4118-9d57-77c6b07ef957" Jul 13 09:08:42.701516 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:42.701324 2552 generic.go:356] "Generic (PLEG): container finished" podID="1c25182e5a9db0ffb01a93fffea4671a" containerID="167e189f654f0458447d57b979410765866f52c8843eb4976e906e806234e08e" exitCode=0 Jul 13 09:08:42.701516 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:42.701375 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal" event={"ID":"1c25182e5a9db0ffb01a93fffea4671a","Type":"ContainerDied","Data":"167e189f654f0458447d57b979410765866f52c8843eb4976e906e806234e08e"} Jul 13 09:08:42.718415 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:42.718350 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-99.ec2.internal" podStartSLOduration=4.7183300379999995 podStartE2EDuration="4.718330038s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 09:08:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:08:39.695127796 +0000 UTC m=+3.575083067" watchObservedRunningTime="2026-07-13 09:08:42.718330038 +0000 UTC m=+6.598285311" Jul 13 09:08:43.660669 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:43.660621 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4p88d" Jul 13 09:08:43.660870 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:43.660756 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4p88d" podUID="d6624d2e-c7a5-44e5-9ace-f88ecb9ff201" Jul 13 09:08:44.660696 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:44.660116 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wxkd7" Jul 13 09:08:44.660696 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:44.660243 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wxkd7" podUID="708d0abc-11a7-403e-add0-08c1d93e33f8" Jul 13 09:08:44.660696 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:44.660281 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsd2z" Jul 13 09:08:44.660696 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:44.660416 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nsd2z" podUID="a24643eb-b486-4118-9d57-77c6b07ef957" Jul 13 09:08:45.660135 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:45.660099 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4p88d" Jul 13 09:08:45.660335 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:45.660246 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4p88d" podUID="d6624d2e-c7a5-44e5-9ace-f88ecb9ff201" Jul 13 09:08:46.314576 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:46.314512 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a24643eb-b486-4118-9d57-77c6b07ef957-metrics-certs\") pod \"network-metrics-daemon-nsd2z\" (UID: \"a24643eb-b486-4118-9d57-77c6b07ef957\") " pod="openshift-multus/network-metrics-daemon-nsd2z" Jul 13 09:08:46.314576 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:46.314579 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/708d0abc-11a7-403e-add0-08c1d93e33f8-original-pull-secret\") pod \"global-pull-secret-syncer-wxkd7\" (UID: \"708d0abc-11a7-403e-add0-08c1d93e33f8\") " pod="kube-system/global-pull-secret-syncer-wxkd7" Jul 13 09:08:46.315075 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:46.314738 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 09:08:46.315075 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:46.314802 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/708d0abc-11a7-403e-add0-08c1d93e33f8-original-pull-secret podName:708d0abc-11a7-403e-add0-08c1d93e33f8 nodeName:}" failed. No retries permitted until 2026-07-13 09:08:54.314781177 +0000 UTC m=+18.194736426 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/708d0abc-11a7-403e-add0-08c1d93e33f8-original-pull-secret") pod "global-pull-secret-syncer-wxkd7" (UID: "708d0abc-11a7-403e-add0-08c1d93e33f8") : object "kube-system"/"original-pull-secret" not registered Jul 13 09:08:46.315075 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:46.314894 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 09:08:46.315075 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:46.314942 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a24643eb-b486-4118-9d57-77c6b07ef957-metrics-certs podName:a24643eb-b486-4118-9d57-77c6b07ef957 nodeName:}" failed. No retries permitted until 2026-07-13 09:08:54.314927987 +0000 UTC m=+18.194883248 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a24643eb-b486-4118-9d57-77c6b07ef957-metrics-certs") pod "network-metrics-daemon-nsd2z" (UID: "a24643eb-b486-4118-9d57-77c6b07ef957") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 09:08:46.415406 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:46.415364 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5nnt\" (UniqueName: \"kubernetes.io/projected/d6624d2e-c7a5-44e5-9ace-f88ecb9ff201-kube-api-access-r5nnt\") pod \"network-check-target-4p88d\" (UID: \"d6624d2e-c7a5-44e5-9ace-f88ecb9ff201\") " pod="openshift-network-diagnostics/network-check-target-4p88d" Jul 13 09:08:46.415583 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:46.415564 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 09:08:46.415649 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:46.415592 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 09:08:46.415649 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:46.415607 2552 projected.go:196] Error preparing data for projected volume kube-api-access-r5nnt for pod openshift-network-diagnostics/network-check-target-4p88d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 09:08:46.415753 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:46.415679 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d6624d2e-c7a5-44e5-9ace-f88ecb9ff201-kube-api-access-r5nnt podName:d6624d2e-c7a5-44e5-9ace-f88ecb9ff201 nodeName:}" failed. No retries permitted until 2026-07-13 09:08:54.415655981 +0000 UTC m=+18.295611252 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-r5nnt" (UniqueName: "kubernetes.io/projected/d6624d2e-c7a5-44e5-9ace-f88ecb9ff201-kube-api-access-r5nnt") pod "network-check-target-4p88d" (UID: "d6624d2e-c7a5-44e5-9ace-f88ecb9ff201") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 09:08:46.662040 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:46.661284 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wxkd7" Jul 13 09:08:46.662040 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:46.661420 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wxkd7" podUID="708d0abc-11a7-403e-add0-08c1d93e33f8" Jul 13 09:08:46.662040 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:46.661459 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsd2z" Jul 13 09:08:46.662040 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:46.661541 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nsd2z" podUID="a24643eb-b486-4118-9d57-77c6b07ef957" Jul 13 09:08:47.660074 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:47.660029 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4p88d" Jul 13 09:08:47.660524 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:47.660162 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4p88d" podUID="d6624d2e-c7a5-44e5-9ace-f88ecb9ff201" Jul 13 09:08:48.660064 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:48.659899 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsd2z" Jul 13 09:08:48.660502 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:48.660346 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wxkd7" Jul 13 09:08:48.660502 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:48.660456 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wxkd7" podUID="708d0abc-11a7-403e-add0-08c1d93e33f8" Jul 13 09:08:48.660930 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:48.660656 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nsd2z" podUID="a24643eb-b486-4118-9d57-77c6b07ef957" Jul 13 09:08:49.660099 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:49.660071 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4p88d" Jul 13 09:08:49.660286 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:49.660188 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4p88d" podUID="d6624d2e-c7a5-44e5-9ace-f88ecb9ff201" Jul 13 09:08:50.659946 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:50.659885 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wxkd7" Jul 13 09:08:50.660369 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:50.660016 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wxkd7" podUID="708d0abc-11a7-403e-add0-08c1d93e33f8" Jul 13 09:08:50.660369 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:50.660072 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsd2z" Jul 13 09:08:50.660369 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:50.660188 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nsd2z" podUID="a24643eb-b486-4118-9d57-77c6b07ef957" Jul 13 09:08:51.659978 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:51.659944 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4p88d" Jul 13 09:08:51.660391 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:51.660043 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4p88d" podUID="d6624d2e-c7a5-44e5-9ace-f88ecb9ff201" Jul 13 09:08:52.660261 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:52.660227 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wxkd7" Jul 13 09:08:52.660685 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:52.660272 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsd2z" Jul 13 09:08:52.660685 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:52.660346 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wxkd7" podUID="708d0abc-11a7-403e-add0-08c1d93e33f8" Jul 13 09:08:52.660685 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:52.660497 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nsd2z" podUID="a24643eb-b486-4118-9d57-77c6b07ef957" Jul 13 09:08:53.660452 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:53.660418 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4p88d" Jul 13 09:08:53.660918 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:53.660536 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4p88d" podUID="d6624d2e-c7a5-44e5-9ace-f88ecb9ff201" Jul 13 09:08:54.379832 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:54.379796 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/708d0abc-11a7-403e-add0-08c1d93e33f8-original-pull-secret\") pod \"global-pull-secret-syncer-wxkd7\" (UID: \"708d0abc-11a7-403e-add0-08c1d93e33f8\") " pod="kube-system/global-pull-secret-syncer-wxkd7" Jul 13 09:08:54.380013 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:54.379870 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a24643eb-b486-4118-9d57-77c6b07ef957-metrics-certs\") pod \"network-metrics-daemon-nsd2z\" (UID: \"a24643eb-b486-4118-9d57-77c6b07ef957\") " pod="openshift-multus/network-metrics-daemon-nsd2z" Jul 13 09:08:54.380013 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:54.379941 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 09:08:54.380013 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:54.379980 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 09:08:54.380013 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:54.380009 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/708d0abc-11a7-403e-add0-08c1d93e33f8-original-pull-secret podName:708d0abc-11a7-403e-add0-08c1d93e33f8 nodeName:}" failed. No retries permitted until 2026-07-13 09:09:10.379987372 +0000 UTC m=+34.259942619 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/708d0abc-11a7-403e-add0-08c1d93e33f8-original-pull-secret") pod "global-pull-secret-syncer-wxkd7" (UID: "708d0abc-11a7-403e-add0-08c1d93e33f8") : object "kube-system"/"original-pull-secret" not registered Jul 13 09:08:54.380251 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:54.380028 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a24643eb-b486-4118-9d57-77c6b07ef957-metrics-certs podName:a24643eb-b486-4118-9d57-77c6b07ef957 nodeName:}" failed. No retries permitted until 2026-07-13 09:09:10.380016278 +0000 UTC m=+34.259971523 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a24643eb-b486-4118-9d57-77c6b07ef957-metrics-certs") pod "network-metrics-daemon-nsd2z" (UID: "a24643eb-b486-4118-9d57-77c6b07ef957") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 09:08:54.481008 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:54.480971 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5nnt\" (UniqueName: \"kubernetes.io/projected/d6624d2e-c7a5-44e5-9ace-f88ecb9ff201-kube-api-access-r5nnt\") pod \"network-check-target-4p88d\" (UID: \"d6624d2e-c7a5-44e5-9ace-f88ecb9ff201\") " pod="openshift-network-diagnostics/network-check-target-4p88d" Jul 13 09:08:54.481186 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:54.481116 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 09:08:54.481186 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:54.481132 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 09:08:54.481186 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:54.481144 2552 projected.go:196] Error preparing data for projected volume kube-api-access-r5nnt for pod openshift-network-diagnostics/network-check-target-4p88d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 09:08:54.481386 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:54.481191 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d6624d2e-c7a5-44e5-9ace-f88ecb9ff201-kube-api-access-r5nnt podName:d6624d2e-c7a5-44e5-9ace-f88ecb9ff201 nodeName:}" failed. No retries permitted until 2026-07-13 09:09:10.481177905 +0000 UTC m=+34.361133150 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-r5nnt" (UniqueName: "kubernetes.io/projected/d6624d2e-c7a5-44e5-9ace-f88ecb9ff201-kube-api-access-r5nnt") pod "network-check-target-4p88d" (UID: "d6624d2e-c7a5-44e5-9ace-f88ecb9ff201") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 09:08:54.659955 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:54.659929 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wxkd7" Jul 13 09:08:54.660056 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:54.660029 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wxkd7" podUID="708d0abc-11a7-403e-add0-08c1d93e33f8" Jul 13 09:08:54.660107 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:54.660088 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsd2z" Jul 13 09:08:54.660222 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:54.660191 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nsd2z" podUID="a24643eb-b486-4118-9d57-77c6b07ef957" Jul 13 09:08:55.659749 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:55.659722 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4p88d" Jul 13 09:08:55.660121 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:55.659829 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4p88d" podUID="d6624d2e-c7a5-44e5-9ace-f88ecb9ff201" Jul 13 09:08:56.660746 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:56.660714 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wxkd7" Jul 13 09:08:56.661184 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:56.660826 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wxkd7" podUID="708d0abc-11a7-403e-add0-08c1d93e33f8" Jul 13 09:08:56.661184 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:56.660922 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsd2z" Jul 13 09:08:56.661184 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:56.661036 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nsd2z" podUID="a24643eb-b486-4118-9d57-77c6b07ef957" Jul 13 09:08:57.660262 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:57.660227 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4p88d" Jul 13 09:08:57.660437 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:57.660325 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4p88d" podUID="d6624d2e-c7a5-44e5-9ace-f88ecb9ff201" Jul 13 09:08:58.660441 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:58.660413 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wxkd7" Jul 13 09:08:58.660772 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:58.660529 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wxkd7" podUID="708d0abc-11a7-403e-add0-08c1d93e33f8" Jul 13 09:08:58.660772 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:58.660695 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsd2z" Jul 13 09:08:58.660880 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:58.660842 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nsd2z" podUID="a24643eb-b486-4118-9d57-77c6b07ef957" Jul 13 09:08:59.660794 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:59.660397 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4p88d" Jul 13 09:08:59.661455 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:08:59.660844 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4p88d" podUID="d6624d2e-c7a5-44e5-9ace-f88ecb9ff201" Jul 13 09:08:59.734594 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:59.734560 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal" event={"ID":"1c25182e5a9db0ffb01a93fffea4671a","Type":"ContainerStarted","Data":"13f6dce1b6ea766988dfeb15db34f0353ab38655aca5364853f97efc4258fc49"} Jul 13 09:08:59.736238 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:59.736202 2552 generic.go:356] "Generic (PLEG): container finished" podID="53caa5bd-dca1-4654-a407-0fc895185601" containerID="75edd387d1306054b86a7cbf624e438889a518178b6e2417f5aa3878824ce9e9" exitCode=0 Jul 13 09:08:59.736363 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:59.736251 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bggk8" event={"ID":"53caa5bd-dca1-4654-a407-0fc895185601","Type":"ContainerDied","Data":"75edd387d1306054b86a7cbf624e438889a518178b6e2417f5aa3878824ce9e9"} Jul 13 09:08:59.739288 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:59.739246 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gp6mv_47250d22-56b6-4503-af3f-36a9e79691b2/ovn-acl-logging/0.log" Jul 13 09:08:59.739728 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:59.739708 2552 generic.go:356] "Generic (PLEG): container finished" podID="47250d22-56b6-4503-af3f-36a9e79691b2" containerID="e297eb46aa2bb0757439921d8084256d8f756e109dc6cd50888fc20c4aac7a4a" exitCode=1 Jul 13 09:08:59.739820 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:59.739733 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" event={"ID":"47250d22-56b6-4503-af3f-36a9e79691b2","Type":"ContainerStarted","Data":"c52f2c786e0fea8adbb590fbc578636fa8e4ac6d47eaa1b2d0cb8fdb3de542cf"} Jul 13 09:08:59.739820 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:59.739805 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" event={"ID":"47250d22-56b6-4503-af3f-36a9e79691b2","Type":"ContainerStarted","Data":"e6d9473d0207b1c141ea57e78adf5957064eb60a336c2817f228aee92c3954c7"} Jul 13 09:08:59.739914 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:59.739857 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" event={"ID":"47250d22-56b6-4503-af3f-36a9e79691b2","Type":"ContainerStarted","Data":"b1e57ed7bd02e6ae0b451caabcee293d780a5f08e4499337614e63a9ee467143"} Jul 13 09:08:59.739914 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:59.739904 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" event={"ID":"47250d22-56b6-4503-af3f-36a9e79691b2","Type":"ContainerDied","Data":"e297eb46aa2bb0757439921d8084256d8f756e109dc6cd50888fc20c4aac7a4a"} Jul 13 09:08:59.740007 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:59.739953 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" event={"ID":"47250d22-56b6-4503-af3f-36a9e79691b2","Type":"ContainerStarted","Data":"13936320d5b74ed8dc2eec47d438fd17197bb9d67c38dd4d85e90a1177878644"} Jul 13 09:08:59.741412 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:59.741384 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lssfs" event={"ID":"d3ff727f-48d2-4148-a379-0698694f0a0e","Type":"ContainerStarted","Data":"3ac31fd2f4876011ac9357004b95abaf5398f2cca60c41f14e3c68f5768add3c"} Jul 13 09:08:59.742961 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:59.742936 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-d9vsl" event={"ID":"1a1d3bed-1a9d-42ce-b6b5-454d4aaac7da","Type":"ContainerStarted","Data":"4f364e92ec3f209272d24ace5506fc4ad354c01f6ee450e8c26055552034be3e"} Jul 13 09:08:59.745001 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:59.744975 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6gxn" event={"ID":"a412e01c-34c9-4949-9c0e-9a83142be38a","Type":"ContainerStarted","Data":"aab6872a08a5f28c10759933aee860608d2222697ce217eba67553502f8e2191"} Jul 13 09:08:59.746537 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:59.746513 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-s8xqc" event={"ID":"777385c6-f18f-4560-902e-67b4b1885fcd","Type":"ContainerStarted","Data":"62f1c541b6bd0e17dafb113043b208198ece694e1f0ffbfe6ab4901a9f541cdc"} Jul 13 09:08:59.748056 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:59.748032 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-c6q7b" event={"ID":"a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d","Type":"ContainerStarted","Data":"24af2b6560968413e416e43a55f589b973a60000cdf90472d430d1d23b214a57"} Jul 13 09:08:59.749679 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:59.749662 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" event={"ID":"3675cf57-c496-4ad4-8a40-c1759c0363b2","Type":"ContainerStarted","Data":"04d9bcc9c82541add0d9c8b46df3c7f0d417fd31d690099d034217c4e6481716"} Jul 13 09:08:59.752758 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:59.752712 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal" podStartSLOduration=21.75269912 podStartE2EDuration="21.75269912s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 09:08:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:08:59.752360467 +0000 UTC m=+23.632315930" watchObservedRunningTime="2026-07-13 09:08:59.75269912 +0000 UTC m=+23.632654386" Jul 13 09:08:59.770510 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:59.770477 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-lssfs" podStartSLOduration=3.529468858 podStartE2EDuration="22.770465833s" totalImagesPullingTime="19.240996975s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 09:08:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:08:59.770300851 +0000 UTC m=+23.650256128" watchObservedRunningTime="2026-07-13 09:08:59.770465833 +0000 UTC m=+23.650421101" Jul 13 09:08:59.810694 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:59.810653 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-d9vsl" podStartSLOduration=3.657475121 podStartE2EDuration="22.810642403s" totalImagesPullingTime="19.153167282s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 09:08:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:08:59.81037145 +0000 UTC m=+23.690326717" watchObservedRunningTime="2026-07-13 09:08:59.810642403 +0000 UTC m=+23.690597670" Jul 13 09:08:59.826347 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:59.826316 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-s8xqc" podStartSLOduration=3.695202642 podStartE2EDuration="22.826306258s" totalImagesPullingTime="19.131103616s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 09:08:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:08:59.82611973 +0000 UTC m=+23.706074997" watchObservedRunningTime="2026-07-13 09:08:59.826306258 +0000 UTC m=+23.706261524" Jul 13 09:08:59.847898 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:08:59.847865 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-c6q7b" podStartSLOduration=7.705808512 podStartE2EDuration="22.847854889s" totalImagesPullingTime="15.142046377s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:08:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:08:59.847609207 +0000 UTC m=+23.727564473" watchObservedRunningTime="2026-07-13 09:08:59.847854889 +0000 UTC m=+23.727810156" Jul 13 09:09:00.282191 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:00.282057 2552 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 13 09:09:00.592012 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:00.591890 2552 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-13T09:09:00.28217968Z","UUID":"81b4428a-346a-448f-8703-ae61fb8a3a93","Handler":null,"Name":"","Endpoint":""} Jul 13 09:09:00.593137 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:00.593120 2552 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 13 09:09:00.593263 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:00.593156 2552 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 13 09:09:00.660532 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:00.660510 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wxkd7" Jul 13 09:09:00.660674 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:00.660651 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wxkd7" podUID="708d0abc-11a7-403e-add0-08c1d93e33f8" Jul 13 09:09:00.660740 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:00.660711 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsd2z" Jul 13 09:09:00.660849 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:00.660829 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nsd2z" podUID="a24643eb-b486-4118-9d57-77c6b07ef957" Jul 13 09:09:00.754529 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:00.754505 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gp6mv_47250d22-56b6-4503-af3f-36a9e79691b2/ovn-acl-logging/0.log" Jul 13 09:09:00.754920 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:00.754893 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" event={"ID":"47250d22-56b6-4503-af3f-36a9e79691b2","Type":"ContainerStarted","Data":"d3e1574a043c349177d43acfd88a3574f992ab451dfb93d0e9a8885e326c25aa"} Jul 13 09:09:00.756561 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:00.756488 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lj9ds" event={"ID":"6900b633-aa4e-448a-9cca-4647d10ec24a","Type":"ContainerStarted","Data":"dcbed9cdc52799305071c38cfa9d31969e5257cf79d84e1dcf1f2449e1c40fbe"} Jul 13 09:09:00.758775 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:00.758706 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6gxn" event={"ID":"a412e01c-34c9-4949-9c0e-9a83142be38a","Type":"ContainerStarted","Data":"b73991249d055c353b1921315a4d45f056bd60b38375adbf45019a51ac3d5ed5"} Jul 13 09:09:00.771401 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:00.771360 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-lj9ds" podStartSLOduration=8.629196027 podStartE2EDuration="23.771348752s" totalImagesPullingTime="15.142152725s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 09:08:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:09:00.771042723 +0000 UTC m=+24.650997993" watchObservedRunningTime="2026-07-13 09:09:00.771348752 +0000 UTC m=+24.651304018" Jul 13 09:09:00.772069 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:00.772027 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-pqrf7" podStartSLOduration=4.6061908339999995 podStartE2EDuration="23.772017079s" totalImagesPullingTime="19.165826245s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 09:08:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:08:59.866768357 +0000 UTC m=+23.746723624" watchObservedRunningTime="2026-07-13 09:09:00.772017079 +0000 UTC m=+24.651972350" Jul 13 09:09:01.660403 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:01.660360 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4p88d" Jul 13 09:09:01.660758 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:01.660488 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4p88d" podUID="d6624d2e-c7a5-44e5-9ace-f88ecb9ff201" Jul 13 09:09:02.660312 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:02.660283 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wxkd7" Jul 13 09:09:02.660312 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:02.660300 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsd2z" Jul 13 09:09:02.660837 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:02.660409 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wxkd7" podUID="708d0abc-11a7-403e-add0-08c1d93e33f8" Jul 13 09:09:02.660837 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:02.660572 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nsd2z" podUID="a24643eb-b486-4118-9d57-77c6b07ef957" Jul 13 09:09:02.770583 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:02.770559 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gp6mv_47250d22-56b6-4503-af3f-36a9e79691b2/ovn-acl-logging/0.log" Jul 13 09:09:02.771042 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:02.770991 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" event={"ID":"47250d22-56b6-4503-af3f-36a9e79691b2","Type":"ContainerStarted","Data":"a11db9d56914f757b9e2542b08eb9ad444b719ebc8388dfcd811e1ce98df1c47"} Jul 13 09:09:02.773611 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:02.773588 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6gxn" event={"ID":"a412e01c-34c9-4949-9c0e-9a83142be38a","Type":"ContainerStarted","Data":"582816c350c849642387b5068c733027696d32bc1d46348e642f37b71fe59e69"} Jul 13 09:09:02.795675 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:02.795604 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6gxn" podStartSLOduration=3.616016941 podStartE2EDuration="25.79559104s" totalImagesPullingTime="22.179574099s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 09:08:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:09:02.79532254 +0000 UTC m=+26.675277808" watchObservedRunningTime="2026-07-13 09:09:02.79559104 +0000 UTC m=+26.675546307" Jul 13 09:09:03.357383 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:03.357347 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-c6q7b" Jul 13 09:09:03.358314 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:03.358288 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-c6q7b" Jul 13 09:09:03.659971 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:03.659910 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4p88d" Jul 13 09:09:03.660118 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:03.660029 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4p88d" podUID="d6624d2e-c7a5-44e5-9ace-f88ecb9ff201" Jul 13 09:09:04.660315 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:04.660147 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wxkd7" Jul 13 09:09:04.660827 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:04.660195 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsd2z" Jul 13 09:09:04.660827 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:04.660419 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wxkd7" podUID="708d0abc-11a7-403e-add0-08c1d93e33f8" Jul 13 09:09:04.660827 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:04.660490 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nsd2z" podUID="a24643eb-b486-4118-9d57-77c6b07ef957" Jul 13 09:09:04.779288 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:04.779255 2552 generic.go:356] "Generic (PLEG): container finished" podID="53caa5bd-dca1-4654-a407-0fc895185601" containerID="caae44e4aaed8fe8f5777e426502f83c433a6fbfb92220721ec339afdc0e6d92" exitCode=0 Jul 13 09:09:04.779386 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:04.779344 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bggk8" event={"ID":"53caa5bd-dca1-4654-a407-0fc895185601","Type":"ContainerDied","Data":"caae44e4aaed8fe8f5777e426502f83c433a6fbfb92220721ec339afdc0e6d92"} Jul 13 09:09:04.782663 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:04.782647 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gp6mv_47250d22-56b6-4503-af3f-36a9e79691b2/ovn-acl-logging/0.log" Jul 13 09:09:04.783016 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:04.782995 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" event={"ID":"47250d22-56b6-4503-af3f-36a9e79691b2","Type":"ContainerStarted","Data":"8ca17c7241232c5556b30703940d3575c74fbe124544eb90a3e4eb00a1e641ff"} Jul 13 09:09:04.783258 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:04.783239 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:09:04.783337 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:04.783312 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:09:04.783417 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:04.783402 2552 scope.go:117] "RemoveContainer" containerID="e297eb46aa2bb0757439921d8084256d8f756e109dc6cd50888fc20c4aac7a4a" Jul 13 09:09:04.797618 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:04.797601 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:09:05.659920 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:05.659890 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4p88d" Jul 13 09:09:05.660254 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:05.660016 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4p88d" podUID="d6624d2e-c7a5-44e5-9ace-f88ecb9ff201" Jul 13 09:09:05.788263 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:05.788240 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gp6mv_47250d22-56b6-4503-af3f-36a9e79691b2/ovn-acl-logging/0.log" Jul 13 09:09:05.788645 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:05.788554 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" event={"ID":"47250d22-56b6-4503-af3f-36a9e79691b2","Type":"ContainerStarted","Data":"98b6a0ec5f3a45ed656018f5a863cf376b217d7f22a1dda503e68e10c233e894"} Jul 13 09:09:05.788875 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:05.788857 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:09:05.802597 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:05.802578 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:09:05.814865 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:05.814817 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" podStartSLOduration=9.209858922 podStartE2EDuration="28.814801118s" totalImagesPullingTime="19.604942196s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 09:08:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:09:05.813332536 +0000 UTC m=+29.693287804" watchObservedRunningTime="2026-07-13 09:09:05.814801118 +0000 UTC m=+29.694756386" Jul 13 09:09:06.478958 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:06.478781 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wxkd7"] Jul 13 09:09:06.479057 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:06.479044 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wxkd7" Jul 13 09:09:06.479144 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:06.479127 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wxkd7" podUID="708d0abc-11a7-403e-add0-08c1d93e33f8" Jul 13 09:09:06.482080 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:06.482045 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4p88d"] Jul 13 09:09:06.482181 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:06.482141 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4p88d" Jul 13 09:09:06.482298 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:06.482273 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4p88d" podUID="d6624d2e-c7a5-44e5-9ace-f88ecb9ff201" Jul 13 09:09:06.482689 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:06.482666 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nsd2z"] Jul 13 09:09:06.482800 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:06.482782 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsd2z" Jul 13 09:09:06.482892 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:06.482871 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nsd2z" podUID="a24643eb-b486-4118-9d57-77c6b07ef957" Jul 13 09:09:06.792262 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:06.792231 2552 generic.go:356] "Generic (PLEG): container finished" podID="53caa5bd-dca1-4654-a407-0fc895185601" containerID="45041c6800bdfa0515a2f9ccef1f62b5dcbef01cf0e2e3f156675870342b2634" exitCode=0 Jul 13 09:09:06.792753 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:06.792311 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bggk8" event={"ID":"53caa5bd-dca1-4654-a407-0fc895185601","Type":"ContainerDied","Data":"45041c6800bdfa0515a2f9ccef1f62b5dcbef01cf0e2e3f156675870342b2634"} Jul 13 09:09:07.753112 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:07.753077 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-c6q7b" Jul 13 09:09:07.753309 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:07.753191 2552 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 13 09:09:07.753764 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:07.753750 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-c6q7b" Jul 13 09:09:08.660078 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:08.660010 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4p88d" Jul 13 09:09:08.660595 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:08.660019 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsd2z" Jul 13 09:09:08.660595 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:08.660131 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wxkd7" Jul 13 09:09:08.660595 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:08.660125 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4p88d" podUID="d6624d2e-c7a5-44e5-9ace-f88ecb9ff201" Jul 13 09:09:08.660595 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:08.660242 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nsd2z" podUID="a24643eb-b486-4118-9d57-77c6b07ef957" Jul 13 09:09:08.660595 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:08.660300 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wxkd7" podUID="708d0abc-11a7-403e-add0-08c1d93e33f8" Jul 13 09:09:08.798780 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:08.798751 2552 generic.go:356] "Generic (PLEG): container finished" podID="53caa5bd-dca1-4654-a407-0fc895185601" containerID="025f2f9f37759e2156f14d446258e4622d25956bad62fc4df00e022be225988b" exitCode=0 Jul 13 09:09:08.798912 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:08.798802 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bggk8" event={"ID":"53caa5bd-dca1-4654-a407-0fc895185601","Type":"ContainerDied","Data":"025f2f9f37759e2156f14d446258e4622d25956bad62fc4df00e022be225988b"} Jul 13 09:09:08.906757 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:08.906734 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:09:09.000833 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:09.000758 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:09:09.097621 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:09.097594 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:09:10.411871 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:10.411842 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a24643eb-b486-4118-9d57-77c6b07ef957-metrics-certs\") pod \"network-metrics-daemon-nsd2z\" (UID: \"a24643eb-b486-4118-9d57-77c6b07ef957\") " pod="openshift-multus/network-metrics-daemon-nsd2z" Jul 13 09:09:10.412589 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:10.411896 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/708d0abc-11a7-403e-add0-08c1d93e33f8-original-pull-secret\") pod \"global-pull-secret-syncer-wxkd7\" (UID: \"708d0abc-11a7-403e-add0-08c1d93e33f8\") " pod="kube-system/global-pull-secret-syncer-wxkd7" Jul 13 09:09:10.412589 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:10.412002 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 09:09:10.412589 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:10.412062 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/708d0abc-11a7-403e-add0-08c1d93e33f8-original-pull-secret podName:708d0abc-11a7-403e-add0-08c1d93e33f8 nodeName:}" failed. No retries permitted until 2026-07-13 09:09:42.412044878 +0000 UTC m=+66.292000140 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/708d0abc-11a7-403e-add0-08c1d93e33f8-original-pull-secret") pod "global-pull-secret-syncer-wxkd7" (UID: "708d0abc-11a7-403e-add0-08c1d93e33f8") : object "kube-system"/"original-pull-secret" not registered Jul 13 09:09:10.412589 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:10.412003 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 09:09:10.412589 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:10.412161 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a24643eb-b486-4118-9d57-77c6b07ef957-metrics-certs podName:a24643eb-b486-4118-9d57-77c6b07ef957 nodeName:}" failed. No retries permitted until 2026-07-13 09:09:42.412138914 +0000 UTC m=+66.292094160 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a24643eb-b486-4118-9d57-77c6b07ef957-metrics-certs") pod "network-metrics-daemon-nsd2z" (UID: "a24643eb-b486-4118-9d57-77c6b07ef957") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 09:09:10.512439 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:10.512407 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5nnt\" (UniqueName: \"kubernetes.io/projected/d6624d2e-c7a5-44e5-9ace-f88ecb9ff201-kube-api-access-r5nnt\") pod \"network-check-target-4p88d\" (UID: \"d6624d2e-c7a5-44e5-9ace-f88ecb9ff201\") " pod="openshift-network-diagnostics/network-check-target-4p88d" Jul 13 09:09:10.512599 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:10.512581 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 09:09:10.512664 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:10.512605 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 09:09:10.512664 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:10.512619 2552 projected.go:196] Error preparing data for projected volume kube-api-access-r5nnt for pod openshift-network-diagnostics/network-check-target-4p88d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 09:09:10.512749 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:10.512677 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d6624d2e-c7a5-44e5-9ace-f88ecb9ff201-kube-api-access-r5nnt podName:d6624d2e-c7a5-44e5-9ace-f88ecb9ff201 nodeName:}" failed. No retries permitted until 2026-07-13 09:09:42.512658546 +0000 UTC m=+66.392613793 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-r5nnt" (UniqueName: "kubernetes.io/projected/d6624d2e-c7a5-44e5-9ace-f88ecb9ff201-kube-api-access-r5nnt") pod "network-check-target-4p88d" (UID: "d6624d2e-c7a5-44e5-9ace-f88ecb9ff201") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 09:09:10.659753 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:10.659724 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4p88d" Jul 13 09:09:10.659914 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:10.659824 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4p88d" podUID="d6624d2e-c7a5-44e5-9ace-f88ecb9ff201" Jul 13 09:09:10.659914 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:10.659867 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsd2z" Jul 13 09:09:10.660035 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:10.659724 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wxkd7" Jul 13 09:09:10.660035 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:10.659958 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nsd2z" podUID="a24643eb-b486-4118-9d57-77c6b07ef957" Jul 13 09:09:10.660139 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:10.660087 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wxkd7" podUID="708d0abc-11a7-403e-add0-08c1d93e33f8" Jul 13 09:09:11.373841 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:11.373627 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:09:12.414388 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.414353 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeReady" Jul 13 09:09:12.414783 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.414507 2552 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 13 09:09:12.463568 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.463538 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-85jbg"] Jul 13 09:09:12.468523 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.468499 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-m5k9t"] Jul 13 09:09:12.468651 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.468632 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-85jbg" Jul 13 09:09:12.471232 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.471195 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jbvrj\"" Jul 13 09:09:12.471321 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.471256 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 13 09:09:12.471321 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.471196 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 13 09:09:12.472046 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.472027 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-m5k9t" Jul 13 09:09:12.474074 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.474053 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-m5k9t"] Jul 13 09:09:12.474837 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.474699 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 13 09:09:12.474837 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.474719 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-b2l9h\"" Jul 13 09:09:12.474837 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.474795 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 13 09:09:12.475118 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.475096 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 13 09:09:12.476690 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.476667 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-85jbg"] Jul 13 09:09:12.530050 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.529986 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-config-volume\") pod \"dns-default-85jbg\" (UID: \"08c6cfc3-a46b-447e-b470-a2f19a4ddcce\") " pod="openshift-dns/dns-default-85jbg" Jul 13 09:09:12.530050 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.530019 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szsvg\" (UniqueName: \"kubernetes.io/projected/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-kube-api-access-szsvg\") pod \"dns-default-85jbg\" (UID: \"08c6cfc3-a46b-447e-b470-a2f19a4ddcce\") " pod="openshift-dns/dns-default-85jbg" Jul 13 09:09:12.530231 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.530057 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsm5g\" (UniqueName: \"kubernetes.io/projected/1e384dd9-ffe8-45e5-81b1-8caf1f660fc5-kube-api-access-bsm5g\") pod \"ingress-canary-m5k9t\" (UID: \"1e384dd9-ffe8-45e5-81b1-8caf1f660fc5\") " pod="openshift-ingress-canary/ingress-canary-m5k9t" Jul 13 09:09:12.530231 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.530150 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-tmp-dir\") pod \"dns-default-85jbg\" (UID: \"08c6cfc3-a46b-447e-b470-a2f19a4ddcce\") " pod="openshift-dns/dns-default-85jbg" Jul 13 09:09:12.530231 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.530191 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-metrics-tls\") pod \"dns-default-85jbg\" (UID: \"08c6cfc3-a46b-447e-b470-a2f19a4ddcce\") " pod="openshift-dns/dns-default-85jbg" Jul 13 09:09:12.530422 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.530263 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e384dd9-ffe8-45e5-81b1-8caf1f660fc5-cert\") pod \"ingress-canary-m5k9t\" (UID: \"1e384dd9-ffe8-45e5-81b1-8caf1f660fc5\") " pod="openshift-ingress-canary/ingress-canary-m5k9t" Jul 13 09:09:12.630556 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.630526 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-config-volume\") pod \"dns-default-85jbg\" (UID: \"08c6cfc3-a46b-447e-b470-a2f19a4ddcce\") " pod="openshift-dns/dns-default-85jbg" Jul 13 09:09:12.630687 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.630567 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szsvg\" (UniqueName: \"kubernetes.io/projected/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-kube-api-access-szsvg\") pod \"dns-default-85jbg\" (UID: \"08c6cfc3-a46b-447e-b470-a2f19a4ddcce\") " pod="openshift-dns/dns-default-85jbg" Jul 13 09:09:12.630687 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.630610 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsm5g\" (UniqueName: \"kubernetes.io/projected/1e384dd9-ffe8-45e5-81b1-8caf1f660fc5-kube-api-access-bsm5g\") pod \"ingress-canary-m5k9t\" (UID: \"1e384dd9-ffe8-45e5-81b1-8caf1f660fc5\") " pod="openshift-ingress-canary/ingress-canary-m5k9t" Jul 13 09:09:12.630687 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.630662 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-tmp-dir\") pod \"dns-default-85jbg\" (UID: \"08c6cfc3-a46b-447e-b470-a2f19a4ddcce\") " pod="openshift-dns/dns-default-85jbg" Jul 13 09:09:12.630839 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.630686 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-metrics-tls\") pod \"dns-default-85jbg\" (UID: \"08c6cfc3-a46b-447e-b470-a2f19a4ddcce\") " pod="openshift-dns/dns-default-85jbg" Jul 13 09:09:12.630839 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.630726 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e384dd9-ffe8-45e5-81b1-8caf1f660fc5-cert\") pod \"ingress-canary-m5k9t\" (UID: \"1e384dd9-ffe8-45e5-81b1-8caf1f660fc5\") " pod="openshift-ingress-canary/ingress-canary-m5k9t" Jul 13 09:09:12.630938 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:12.630857 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 09:09:12.630987 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:12.630951 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-metrics-tls podName:08c6cfc3-a46b-447e-b470-a2f19a4ddcce nodeName:}" failed. No retries permitted until 2026-07-13 09:09:13.130927885 +0000 UTC m=+37.010883143 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-metrics-tls") pod "dns-default-85jbg" (UID: "08c6cfc3-a46b-447e-b470-a2f19a4ddcce") : secret "dns-default-metrics-tls" not found Jul 13 09:09:12.631078 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.631054 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-tmp-dir\") pod \"dns-default-85jbg\" (UID: \"08c6cfc3-a46b-447e-b470-a2f19a4ddcce\") " pod="openshift-dns/dns-default-85jbg" Jul 13 09:09:12.631320 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.631272 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-config-volume\") pod \"dns-default-85jbg\" (UID: \"08c6cfc3-a46b-447e-b470-a2f19a4ddcce\") " pod="openshift-dns/dns-default-85jbg" Jul 13 09:09:12.635553 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.635531 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e384dd9-ffe8-45e5-81b1-8caf1f660fc5-cert\") pod \"ingress-canary-m5k9t\" (UID: \"1e384dd9-ffe8-45e5-81b1-8caf1f660fc5\") " pod="openshift-ingress-canary/ingress-canary-m5k9t" Jul 13 09:09:12.639366 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.639344 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsm5g\" (UniqueName: \"kubernetes.io/projected/1e384dd9-ffe8-45e5-81b1-8caf1f660fc5-kube-api-access-bsm5g\") pod \"ingress-canary-m5k9t\" (UID: \"1e384dd9-ffe8-45e5-81b1-8caf1f660fc5\") " pod="openshift-ingress-canary/ingress-canary-m5k9t" Jul 13 09:09:12.639480 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.639463 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szsvg\" (UniqueName: \"kubernetes.io/projected/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-kube-api-access-szsvg\") pod \"dns-default-85jbg\" (UID: \"08c6cfc3-a46b-447e-b470-a2f19a4ddcce\") " pod="openshift-dns/dns-default-85jbg" Jul 13 09:09:12.663594 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.663570 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4p88d" Jul 13 09:09:12.663689 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.663580 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wxkd7" Jul 13 09:09:12.663689 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.663682 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsd2z" Jul 13 09:09:12.666199 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.666178 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 13 09:09:12.666332 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.666315 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 13 09:09:12.666459 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.666438 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mkvxw\"" Jul 13 09:09:12.666575 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.666560 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 13 09:09:12.666644 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.666629 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 13 09:09:12.666696 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.666652 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zlq2v\"" Jul 13 09:09:12.787426 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:12.787358 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-m5k9t" Jul 13 09:09:13.104128 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:13.104101 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-m5k9t"] Jul 13 09:09:13.107526 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:09:13.107502 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e384dd9_ffe8_45e5_81b1_8caf1f660fc5.slice/crio-d71411b398f5f14577e6cc8e857297139cbc53075a986dae61f0930088dd42b3 WatchSource:0}: Error finding container d71411b398f5f14577e6cc8e857297139cbc53075a986dae61f0930088dd42b3: Status 404 returned error can't find the container with id d71411b398f5f14577e6cc8e857297139cbc53075a986dae61f0930088dd42b3 Jul 13 09:09:13.133029 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:13.132895 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-metrics-tls\") pod \"dns-default-85jbg\" (UID: \"08c6cfc3-a46b-447e-b470-a2f19a4ddcce\") " pod="openshift-dns/dns-default-85jbg" Jul 13 09:09:13.133141 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:13.133038 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 09:09:13.133141 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:13.133104 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-metrics-tls podName:08c6cfc3-a46b-447e-b470-a2f19a4ddcce nodeName:}" failed. No retries permitted until 2026-07-13 09:09:14.133085091 +0000 UTC m=+38.013040475 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-metrics-tls") pod "dns-default-85jbg" (UID: "08c6cfc3-a46b-447e-b470-a2f19a4ddcce") : secret "dns-default-metrics-tls" not found Jul 13 09:09:13.814364 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:13.814330 2552 generic.go:356] "Generic (PLEG): container finished" podID="53caa5bd-dca1-4654-a407-0fc895185601" containerID="5d4e65f3dd86742b87f59f59cbfa047f0525a543e4f41bf5f9b5079da5248b2d" exitCode=0 Jul 13 09:09:13.814769 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:13.814413 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bggk8" event={"ID":"53caa5bd-dca1-4654-a407-0fc895185601","Type":"ContainerDied","Data":"5d4e65f3dd86742b87f59f59cbfa047f0525a543e4f41bf5f9b5079da5248b2d"} Jul 13 09:09:13.815797 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:13.815688 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-m5k9t" event={"ID":"1e384dd9-ffe8-45e5-81b1-8caf1f660fc5","Type":"ContainerStarted","Data":"d71411b398f5f14577e6cc8e857297139cbc53075a986dae61f0930088dd42b3"} Jul 13 09:09:14.139408 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:14.139372 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-metrics-tls\") pod \"dns-default-85jbg\" (UID: \"08c6cfc3-a46b-447e-b470-a2f19a4ddcce\") " pod="openshift-dns/dns-default-85jbg" Jul 13 09:09:14.139569 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:14.139502 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 09:09:14.139629 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:14.139581 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-metrics-tls podName:08c6cfc3-a46b-447e-b470-a2f19a4ddcce nodeName:}" failed. No retries permitted until 2026-07-13 09:09:16.139560175 +0000 UTC m=+40.019515433 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-metrics-tls") pod "dns-default-85jbg" (UID: "08c6cfc3-a46b-447e-b470-a2f19a4ddcce") : secret "dns-default-metrics-tls" not found Jul 13 09:09:14.821284 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:14.821249 2552 generic.go:356] "Generic (PLEG): container finished" podID="53caa5bd-dca1-4654-a407-0fc895185601" containerID="a474ba7256d8eb587ead53e3c659bd73ce39a115d0cdfb3b688b2685895aafde" exitCode=0 Jul 13 09:09:14.821864 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:14.821311 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bggk8" event={"ID":"53caa5bd-dca1-4654-a407-0fc895185601","Type":"ContainerDied","Data":"a474ba7256d8eb587ead53e3c659bd73ce39a115d0cdfb3b688b2685895aafde"} Jul 13 09:09:15.826278 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:15.826245 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bggk8" event={"ID":"53caa5bd-dca1-4654-a407-0fc895185601","Type":"ContainerStarted","Data":"61bcfef97420b89dbede123278abe8c4bea9a944943dc4baf62ed15e2204df6f"} Jul 13 09:09:15.827769 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:15.827735 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-m5k9t" event={"ID":"1e384dd9-ffe8-45e5-81b1-8caf1f660fc5","Type":"ContainerStarted","Data":"47f6d7ca39fcdc5c03f06d0b2fcc496213074e8878e5ef5c7ea3e1eb546a2546"} Jul 13 09:09:15.851546 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:15.851506 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-bggk8" podStartSLOduration=8.037333336 podStartE2EDuration="38.85149194s" totalImagesPullingTime="30.814158604s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 09:08:37 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:09:15.849786412 +0000 UTC m=+39.729741680" watchObservedRunningTime="2026-07-13 09:09:15.85149194 +0000 UTC m=+39.731447209" Jul 13 09:09:15.867416 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:15.867382 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-m5k9t" podStartSLOduration=1.750624351 podStartE2EDuration="3.86736734s" totalImagesPullingTime="2.116742989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:09:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:09:15.867220557 +0000 UTC m=+39.747175832" watchObservedRunningTime="2026-07-13 09:09:15.86736734 +0000 UTC m=+39.747322608" Jul 13 09:09:16.082458 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:16.082423 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:09:16.142486 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:16.142420 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:09:16.153444 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:16.153424 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-metrics-tls\") pod \"dns-default-85jbg\" (UID: \"08c6cfc3-a46b-447e-b470-a2f19a4ddcce\") " pod="openshift-dns/dns-default-85jbg" Jul 13 09:09:16.153517 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:16.153507 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 09:09:16.153567 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:16.153549 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-metrics-tls podName:08c6cfc3-a46b-447e-b470-a2f19a4ddcce nodeName:}" failed. No retries permitted until 2026-07-13 09:09:20.153537156 +0000 UTC m=+44.033492401 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-metrics-tls") pod "dns-default-85jbg" (UID: "08c6cfc3-a46b-447e-b470-a2f19a4ddcce") : secret "dns-default-metrics-tls" not found Jul 13 09:09:16.624706 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:16.624601 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:09:16.680625 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:16.680600 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:09:17.772187 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:17.772156 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:09:20.178723 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:20.178692 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-metrics-tls\") pod \"dns-default-85jbg\" (UID: \"08c6cfc3-a46b-447e-b470-a2f19a4ddcce\") " pod="openshift-dns/dns-default-85jbg" Jul 13 09:09:20.179083 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:20.178833 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 09:09:20.179083 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:20.178892 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-metrics-tls podName:08c6cfc3-a46b-447e-b470-a2f19a4ddcce nodeName:}" failed. No retries permitted until 2026-07-13 09:09:28.178876797 +0000 UTC m=+52.058832046 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-metrics-tls") pod "dns-default-85jbg" (UID: "08c6cfc3-a46b-447e-b470-a2f19a4ddcce") : secret "dns-default-metrics-tls" not found Jul 13 09:09:21.599783 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:21.599753 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:09:21.652394 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:21.652369 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:09:23.470465 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:23.470427 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:09:23.528352 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:23.528322 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:09:23.580972 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:23.580944 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:09:28.230230 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:28.230187 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-metrics-tls\") pod \"dns-default-85jbg\" (UID: \"08c6cfc3-a46b-447e-b470-a2f19a4ddcce\") " pod="openshift-dns/dns-default-85jbg" Jul 13 09:09:28.230646 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:28.230312 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 09:09:28.230646 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:28.230368 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-metrics-tls podName:08c6cfc3-a46b-447e-b470-a2f19a4ddcce nodeName:}" failed. No retries permitted until 2026-07-13 09:09:44.230352612 +0000 UTC m=+68.110307857 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-metrics-tls") pod "dns-default-85jbg" (UID: "08c6cfc3-a46b-447e-b470-a2f19a4ddcce") : secret "dns-default-metrics-tls" not found Jul 13 09:09:33.826325 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:33.826293 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:09:33.891332 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:33.891302 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:09:36.959595 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:36.959567 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dd667957b-j9grk"] Jul 13 09:09:36.991507 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:36.991486 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dd667957b-j9grk"] Jul 13 09:09:36.991647 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:36.991567 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dd667957b-j9grk" Jul 13 09:09:36.994559 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:36.994528 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 13 09:09:36.995951 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:36.995925 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-jb7xn\"" Jul 13 09:09:36.996195 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:36.995962 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 13 09:09:36.996195 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:36.995966 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 13 09:09:36.996195 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:36.996007 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 13 09:09:37.086245 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:37.086203 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/768513db-9c85-4e82-be90-b33e0ca71c94-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5dd667957b-j9grk\" (UID: \"768513db-9c85-4e82-be90-b33e0ca71c94\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dd667957b-j9grk" Jul 13 09:09:37.086376 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:37.086280 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sq8t\" (UniqueName: \"kubernetes.io/projected/768513db-9c85-4e82-be90-b33e0ca71c94-kube-api-access-9sq8t\") pod \"managed-serviceaccount-addon-agent-5dd667957b-j9grk\" (UID: \"768513db-9c85-4e82-be90-b33e0ca71c94\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dd667957b-j9grk" Jul 13 09:09:37.187625 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:37.187601 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sq8t\" (UniqueName: \"kubernetes.io/projected/768513db-9c85-4e82-be90-b33e0ca71c94-kube-api-access-9sq8t\") pod \"managed-serviceaccount-addon-agent-5dd667957b-j9grk\" (UID: \"768513db-9c85-4e82-be90-b33e0ca71c94\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dd667957b-j9grk" Jul 13 09:09:37.187711 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:37.187674 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/768513db-9c85-4e82-be90-b33e0ca71c94-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5dd667957b-j9grk\" (UID: \"768513db-9c85-4e82-be90-b33e0ca71c94\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dd667957b-j9grk" Jul 13 09:09:37.191105 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:37.191085 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/768513db-9c85-4e82-be90-b33e0ca71c94-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5dd667957b-j9grk\" (UID: \"768513db-9c85-4e82-be90-b33e0ca71c94\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dd667957b-j9grk" Jul 13 09:09:37.196311 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:37.196289 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sq8t\" (UniqueName: \"kubernetes.io/projected/768513db-9c85-4e82-be90-b33e0ca71c94-kube-api-access-9sq8t\") pod \"managed-serviceaccount-addon-agent-5dd667957b-j9grk\" (UID: \"768513db-9c85-4e82-be90-b33e0ca71c94\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dd667957b-j9grk" Jul 13 09:09:37.315847 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:37.315773 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dd667957b-j9grk" Jul 13 09:09:37.453147 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:37.453117 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dd667957b-j9grk"] Jul 13 09:09:37.456890 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:09:37.456866 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod768513db_9c85_4e82_be90_b33e0ca71c94.slice/crio-3c0fa17f110cd488c0ad789ff1be881bf4e3251df1843aadbce3ee1b25ac2053 WatchSource:0}: Error finding container 3c0fa17f110cd488c0ad789ff1be881bf4e3251df1843aadbce3ee1b25ac2053: Status 404 returned error can't find the container with id 3c0fa17f110cd488c0ad789ff1be881bf4e3251df1843aadbce3ee1b25ac2053 Jul 13 09:09:37.808550 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:37.808528 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gp6mv" Jul 13 09:09:37.879642 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:37.879618 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dd667957b-j9grk" event={"ID":"768513db-9c85-4e82-be90-b33e0ca71c94","Type":"ContainerStarted","Data":"3c0fa17f110cd488c0ad789ff1be881bf4e3251df1843aadbce3ee1b25ac2053"} Jul 13 09:09:37.976264 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:37.976239 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:09:38.086391 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:38.086363 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:09:38.151778 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:38.151715 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:09:39.886395 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:39.886362 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dd667957b-j9grk" event={"ID":"768513db-9c85-4e82-be90-b33e0ca71c94","Type":"ContainerStarted","Data":"ff11a8fecdeaf95fda4a224e858924c9c0d7ee68af32bd505d8a58084f4277ba"} Jul 13 09:09:39.902182 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:39.902134 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dd667957b-j9grk" podStartSLOduration=1.6246233079999999 podStartE2EDuration="3.902115678s" totalImagesPullingTime="2.27749237s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:09:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:09:39.901952048 +0000 UTC m=+63.781907316" watchObservedRunningTime="2026-07-13 09:09:39.902115678 +0000 UTC m=+63.782070944" Jul 13 09:09:42.425929 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:42.425886 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a24643eb-b486-4118-9d57-77c6b07ef957-metrics-certs\") pod \"network-metrics-daemon-nsd2z\" (UID: \"a24643eb-b486-4118-9d57-77c6b07ef957\") " pod="openshift-multus/network-metrics-daemon-nsd2z" Jul 13 09:09:42.426359 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:42.425941 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/708d0abc-11a7-403e-add0-08c1d93e33f8-original-pull-secret\") pod \"global-pull-secret-syncer-wxkd7\" (UID: \"708d0abc-11a7-403e-add0-08c1d93e33f8\") " pod="kube-system/global-pull-secret-syncer-wxkd7" Jul 13 09:09:42.428344 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:42.428323 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 13 09:09:42.428440 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:42.428424 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 13 09:09:42.436575 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:42.436557 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 13 09:09:42.436664 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:42.436620 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a24643eb-b486-4118-9d57-77c6b07ef957-metrics-certs podName:a24643eb-b486-4118-9d57-77c6b07ef957 nodeName:}" failed. No retries permitted until 2026-07-13 09:10:46.436604721 +0000 UTC m=+130.316559966 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a24643eb-b486-4118-9d57-77c6b07ef957-metrics-certs") pod "network-metrics-daemon-nsd2z" (UID: "a24643eb-b486-4118-9d57-77c6b07ef957") : secret "metrics-daemon-secret" not found Jul 13 09:09:42.438608 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:42.438591 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/708d0abc-11a7-403e-add0-08c1d93e33f8-original-pull-secret\") pod \"global-pull-secret-syncer-wxkd7\" (UID: \"708d0abc-11a7-403e-add0-08c1d93e33f8\") " pod="kube-system/global-pull-secret-syncer-wxkd7" Jul 13 09:09:42.526700 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:42.526675 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5nnt\" (UniqueName: \"kubernetes.io/projected/d6624d2e-c7a5-44e5-9ace-f88ecb9ff201-kube-api-access-r5nnt\") pod \"network-check-target-4p88d\" (UID: \"d6624d2e-c7a5-44e5-9ace-f88ecb9ff201\") " pod="openshift-network-diagnostics/network-check-target-4p88d" Jul 13 09:09:42.529286 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:42.529269 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 13 09:09:42.539571 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:42.539552 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 13 09:09:42.549800 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:42.549775 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5nnt\" (UniqueName: \"kubernetes.io/projected/d6624d2e-c7a5-44e5-9ace-f88ecb9ff201-kube-api-access-r5nnt\") pod \"network-check-target-4p88d\" (UID: \"d6624d2e-c7a5-44e5-9ace-f88ecb9ff201\") " pod="openshift-network-diagnostics/network-check-target-4p88d" Jul 13 09:09:42.676791 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:42.676743 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mkvxw\"" Jul 13 09:09:42.681175 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:42.681162 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wxkd7" Jul 13 09:09:42.684918 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:42.684905 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4p88d" Jul 13 09:09:42.827137 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:42.827113 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wxkd7"] Jul 13 09:09:42.829649 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:09:42.829625 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod708d0abc_11a7_403e_add0_08c1d93e33f8.slice/crio-35df97ed1712bb4f9da2a9fc2c03d5b2ea9d8f21087ecbe96e986712cd3e7aab WatchSource:0}: Error finding container 35df97ed1712bb4f9da2a9fc2c03d5b2ea9d8f21087ecbe96e986712cd3e7aab: Status 404 returned error can't find the container with id 35df97ed1712bb4f9da2a9fc2c03d5b2ea9d8f21087ecbe96e986712cd3e7aab Jul 13 09:09:42.843318 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:42.843288 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4p88d"] Jul 13 09:09:42.846328 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:09:42.846307 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6624d2e_c7a5_44e5_9ace_f88ecb9ff201.slice/crio-54c2a24531c3c5c27692f4e511fcb8f7f39bd341d6bcc71e7c3e3aec5aeecccf WatchSource:0}: Error finding container 54c2a24531c3c5c27692f4e511fcb8f7f39bd341d6bcc71e7c3e3aec5aeecccf: Status 404 returned error can't find the container with id 54c2a24531c3c5c27692f4e511fcb8f7f39bd341d6bcc71e7c3e3aec5aeecccf Jul 13 09:09:42.894193 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:42.894165 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4p88d" event={"ID":"d6624d2e-c7a5-44e5-9ace-f88ecb9ff201","Type":"ContainerStarted","Data":"54c2a24531c3c5c27692f4e511fcb8f7f39bd341d6bcc71e7c3e3aec5aeecccf"} Jul 13 09:09:42.895245 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:42.895226 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wxkd7" event={"ID":"708d0abc-11a7-403e-add0-08c1d93e33f8","Type":"ContainerStarted","Data":"35df97ed1712bb4f9da2a9fc2c03d5b2ea9d8f21087ecbe96e986712cd3e7aab"} Jul 13 09:09:43.416734 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:43.416697 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:09:43.518414 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:43.518389 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:09:44.238441 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:44.238399 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-metrics-tls\") pod \"dns-default-85jbg\" (UID: \"08c6cfc3-a46b-447e-b470-a2f19a4ddcce\") " pod="openshift-dns/dns-default-85jbg" Jul 13 09:09:44.238627 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:44.238563 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 09:09:44.238688 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:09:44.238642 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-metrics-tls podName:08c6cfc3-a46b-447e-b470-a2f19a4ddcce nodeName:}" failed. No retries permitted until 2026-07-13 09:10:16.238620133 +0000 UTC m=+100.118575401 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-metrics-tls") pod "dns-default-85jbg" (UID: "08c6cfc3-a46b-447e-b470-a2f19a4ddcce") : secret "dns-default-metrics-tls" not found Jul 13 09:09:47.910352 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:47.910274 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4p88d" event={"ID":"d6624d2e-c7a5-44e5-9ace-f88ecb9ff201","Type":"ContainerStarted","Data":"5c43a7d8b651cc063343c62f3c5f4210678a899c0daebb669843aeb5d1ebfb54"} Jul 13 09:09:47.910735 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:47.910394 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-4p88d" Jul 13 09:09:47.911743 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:47.911721 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wxkd7" event={"ID":"708d0abc-11a7-403e-add0-08c1d93e33f8","Type":"ContainerStarted","Data":"b72fbdb26de5a7cd7f0506f74f716aad8a34daa2a0eaf560ea4822f00996f382"} Jul 13 09:09:47.926829 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:47.926789 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-4p88d" podStartSLOduration=66.172398906 podStartE2EDuration="1m10.926777676s" totalImagesPullingTime="4.75437877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:08:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:09:47.926108308 +0000 UTC m=+71.806063578" watchObservedRunningTime="2026-07-13 09:09:47.926777676 +0000 UTC m=+71.806732920" Jul 13 09:09:47.941152 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:47.941115 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-wxkd7" podStartSLOduration=66.165613565 podStartE2EDuration="1m10.941102429s" totalImagesPullingTime="4.775488864s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 09:08:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:09:47.940773501 +0000 UTC m=+71.820728780" watchObservedRunningTime="2026-07-13 09:09:47.941102429 +0000 UTC m=+71.821057674" Jul 13 09:09:48.943140 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:48.943104 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:09:49.325470 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:09:49.325403 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:10:09.839588 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:10:09.839554 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:10:16.335421 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:10:16.335386 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-metrics-tls\") pod \"dns-default-85jbg\" (UID: \"08c6cfc3-a46b-447e-b470-a2f19a4ddcce\") " pod="openshift-dns/dns-default-85jbg" Jul 13 09:10:16.335826 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:10:16.335515 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 09:10:16.335826 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:10:16.335567 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-metrics-tls podName:08c6cfc3-a46b-447e-b470-a2f19a4ddcce nodeName:}" failed. No retries permitted until 2026-07-13 09:11:20.335552821 +0000 UTC m=+164.215508066 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-metrics-tls") pod "dns-default-85jbg" (UID: "08c6cfc3-a46b-447e-b470-a2f19a4ddcce") : secret "dns-default-metrics-tls" not found Jul 13 09:10:18.915772 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:10:18.915738 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-4p88d" Jul 13 09:10:46.524746 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:10:46.524698 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a24643eb-b486-4118-9d57-77c6b07ef957-metrics-certs\") pod \"network-metrics-daemon-nsd2z\" (UID: \"a24643eb-b486-4118-9d57-77c6b07ef957\") " pod="openshift-multus/network-metrics-daemon-nsd2z" Jul 13 09:10:46.525307 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:10:46.524855 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 13 09:10:46.525307 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:10:46.524943 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a24643eb-b486-4118-9d57-77c6b07ef957-metrics-certs podName:a24643eb-b486-4118-9d57-77c6b07ef957 nodeName:}" failed. No retries permitted until 2026-07-13 09:12:48.524923556 +0000 UTC m=+252.404878801 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a24643eb-b486-4118-9d57-77c6b07ef957-metrics-certs") pod "network-metrics-daemon-nsd2z" (UID: "a24643eb-b486-4118-9d57-77c6b07ef957") : secret "metrics-daemon-secret" not found Jul 13 09:11:15.482402 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:11:15.482356 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-85jbg" podUID="08c6cfc3-a46b-447e-b470-a2f19a4ddcce" Jul 13 09:11:15.686436 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:11:15.686407 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-nsd2z" podUID="a24643eb-b486-4118-9d57-77c6b07ef957" Jul 13 09:11:16.121696 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:11:16.121670 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-85jbg" Jul 13 09:11:20.344282 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:11:20.344242 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-metrics-tls\") pod \"dns-default-85jbg\" (UID: \"08c6cfc3-a46b-447e-b470-a2f19a4ddcce\") " pod="openshift-dns/dns-default-85jbg" Jul 13 09:11:20.344754 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:11:20.344367 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 09:11:20.344754 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:11:20.344419 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-metrics-tls podName:08c6cfc3-a46b-447e-b470-a2f19a4ddcce nodeName:}" failed. No retries permitted until 2026-07-13 09:13:22.344404744 +0000 UTC m=+286.224359989 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-metrics-tls") pod "dns-default-85jbg" (UID: "08c6cfc3-a46b-447e-b470-a2f19a4ddcce") : secret "dns-default-metrics-tls" not found Jul 13 09:11:28.660638 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:11:28.660601 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsd2z" Jul 13 09:11:40.180908 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:11:40.180806 2552 generic.go:356] "Generic (PLEG): container finished" podID="768513db-9c85-4e82-be90-b33e0ca71c94" containerID="ff11a8fecdeaf95fda4a224e858924c9c0d7ee68af32bd505d8a58084f4277ba" exitCode=255 Jul 13 09:11:40.180908 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:11:40.180888 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dd667957b-j9grk" event={"ID":"768513db-9c85-4e82-be90-b33e0ca71c94","Type":"ContainerDied","Data":"ff11a8fecdeaf95fda4a224e858924c9c0d7ee68af32bd505d8a58084f4277ba"} Jul 13 09:11:40.186555 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:11:40.186536 2552 scope.go:117] "RemoveContainer" containerID="ff11a8fecdeaf95fda4a224e858924c9c0d7ee68af32bd505d8a58084f4277ba" Jul 13 09:11:41.185479 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:11:41.185443 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dd667957b-j9grk" event={"ID":"768513db-9c85-4e82-be90-b33e0ca71c94","Type":"ContainerStarted","Data":"4fc565df5251dec8b492f5a1ef4ebd6d6744eb1c6a5a343efda0e42c2baba10d"} Jul 13 09:12:14.454918 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:14.454888 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:38.554930 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.554897 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-77b8c6656d-wxkr8"] Jul 13 09:12:38.557631 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.557615 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-77b8c6656d-wxkr8" Jul 13 09:12:38.563743 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.563459 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 13 09:12:38.564976 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.564956 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 13 09:12:38.565072 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.564982 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 13 09:12:38.565072 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.564959 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 13 09:12:38.565180 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.565102 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-7nznk\"" Jul 13 09:12:38.565476 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.565462 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 13 09:12:38.565830 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.565816 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 13 09:12:38.592106 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.592079 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-77b8c6656d-wxkr8"] Jul 13 09:12:38.640983 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.640960 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-f9wzl"] Jul 13 09:12:38.643585 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.643570 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-f9wzl" Jul 13 09:12:38.645284 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.645265 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-68c5fcfc4f-v2f5q"] Jul 13 09:12:38.646524 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.646505 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9a3e094-4012-4054-83f7-9e753c83e8ad-service-ca-bundle\") pod \"router-default-77b8c6656d-wxkr8\" (UID: \"b9a3e094-4012-4054-83f7-9e753c83e8ad\") " pod="openshift-ingress/router-default-77b8c6656d-wxkr8" Jul 13 09:12:38.646619 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.646539 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djmnd\" (UniqueName: \"kubernetes.io/projected/b9a3e094-4012-4054-83f7-9e753c83e8ad-kube-api-access-djmnd\") pod \"router-default-77b8c6656d-wxkr8\" (UID: \"b9a3e094-4012-4054-83f7-9e753c83e8ad\") " pod="openshift-ingress/router-default-77b8c6656d-wxkr8" Jul 13 09:12:38.646619 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.646568 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9a3e094-4012-4054-83f7-9e753c83e8ad-metrics-certs\") pod \"router-default-77b8c6656d-wxkr8\" (UID: \"b9a3e094-4012-4054-83f7-9e753c83e8ad\") " pod="openshift-ingress/router-default-77b8c6656d-wxkr8" Jul 13 09:12:38.646619 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.646587 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-5rhp4\"" Jul 13 09:12:38.646619 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.646588 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b9a3e094-4012-4054-83f7-9e753c83e8ad-stats-auth\") pod \"router-default-77b8c6656d-wxkr8\" (UID: \"b9a3e094-4012-4054-83f7-9e753c83e8ad\") " pod="openshift-ingress/router-default-77b8c6656d-wxkr8" Jul 13 09:12:38.646818 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.646656 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b9a3e094-4012-4054-83f7-9e753c83e8ad-default-certificate\") pod \"router-default-77b8c6656d-wxkr8\" (UID: \"b9a3e094-4012-4054-83f7-9e753c83e8ad\") " pod="openshift-ingress/router-default-77b8c6656d-wxkr8" Jul 13 09:12:38.647883 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.647869 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" Jul 13 09:12:38.651785 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.651770 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 13 09:12:38.652274 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.652258 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qppg4\"" Jul 13 09:12:38.653290 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.653271 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 13 09:12:38.653398 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.653379 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 13 09:12:38.663328 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.663312 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-f9wzl"] Jul 13 09:12:38.663534 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.663517 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 13 09:12:38.680109 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.680085 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-68c5fcfc4f-v2f5q"] Jul 13 09:12:38.747472 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.747450 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-trusted-ca\") pod \"image-registry-68c5fcfc4f-v2f5q\" (UID: \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\") " pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" Jul 13 09:12:38.747566 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.747481 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-installation-pull-secrets\") pod \"image-registry-68c5fcfc4f-v2f5q\" (UID: \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\") " pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" Jul 13 09:12:38.747566 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.747508 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgspk\" (UniqueName: \"kubernetes.io/projected/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-kube-api-access-zgspk\") pod \"image-registry-68c5fcfc4f-v2f5q\" (UID: \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\") " pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" Jul 13 09:12:38.747566 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.747557 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djmnd\" (UniqueName: \"kubernetes.io/projected/b9a3e094-4012-4054-83f7-9e753c83e8ad-kube-api-access-djmnd\") pod \"router-default-77b8c6656d-wxkr8\" (UID: \"b9a3e094-4012-4054-83f7-9e753c83e8ad\") " pod="openshift-ingress/router-default-77b8c6656d-wxkr8" Jul 13 09:12:38.747668 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.747573 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-bound-sa-token\") pod \"image-registry-68c5fcfc4f-v2f5q\" (UID: \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\") " pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" Jul 13 09:12:38.747668 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.747591 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-registry-tls\") pod \"image-registry-68c5fcfc4f-v2f5q\" (UID: \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\") " pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" Jul 13 09:12:38.747668 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.747626 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-ca-trust-extracted\") pod \"image-registry-68c5fcfc4f-v2f5q\" (UID: \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\") " pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" Jul 13 09:12:38.747668 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.747647 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9a3e094-4012-4054-83f7-9e753c83e8ad-metrics-certs\") pod \"router-default-77b8c6656d-wxkr8\" (UID: \"b9a3e094-4012-4054-83f7-9e753c83e8ad\") " pod="openshift-ingress/router-default-77b8c6656d-wxkr8" Jul 13 09:12:38.747815 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.747669 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b9a3e094-4012-4054-83f7-9e753c83e8ad-stats-auth\") pod \"router-default-77b8c6656d-wxkr8\" (UID: \"b9a3e094-4012-4054-83f7-9e753c83e8ad\") " pod="openshift-ingress/router-default-77b8c6656d-wxkr8" Jul 13 09:12:38.747815 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.747692 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-registry-certificates\") pod \"image-registry-68c5fcfc4f-v2f5q\" (UID: \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\") " pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" Jul 13 09:12:38.747815 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.747721 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-image-registry-private-configuration\") pod \"image-registry-68c5fcfc4f-v2f5q\" (UID: \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\") " pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" Jul 13 09:12:38.747815 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.747751 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b9a3e094-4012-4054-83f7-9e753c83e8ad-default-certificate\") pod \"router-default-77b8c6656d-wxkr8\" (UID: \"b9a3e094-4012-4054-83f7-9e753c83e8ad\") " pod="openshift-ingress/router-default-77b8c6656d-wxkr8" Jul 13 09:12:38.747815 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.747778 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hdz4\" (UniqueName: \"kubernetes.io/projected/f4759f8f-dd89-4de2-884d-304717840654-kube-api-access-9hdz4\") pod \"network-check-source-869dcfcdcd-f9wzl\" (UID: \"f4759f8f-dd89-4de2-884d-304717840654\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-f9wzl" Jul 13 09:12:38.747815 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:38.747754 2552 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 09:12:38.748082 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:38.747859 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9a3e094-4012-4054-83f7-9e753c83e8ad-metrics-certs podName:b9a3e094-4012-4054-83f7-9e753c83e8ad nodeName:}" failed. No retries permitted until 2026-07-13 09:12:39.247840374 +0000 UTC m=+243.127795620 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b9a3e094-4012-4054-83f7-9e753c83e8ad-metrics-certs") pod "router-default-77b8c6656d-wxkr8" (UID: "b9a3e094-4012-4054-83f7-9e753c83e8ad") : secret "router-metrics-certs-default" not found Jul 13 09:12:38.748082 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.747880 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9a3e094-4012-4054-83f7-9e753c83e8ad-service-ca-bundle\") pod \"router-default-77b8c6656d-wxkr8\" (UID: \"b9a3e094-4012-4054-83f7-9e753c83e8ad\") " pod="openshift-ingress/router-default-77b8c6656d-wxkr8" Jul 13 09:12:38.748082 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:38.747977 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b9a3e094-4012-4054-83f7-9e753c83e8ad-service-ca-bundle podName:b9a3e094-4012-4054-83f7-9e753c83e8ad nodeName:}" failed. No retries permitted until 2026-07-13 09:12:39.247969153 +0000 UTC m=+243.127924410 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b9a3e094-4012-4054-83f7-9e753c83e8ad-service-ca-bundle") pod "router-default-77b8c6656d-wxkr8" (UID: "b9a3e094-4012-4054-83f7-9e753c83e8ad") : configmap references non-existent config key: service-ca.crt Jul 13 09:12:38.750020 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.750001 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b9a3e094-4012-4054-83f7-9e753c83e8ad-stats-auth\") pod \"router-default-77b8c6656d-wxkr8\" (UID: \"b9a3e094-4012-4054-83f7-9e753c83e8ad\") " pod="openshift-ingress/router-default-77b8c6656d-wxkr8" Jul 13 09:12:38.750157 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.750136 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b9a3e094-4012-4054-83f7-9e753c83e8ad-default-certificate\") pod \"router-default-77b8c6656d-wxkr8\" (UID: \"b9a3e094-4012-4054-83f7-9e753c83e8ad\") " pod="openshift-ingress/router-default-77b8c6656d-wxkr8" Jul 13 09:12:38.768322 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.768299 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djmnd\" (UniqueName: \"kubernetes.io/projected/b9a3e094-4012-4054-83f7-9e753c83e8ad-kube-api-access-djmnd\") pod \"router-default-77b8c6656d-wxkr8\" (UID: \"b9a3e094-4012-4054-83f7-9e753c83e8ad\") " pod="openshift-ingress/router-default-77b8c6656d-wxkr8" Jul 13 09:12:38.849008 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.848976 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-trusted-ca\") pod \"image-registry-68c5fcfc4f-v2f5q\" (UID: \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\") " pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" Jul 13 09:12:38.849008 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.849011 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-installation-pull-secrets\") pod \"image-registry-68c5fcfc4f-v2f5q\" (UID: \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\") " pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" Jul 13 09:12:38.849188 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.849029 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgspk\" (UniqueName: \"kubernetes.io/projected/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-kube-api-access-zgspk\") pod \"image-registry-68c5fcfc4f-v2f5q\" (UID: \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\") " pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" Jul 13 09:12:38.849188 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.849051 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-bound-sa-token\") pod \"image-registry-68c5fcfc4f-v2f5q\" (UID: \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\") " pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" Jul 13 09:12:38.849188 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.849168 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-registry-tls\") pod \"image-registry-68c5fcfc4f-v2f5q\" (UID: \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\") " pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" Jul 13 09:12:38.849378 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.849235 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-ca-trust-extracted\") pod \"image-registry-68c5fcfc4f-v2f5q\" (UID: \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\") " pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" Jul 13 09:12:38.849378 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:38.849292 2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 09:12:38.849378 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:38.849308 2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68c5fcfc4f-v2f5q: secret "image-registry-tls" not found Jul 13 09:12:38.849378 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:38.849369 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-registry-tls podName:c0a4a73b-5bd3-4ff2-bd69-8a68bc640539 nodeName:}" failed. No retries permitted until 2026-07-13 09:12:39.349353491 +0000 UTC m=+243.229308736 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-registry-tls") pod "image-registry-68c5fcfc4f-v2f5q" (UID: "c0a4a73b-5bd3-4ff2-bd69-8a68bc640539") : secret "image-registry-tls" not found Jul 13 09:12:38.849584 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.849291 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-registry-certificates\") pod \"image-registry-68c5fcfc4f-v2f5q\" (UID: \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\") " pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" Jul 13 09:12:38.849584 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.849434 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-image-registry-private-configuration\") pod \"image-registry-68c5fcfc4f-v2f5q\" (UID: \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\") " pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" Jul 13 09:12:38.849584 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.849472 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hdz4\" (UniqueName: \"kubernetes.io/projected/f4759f8f-dd89-4de2-884d-304717840654-kube-api-access-9hdz4\") pod \"network-check-source-869dcfcdcd-f9wzl\" (UID: \"f4759f8f-dd89-4de2-884d-304717840654\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-f9wzl" Jul 13 09:12:38.849733 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.849608 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-ca-trust-extracted\") pod \"image-registry-68c5fcfc4f-v2f5q\" (UID: \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\") " pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" Jul 13 09:12:38.849786 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.849752 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-registry-certificates\") pod \"image-registry-68c5fcfc4f-v2f5q\" (UID: \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\") " pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" Jul 13 09:12:38.850438 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.850405 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-trusted-ca\") pod \"image-registry-68c5fcfc4f-v2f5q\" (UID: \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\") " pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" Jul 13 09:12:38.851450 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.851426 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-installation-pull-secrets\") pod \"image-registry-68c5fcfc4f-v2f5q\" (UID: \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\") " pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" Jul 13 09:12:38.854947 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.854924 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-image-registry-private-configuration\") pod \"image-registry-68c5fcfc4f-v2f5q\" (UID: \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\") " pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" Jul 13 09:12:38.858607 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.858581 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hdz4\" (UniqueName: \"kubernetes.io/projected/f4759f8f-dd89-4de2-884d-304717840654-kube-api-access-9hdz4\") pod \"network-check-source-869dcfcdcd-f9wzl\" (UID: \"f4759f8f-dd89-4de2-884d-304717840654\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-f9wzl" Jul 13 09:12:38.859142 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.859119 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-bound-sa-token\") pod \"image-registry-68c5fcfc4f-v2f5q\" (UID: \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\") " pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" Jul 13 09:12:38.859270 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.859254 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgspk\" (UniqueName: \"kubernetes.io/projected/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-kube-api-access-zgspk\") pod \"image-registry-68c5fcfc4f-v2f5q\" (UID: \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\") " pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" Jul 13 09:12:38.953549 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:38.953525 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-f9wzl" Jul 13 09:12:39.071435 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:39.071408 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-f9wzl"] Jul 13 09:12:39.073829 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:12:39.073803 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4759f8f_dd89_4de2_884d_304717840654.slice/crio-22d26e2c474c6ca498d830e16a4ea1263c089be6a198a3d1cb0a37200c165195 WatchSource:0}: Error finding container 22d26e2c474c6ca498d830e16a4ea1263c089be6a198a3d1cb0a37200c165195: Status 404 returned error can't find the container with id 22d26e2c474c6ca498d830e16a4ea1263c089be6a198a3d1cb0a37200c165195 Jul 13 09:12:39.254017 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:39.253987 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9a3e094-4012-4054-83f7-9e753c83e8ad-metrics-certs\") pod \"router-default-77b8c6656d-wxkr8\" (UID: \"b9a3e094-4012-4054-83f7-9e753c83e8ad\") " pod="openshift-ingress/router-default-77b8c6656d-wxkr8" Jul 13 09:12:39.254157 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:39.254113 2552 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 09:12:39.254157 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:39.254142 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9a3e094-4012-4054-83f7-9e753c83e8ad-service-ca-bundle\") pod \"router-default-77b8c6656d-wxkr8\" (UID: \"b9a3e094-4012-4054-83f7-9e753c83e8ad\") " pod="openshift-ingress/router-default-77b8c6656d-wxkr8" Jul 13 09:12:39.254314 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:39.254162 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9a3e094-4012-4054-83f7-9e753c83e8ad-metrics-certs podName:b9a3e094-4012-4054-83f7-9e753c83e8ad nodeName:}" failed. No retries permitted until 2026-07-13 09:12:40.254148081 +0000 UTC m=+244.134103325 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b9a3e094-4012-4054-83f7-9e753c83e8ad-metrics-certs") pod "router-default-77b8c6656d-wxkr8" (UID: "b9a3e094-4012-4054-83f7-9e753c83e8ad") : secret "router-metrics-certs-default" not found Jul 13 09:12:39.254314 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:39.254245 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b9a3e094-4012-4054-83f7-9e753c83e8ad-service-ca-bundle podName:b9a3e094-4012-4054-83f7-9e753c83e8ad nodeName:}" failed. No retries permitted until 2026-07-13 09:12:40.254228146 +0000 UTC m=+244.134183392 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b9a3e094-4012-4054-83f7-9e753c83e8ad-service-ca-bundle") pod "router-default-77b8c6656d-wxkr8" (UID: "b9a3e094-4012-4054-83f7-9e753c83e8ad") : configmap references non-existent config key: service-ca.crt Jul 13 09:12:39.294327 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:39.294304 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:39.355010 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:39.354988 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-registry-tls\") pod \"image-registry-68c5fcfc4f-v2f5q\" (UID: \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\") " pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" Jul 13 09:12:39.355110 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:39.355097 2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 09:12:39.355174 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:39.355112 2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68c5fcfc4f-v2f5q: secret "image-registry-tls" not found Jul 13 09:12:39.355174 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:39.355162 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-registry-tls podName:c0a4a73b-5bd3-4ff2-bd69-8a68bc640539 nodeName:}" failed. No retries permitted until 2026-07-13 09:12:40.355147311 +0000 UTC m=+244.235102559 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-registry-tls") pod "image-registry-68c5fcfc4f-v2f5q" (UID: "c0a4a73b-5bd3-4ff2-bd69-8a68bc640539") : secret "image-registry-tls" not found Jul 13 09:12:39.356948 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:39.356888 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-f9wzl" event={"ID":"f4759f8f-dd89-4de2-884d-304717840654","Type":"ContainerStarted","Data":"3f17cbc51dc016712463279ef40ca46b158c63d41371deaadca1413d93902d6d"} Jul 13 09:12:39.356948 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:39.356937 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-f9wzl" event={"ID":"f4759f8f-dd89-4de2-884d-304717840654","Type":"ContainerStarted","Data":"22d26e2c474c6ca498d830e16a4ea1263c089be6a198a3d1cb0a37200c165195"} Jul 13 09:12:39.378717 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:39.378673 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-f9wzl" podStartSLOduration=1.37865751 podStartE2EDuration="1.37865751s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:12:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:12:39.378012265 +0000 UTC m=+243.257967533" watchObservedRunningTime="2026-07-13 09:12:39.37865751 +0000 UTC m=+243.258612778" Jul 13 09:12:39.383687 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:39.383663 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:40.260054 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:40.260019 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9a3e094-4012-4054-83f7-9e753c83e8ad-metrics-certs\") pod \"router-default-77b8c6656d-wxkr8\" (UID: \"b9a3e094-4012-4054-83f7-9e753c83e8ad\") " pod="openshift-ingress/router-default-77b8c6656d-wxkr8" Jul 13 09:12:40.260427 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:40.260073 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9a3e094-4012-4054-83f7-9e753c83e8ad-service-ca-bundle\") pod \"router-default-77b8c6656d-wxkr8\" (UID: \"b9a3e094-4012-4054-83f7-9e753c83e8ad\") " pod="openshift-ingress/router-default-77b8c6656d-wxkr8" Jul 13 09:12:40.260427 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:40.260156 2552 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 09:12:40.260427 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:40.260187 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b9a3e094-4012-4054-83f7-9e753c83e8ad-service-ca-bundle podName:b9a3e094-4012-4054-83f7-9e753c83e8ad nodeName:}" failed. No retries permitted until 2026-07-13 09:12:42.26017443 +0000 UTC m=+246.140129679 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b9a3e094-4012-4054-83f7-9e753c83e8ad-service-ca-bundle") pod "router-default-77b8c6656d-wxkr8" (UID: "b9a3e094-4012-4054-83f7-9e753c83e8ad") : configmap references non-existent config key: service-ca.crt Jul 13 09:12:40.260427 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:40.260205 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9a3e094-4012-4054-83f7-9e753c83e8ad-metrics-certs podName:b9a3e094-4012-4054-83f7-9e753c83e8ad nodeName:}" failed. No retries permitted until 2026-07-13 09:12:42.260193408 +0000 UTC m=+246.140148653 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b9a3e094-4012-4054-83f7-9e753c83e8ad-metrics-certs") pod "router-default-77b8c6656d-wxkr8" (UID: "b9a3e094-4012-4054-83f7-9e753c83e8ad") : secret "router-metrics-certs-default" not found Jul 13 09:12:40.360514 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:40.360485 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-registry-tls\") pod \"image-registry-68c5fcfc4f-v2f5q\" (UID: \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\") " pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" Jul 13 09:12:40.360626 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:40.360602 2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 09:12:40.360626 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:40.360615 2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68c5fcfc4f-v2f5q: secret "image-registry-tls" not found Jul 13 09:12:40.360704 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:40.360657 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-registry-tls podName:c0a4a73b-5bd3-4ff2-bd69-8a68bc640539 nodeName:}" failed. No retries permitted until 2026-07-13 09:12:42.360645623 +0000 UTC m=+246.240600868 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-registry-tls") pod "image-registry-68c5fcfc4f-v2f5q" (UID: "c0a4a73b-5bd3-4ff2-bd69-8a68bc640539") : secret "image-registry-tls" not found Jul 13 09:12:41.497818 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:41.497789 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:41.600123 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:41.600085 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:42.276614 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:42.276582 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9a3e094-4012-4054-83f7-9e753c83e8ad-metrics-certs\") pod \"router-default-77b8c6656d-wxkr8\" (UID: \"b9a3e094-4012-4054-83f7-9e753c83e8ad\") " pod="openshift-ingress/router-default-77b8c6656d-wxkr8" Jul 13 09:12:42.276785 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:42.276631 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9a3e094-4012-4054-83f7-9e753c83e8ad-service-ca-bundle\") pod \"router-default-77b8c6656d-wxkr8\" (UID: \"b9a3e094-4012-4054-83f7-9e753c83e8ad\") " pod="openshift-ingress/router-default-77b8c6656d-wxkr8" Jul 13 09:12:42.276785 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:42.276717 2552 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 09:12:42.276785 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:42.276729 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b9a3e094-4012-4054-83f7-9e753c83e8ad-service-ca-bundle podName:b9a3e094-4012-4054-83f7-9e753c83e8ad nodeName:}" failed. No retries permitted until 2026-07-13 09:12:46.276715479 +0000 UTC m=+250.156670727 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b9a3e094-4012-4054-83f7-9e753c83e8ad-service-ca-bundle") pod "router-default-77b8c6656d-wxkr8" (UID: "b9a3e094-4012-4054-83f7-9e753c83e8ad") : configmap references non-existent config key: service-ca.crt Jul 13 09:12:42.276785 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:42.276768 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9a3e094-4012-4054-83f7-9e753c83e8ad-metrics-certs podName:b9a3e094-4012-4054-83f7-9e753c83e8ad nodeName:}" failed. No retries permitted until 2026-07-13 09:12:46.276753404 +0000 UTC m=+250.156708650 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b9a3e094-4012-4054-83f7-9e753c83e8ad-metrics-certs") pod "router-default-77b8c6656d-wxkr8" (UID: "b9a3e094-4012-4054-83f7-9e753c83e8ad") : secret "router-metrics-certs-default" not found Jul 13 09:12:42.376991 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:42.376969 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-registry-tls\") pod \"image-registry-68c5fcfc4f-v2f5q\" (UID: \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\") " pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" Jul 13 09:12:42.377121 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:42.377081 2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 09:12:42.377121 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:42.377094 2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68c5fcfc4f-v2f5q: secret "image-registry-tls" not found Jul 13 09:12:42.377186 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:42.377138 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-registry-tls podName:c0a4a73b-5bd3-4ff2-bd69-8a68bc640539 nodeName:}" failed. No retries permitted until 2026-07-13 09:12:46.377125389 +0000 UTC m=+250.257080657 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-registry-tls") pod "image-registry-68c5fcfc4f-v2f5q" (UID: "c0a4a73b-5bd3-4ff2-bd69-8a68bc640539") : secret "image-registry-tls" not found Jul 13 09:12:43.734527 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:43.734493 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:43.846885 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:43.846855 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:43.911479 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:43.911446 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-nllgk"] Jul 13 09:12:43.914627 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:43.914605 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nllgk" Jul 13 09:12:43.919114 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:43.919090 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 13 09:12:43.919227 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:43.919176 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 13 09:12:43.919534 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:43.919520 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 13 09:12:43.919577 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:43.919546 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kg5m9\"" Jul 13 09:12:43.919948 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:43.919928 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 13 09:12:43.936131 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:43.936103 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nllgk"] Jul 13 09:12:43.964803 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:43.964786 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:43.987127 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:43.987081 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c913a302-9335-4e64-bffc-22ce094e431b-crio-socket\") pod \"insights-runtime-extractor-nllgk\" (UID: \"c913a302-9335-4e64-bffc-22ce094e431b\") " pod="openshift-insights/insights-runtime-extractor-nllgk" Jul 13 09:12:43.987127 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:43.987120 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sz9rs\" (UniqueName: \"kubernetes.io/projected/c913a302-9335-4e64-bffc-22ce094e431b-kube-api-access-sz9rs\") pod \"insights-runtime-extractor-nllgk\" (UID: \"c913a302-9335-4e64-bffc-22ce094e431b\") " pod="openshift-insights/insights-runtime-extractor-nllgk" Jul 13 09:12:43.987250 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:43.987145 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c913a302-9335-4e64-bffc-22ce094e431b-data-volume\") pod \"insights-runtime-extractor-nllgk\" (UID: \"c913a302-9335-4e64-bffc-22ce094e431b\") " pod="openshift-insights/insights-runtime-extractor-nllgk" Jul 13 09:12:43.987250 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:43.987170 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c913a302-9335-4e64-bffc-22ce094e431b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nllgk\" (UID: \"c913a302-9335-4e64-bffc-22ce094e431b\") " pod="openshift-insights/insights-runtime-extractor-nllgk" Jul 13 09:12:43.987250 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:43.987191 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c913a302-9335-4e64-bffc-22ce094e431b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nllgk\" (UID: \"c913a302-9335-4e64-bffc-22ce094e431b\") " pod="openshift-insights/insights-runtime-extractor-nllgk" Jul 13 09:12:44.063907 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:44.063879 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:44.087887 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:44.087868 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sz9rs\" (UniqueName: \"kubernetes.io/projected/c913a302-9335-4e64-bffc-22ce094e431b-kube-api-access-sz9rs\") pod \"insights-runtime-extractor-nllgk\" (UID: \"c913a302-9335-4e64-bffc-22ce094e431b\") " pod="openshift-insights/insights-runtime-extractor-nllgk" Jul 13 09:12:44.087965 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:44.087902 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c913a302-9335-4e64-bffc-22ce094e431b-data-volume\") pod \"insights-runtime-extractor-nllgk\" (UID: \"c913a302-9335-4e64-bffc-22ce094e431b\") " pod="openshift-insights/insights-runtime-extractor-nllgk" Jul 13 09:12:44.087965 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:44.087923 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c913a302-9335-4e64-bffc-22ce094e431b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nllgk\" (UID: \"c913a302-9335-4e64-bffc-22ce094e431b\") " pod="openshift-insights/insights-runtime-extractor-nllgk" Jul 13 09:12:44.088073 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:44.087983 2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 09:12:44.088073 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:44.088038 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c913a302-9335-4e64-bffc-22ce094e431b-kube-rbac-proxy-cm podName:c913a302-9335-4e64-bffc-22ce094e431b nodeName:}" failed. No retries permitted until 2026-07-13 09:12:44.588019968 +0000 UTC m=+248.467975230 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c913a302-9335-4e64-bffc-22ce094e431b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nllgk" (UID: "c913a302-9335-4e64-bffc-22ce094e431b") : configmap "kube-rbac-proxy" not found Jul 13 09:12:44.088073 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:44.088053 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c913a302-9335-4e64-bffc-22ce094e431b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nllgk\" (UID: \"c913a302-9335-4e64-bffc-22ce094e431b\") " pod="openshift-insights/insights-runtime-extractor-nllgk" Jul 13 09:12:44.088247 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:44.088120 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c913a302-9335-4e64-bffc-22ce094e431b-crio-socket\") pod \"insights-runtime-extractor-nllgk\" (UID: \"c913a302-9335-4e64-bffc-22ce094e431b\") " pod="openshift-insights/insights-runtime-extractor-nllgk" Jul 13 09:12:44.088247 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:44.088199 2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 09:12:44.088247 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:44.088226 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c913a302-9335-4e64-bffc-22ce094e431b-crio-socket\") pod \"insights-runtime-extractor-nllgk\" (UID: \"c913a302-9335-4e64-bffc-22ce094e431b\") " pod="openshift-insights/insights-runtime-extractor-nllgk" Jul 13 09:12:44.088348 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:44.088270 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c913a302-9335-4e64-bffc-22ce094e431b-data-volume\") pod \"insights-runtime-extractor-nllgk\" (UID: \"c913a302-9335-4e64-bffc-22ce094e431b\") " pod="openshift-insights/insights-runtime-extractor-nllgk" Jul 13 09:12:44.088348 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:44.088273 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c913a302-9335-4e64-bffc-22ce094e431b-insights-runtime-extractor-tls podName:c913a302-9335-4e64-bffc-22ce094e431b nodeName:}" failed. No retries permitted until 2026-07-13 09:12:44.58825774 +0000 UTC m=+248.468212992 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c913a302-9335-4e64-bffc-22ce094e431b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nllgk" (UID: "c913a302-9335-4e64-bffc-22ce094e431b") : secret "insights-runtime-extractor-tls" not found Jul 13 09:12:44.096348 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:44.096331 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sz9rs\" (UniqueName: \"kubernetes.io/projected/c913a302-9335-4e64-bffc-22ce094e431b-kube-api-access-sz9rs\") pod \"insights-runtime-extractor-nllgk\" (UID: \"c913a302-9335-4e64-bffc-22ce094e431b\") " pod="openshift-insights/insights-runtime-extractor-nllgk" Jul 13 09:12:44.164091 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:44.164069 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:44.243649 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:44.243601 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:44.424555 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:44.424536 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:44.522744 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:44.522684 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:44.592585 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:44.592548 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c913a302-9335-4e64-bffc-22ce094e431b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nllgk\" (UID: \"c913a302-9335-4e64-bffc-22ce094e431b\") " pod="openshift-insights/insights-runtime-extractor-nllgk" Jul 13 09:12:44.592700 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:44.592684 2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 09:12:44.592745 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:44.592702 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c913a302-9335-4e64-bffc-22ce094e431b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nllgk\" (UID: \"c913a302-9335-4e64-bffc-22ce094e431b\") " pod="openshift-insights/insights-runtime-extractor-nllgk" Jul 13 09:12:44.592745 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:44.592742 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c913a302-9335-4e64-bffc-22ce094e431b-insights-runtime-extractor-tls podName:c913a302-9335-4e64-bffc-22ce094e431b nodeName:}" failed. No retries permitted until 2026-07-13 09:12:45.592728679 +0000 UTC m=+249.472683923 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c913a302-9335-4e64-bffc-22ce094e431b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nllgk" (UID: "c913a302-9335-4e64-bffc-22ce094e431b") : secret "insights-runtime-extractor-tls" not found Jul 13 09:12:44.592830 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:44.592769 2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 09:12:44.592830 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:44.592811 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c913a302-9335-4e64-bffc-22ce094e431b-kube-rbac-proxy-cm podName:c913a302-9335-4e64-bffc-22ce094e431b nodeName:}" failed. No retries permitted until 2026-07-13 09:12:45.592800927 +0000 UTC m=+249.472756172 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c913a302-9335-4e64-bffc-22ce094e431b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nllgk" (UID: "c913a302-9335-4e64-bffc-22ce094e431b") : configmap "kube-rbac-proxy" not found Jul 13 09:12:44.867957 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:44.867934 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:44.946285 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:44.946260 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:45.599064 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:45.599028 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c913a302-9335-4e64-bffc-22ce094e431b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nllgk\" (UID: \"c913a302-9335-4e64-bffc-22ce094e431b\") " pod="openshift-insights/insights-runtime-extractor-nllgk" Jul 13 09:12:45.599064 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:45.599072 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c913a302-9335-4e64-bffc-22ce094e431b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nllgk\" (UID: \"c913a302-9335-4e64-bffc-22ce094e431b\") " pod="openshift-insights/insights-runtime-extractor-nllgk" Jul 13 09:12:45.599395 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:45.599151 2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 09:12:45.599395 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:45.599206 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c913a302-9335-4e64-bffc-22ce094e431b-kube-rbac-proxy-cm podName:c913a302-9335-4e64-bffc-22ce094e431b nodeName:}" failed. No retries permitted until 2026-07-13 09:12:47.599191937 +0000 UTC m=+251.479147181 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c913a302-9335-4e64-bffc-22ce094e431b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nllgk" (UID: "c913a302-9335-4e64-bffc-22ce094e431b") : configmap "kube-rbac-proxy" not found Jul 13 09:12:45.599395 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:45.599248 2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 09:12:45.599395 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:45.599301 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c913a302-9335-4e64-bffc-22ce094e431b-insights-runtime-extractor-tls podName:c913a302-9335-4e64-bffc-22ce094e431b nodeName:}" failed. No retries permitted until 2026-07-13 09:12:47.599287052 +0000 UTC m=+251.479242319 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c913a302-9335-4e64-bffc-22ce094e431b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nllgk" (UID: "c913a302-9335-4e64-bffc-22ce094e431b") : secret "insights-runtime-extractor-tls" not found Jul 13 09:12:45.890629 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:45.890570 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-d9vsl_1a1d3bed-1a9d-42ce-b6b5-454d4aaac7da/dns-node-resolver/0.log" Jul 13 09:12:46.023225 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:46.023195 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-dbksx"] Jul 13 09:12:46.026337 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:46.026321 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-dbksx" Jul 13 09:12:46.030098 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:46.030074 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 13 09:12:46.031338 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:46.031233 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-g7q5v\"" Jul 13 09:12:46.031338 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:46.031311 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 13 09:12:46.031521 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:46.031308 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 13 09:12:46.031521 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:46.031343 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 13 09:12:46.037836 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:46.037816 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-dbksx"] Jul 13 09:12:46.063340 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:46.063290 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:46.104090 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:46.104070 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0b18b2f3-cbf9-4100-b6d2-494654332363-signing-key\") pod \"service-ca-79f68b9f4f-dbksx\" (UID: \"0b18b2f3-cbf9-4100-b6d2-494654332363\") " pod="openshift-service-ca/service-ca-79f68b9f4f-dbksx" Jul 13 09:12:46.104153 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:46.104101 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkms9\" (UniqueName: \"kubernetes.io/projected/0b18b2f3-cbf9-4100-b6d2-494654332363-kube-api-access-bkms9\") pod \"service-ca-79f68b9f4f-dbksx\" (UID: \"0b18b2f3-cbf9-4100-b6d2-494654332363\") " pod="openshift-service-ca/service-ca-79f68b9f4f-dbksx" Jul 13 09:12:46.104244 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:46.104225 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0b18b2f3-cbf9-4100-b6d2-494654332363-signing-cabundle\") pod \"service-ca-79f68b9f4f-dbksx\" (UID: \"0b18b2f3-cbf9-4100-b6d2-494654332363\") " pod="openshift-service-ca/service-ca-79f68b9f4f-dbksx" Jul 13 09:12:46.138911 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:46.138892 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:46.205310 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:46.205266 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0b18b2f3-cbf9-4100-b6d2-494654332363-signing-cabundle\") pod \"service-ca-79f68b9f4f-dbksx\" (UID: \"0b18b2f3-cbf9-4100-b6d2-494654332363\") " pod="openshift-service-ca/service-ca-79f68b9f4f-dbksx" Jul 13 09:12:46.205369 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:46.205312 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0b18b2f3-cbf9-4100-b6d2-494654332363-signing-key\") pod \"service-ca-79f68b9f4f-dbksx\" (UID: \"0b18b2f3-cbf9-4100-b6d2-494654332363\") " pod="openshift-service-ca/service-ca-79f68b9f4f-dbksx" Jul 13 09:12:46.205444 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:46.205425 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkms9\" (UniqueName: \"kubernetes.io/projected/0b18b2f3-cbf9-4100-b6d2-494654332363-kube-api-access-bkms9\") pod \"service-ca-79f68b9f4f-dbksx\" (UID: \"0b18b2f3-cbf9-4100-b6d2-494654332363\") " pod="openshift-service-ca/service-ca-79f68b9f4f-dbksx" Jul 13 09:12:46.205954 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:46.205930 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0b18b2f3-cbf9-4100-b6d2-494654332363-signing-cabundle\") pod \"service-ca-79f68b9f4f-dbksx\" (UID: \"0b18b2f3-cbf9-4100-b6d2-494654332363\") " pod="openshift-service-ca/service-ca-79f68b9f4f-dbksx" Jul 13 09:12:46.207536 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:46.207516 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0b18b2f3-cbf9-4100-b6d2-494654332363-signing-key\") pod \"service-ca-79f68b9f4f-dbksx\" (UID: \"0b18b2f3-cbf9-4100-b6d2-494654332363\") " pod="openshift-service-ca/service-ca-79f68b9f4f-dbksx" Jul 13 09:12:46.216734 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:46.216709 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkms9\" (UniqueName: \"kubernetes.io/projected/0b18b2f3-cbf9-4100-b6d2-494654332363-kube-api-access-bkms9\") pod \"service-ca-79f68b9f4f-dbksx\" (UID: \"0b18b2f3-cbf9-4100-b6d2-494654332363\") " pod="openshift-service-ca/service-ca-79f68b9f4f-dbksx" Jul 13 09:12:46.306277 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:46.306253 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9a3e094-4012-4054-83f7-9e753c83e8ad-service-ca-bundle\") pod \"router-default-77b8c6656d-wxkr8\" (UID: \"b9a3e094-4012-4054-83f7-9e753c83e8ad\") " pod="openshift-ingress/router-default-77b8c6656d-wxkr8" Jul 13 09:12:46.306352 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:46.306311 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9a3e094-4012-4054-83f7-9e753c83e8ad-metrics-certs\") pod \"router-default-77b8c6656d-wxkr8\" (UID: \"b9a3e094-4012-4054-83f7-9e753c83e8ad\") " pod="openshift-ingress/router-default-77b8c6656d-wxkr8" Jul 13 09:12:46.306390 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:46.306380 2552 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 09:12:46.306425 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:46.306401 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b9a3e094-4012-4054-83f7-9e753c83e8ad-service-ca-bundle podName:b9a3e094-4012-4054-83f7-9e753c83e8ad nodeName:}" failed. No retries permitted until 2026-07-13 09:12:54.306382347 +0000 UTC m=+258.186337620 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b9a3e094-4012-4054-83f7-9e753c83e8ad-service-ca-bundle") pod "router-default-77b8c6656d-wxkr8" (UID: "b9a3e094-4012-4054-83f7-9e753c83e8ad") : configmap references non-existent config key: service-ca.crt Jul 13 09:12:46.306425 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:46.306421 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9a3e094-4012-4054-83f7-9e753c83e8ad-metrics-certs podName:b9a3e094-4012-4054-83f7-9e753c83e8ad nodeName:}" failed. No retries permitted until 2026-07-13 09:12:54.306415217 +0000 UTC m=+258.186370462 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b9a3e094-4012-4054-83f7-9e753c83e8ad-metrics-certs") pod "router-default-77b8c6656d-wxkr8" (UID: "b9a3e094-4012-4054-83f7-9e753c83e8ad") : secret "router-metrics-certs-default" not found Jul 13 09:12:46.337486 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:46.337460 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-dbksx" Jul 13 09:12:46.407326 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:46.407300 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-registry-tls\") pod \"image-registry-68c5fcfc4f-v2f5q\" (UID: \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\") " pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" Jul 13 09:12:46.407451 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:46.407434 2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 09:12:46.407493 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:46.407453 2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68c5fcfc4f-v2f5q: secret "image-registry-tls" not found Jul 13 09:12:46.407526 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:46.407501 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-registry-tls podName:c0a4a73b-5bd3-4ff2-bd69-8a68bc640539 nodeName:}" failed. No retries permitted until 2026-07-13 09:12:54.407485686 +0000 UTC m=+258.287440952 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-registry-tls") pod "image-registry-68c5fcfc4f-v2f5q" (UID: "c0a4a73b-5bd3-4ff2-bd69-8a68bc640539") : secret "image-registry-tls" not found Jul 13 09:12:46.456799 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:46.456746 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-dbksx"] Jul 13 09:12:46.459480 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:12:46.459454 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b18b2f3_cbf9_4100_b6d2_494654332363.slice/crio-ca00fb77ee6b7d327120e2904553e4269248a3c6bb9bbc33f9b86178bc31ab4a WatchSource:0}: Error finding container ca00fb77ee6b7d327120e2904553e4269248a3c6bb9bbc33f9b86178bc31ab4a: Status 404 returned error can't find the container with id ca00fb77ee6b7d327120e2904553e4269248a3c6bb9bbc33f9b86178bc31ab4a Jul 13 09:12:47.090707 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:47.090680 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-s8xqc_777385c6-f18f-4560-902e-67b4b1885fcd/node-ca/0.log" Jul 13 09:12:47.381316 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:47.381237 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-dbksx" event={"ID":"0b18b2f3-cbf9-4100-b6d2-494654332363","Type":"ContainerStarted","Data":"ca00fb77ee6b7d327120e2904553e4269248a3c6bb9bbc33f9b86178bc31ab4a"} Jul 13 09:12:47.423495 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:47.423466 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:47.616331 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:47.616301 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c913a302-9335-4e64-bffc-22ce094e431b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nllgk\" (UID: \"c913a302-9335-4e64-bffc-22ce094e431b\") " pod="openshift-insights/insights-runtime-extractor-nllgk" Jul 13 09:12:47.616478 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:47.616341 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c913a302-9335-4e64-bffc-22ce094e431b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nllgk\" (UID: \"c913a302-9335-4e64-bffc-22ce094e431b\") " pod="openshift-insights/insights-runtime-extractor-nllgk" Jul 13 09:12:47.616478 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:47.616423 2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 09:12:47.616478 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:47.616444 2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 09:12:47.616610 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:47.616487 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c913a302-9335-4e64-bffc-22ce094e431b-kube-rbac-proxy-cm podName:c913a302-9335-4e64-bffc-22ce094e431b nodeName:}" failed. No retries permitted until 2026-07-13 09:12:51.616470894 +0000 UTC m=+255.496426144 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c913a302-9335-4e64-bffc-22ce094e431b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nllgk" (UID: "c913a302-9335-4e64-bffc-22ce094e431b") : configmap "kube-rbac-proxy" not found Jul 13 09:12:47.616610 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:47.616506 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c913a302-9335-4e64-bffc-22ce094e431b-insights-runtime-extractor-tls podName:c913a302-9335-4e64-bffc-22ce094e431b nodeName:}" failed. No retries permitted until 2026-07-13 09:12:51.616497022 +0000 UTC m=+255.496452269 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c913a302-9335-4e64-bffc-22ce094e431b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nllgk" (UID: "c913a302-9335-4e64-bffc-22ce094e431b") : secret "insights-runtime-extractor-tls" not found Jul 13 09:12:47.890838 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:47.890809 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-m5k9t_1e384dd9-ffe8-45e5-81b1-8caf1f660fc5/serve-healthcheck-canary/0.log" Jul 13 09:12:48.385998 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:48.385968 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-dbksx" event={"ID":"0b18b2f3-cbf9-4100-b6d2-494654332363","Type":"ContainerStarted","Data":"26a7686f2d4d319ffaa7e2481e2a60db52d9d5906e4502b12ef46441aa5f54bc"} Jul 13 09:12:48.407098 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:48.407016 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-dbksx" podStartSLOduration=0.759288064 podStartE2EDuration="2.407002735s" totalImagesPullingTime="1.647714671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:12:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:12:48.405990784 +0000 UTC m=+252.285946052" watchObservedRunningTime="2026-07-13 09:12:48.407002735 +0000 UTC m=+252.286958001" Jul 13 09:12:48.624137 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:48.624107 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a24643eb-b486-4118-9d57-77c6b07ef957-metrics-certs\") pod \"network-metrics-daemon-nsd2z\" (UID: \"a24643eb-b486-4118-9d57-77c6b07ef957\") " pod="openshift-multus/network-metrics-daemon-nsd2z" Jul 13 09:12:48.624278 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:48.624251 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 13 09:12:48.624346 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:48.624315 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a24643eb-b486-4118-9d57-77c6b07ef957-metrics-certs podName:a24643eb-b486-4118-9d57-77c6b07ef957 nodeName:}" failed. No retries permitted until 2026-07-13 09:14:50.624295074 +0000 UTC m=+374.504250330 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a24643eb-b486-4118-9d57-77c6b07ef957-metrics-certs") pod "network-metrics-daemon-nsd2z" (UID: "a24643eb-b486-4118-9d57-77c6b07ef957") : secret "metrics-daemon-secret" not found Jul 13 09:12:49.536673 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:49.536645 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:49.775940 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:49.775897 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:50.028471 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:50.028429 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:50.110590 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:50.110561 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:50.288540 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:50.288450 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:50.389346 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:50.389323 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:50.484271 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:50.484243 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:51.247047 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:51.247017 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:51.648655 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:51.648624 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c913a302-9335-4e64-bffc-22ce094e431b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nllgk\" (UID: \"c913a302-9335-4e64-bffc-22ce094e431b\") " pod="openshift-insights/insights-runtime-extractor-nllgk" Jul 13 09:12:51.648844 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:51.648664 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c913a302-9335-4e64-bffc-22ce094e431b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nllgk\" (UID: \"c913a302-9335-4e64-bffc-22ce094e431b\") " pod="openshift-insights/insights-runtime-extractor-nllgk" Jul 13 09:12:51.648844 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:51.648692 2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 09:12:51.648844 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:51.648707 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:51.648844 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:51.648742 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c913a302-9335-4e64-bffc-22ce094e431b-kube-rbac-proxy-cm podName:c913a302-9335-4e64-bffc-22ce094e431b nodeName:}" failed. No retries permitted until 2026-07-13 09:12:59.648727638 +0000 UTC m=+263.528682883 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c913a302-9335-4e64-bffc-22ce094e431b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nllgk" (UID: "c913a302-9335-4e64-bffc-22ce094e431b") : configmap "kube-rbac-proxy" not found Jul 13 09:12:51.648844 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:51.648802 2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 09:12:51.648844 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:51.648844 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c913a302-9335-4e64-bffc-22ce094e431b-insights-runtime-extractor-tls podName:c913a302-9335-4e64-bffc-22ce094e431b nodeName:}" failed. No retries permitted until 2026-07-13 09:12:59.648833563 +0000 UTC m=+263.528788809 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c913a302-9335-4e64-bffc-22ce094e431b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nllgk" (UID: "c913a302-9335-4e64-bffc-22ce094e431b") : secret "insights-runtime-extractor-tls" not found Jul 13 09:12:51.738650 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:51.738628 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:51.847877 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:51.847854 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:52.432896 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:52.432864 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:52.830859 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:52.830831 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:53.245931 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:53.245875 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:53.635393 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:53.635368 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:53.700016 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:53.699993 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:54.037402 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:54.037341 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:54.369283 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:54.369256 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9a3e094-4012-4054-83f7-9e753c83e8ad-service-ca-bundle\") pod \"router-default-77b8c6656d-wxkr8\" (UID: \"b9a3e094-4012-4054-83f7-9e753c83e8ad\") " pod="openshift-ingress/router-default-77b8c6656d-wxkr8" Jul 13 09:12:54.369422 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:54.369314 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9a3e094-4012-4054-83f7-9e753c83e8ad-metrics-certs\") pod \"router-default-77b8c6656d-wxkr8\" (UID: \"b9a3e094-4012-4054-83f7-9e753c83e8ad\") " pod="openshift-ingress/router-default-77b8c6656d-wxkr8" Jul 13 09:12:54.369422 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:12:54.369412 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b9a3e094-4012-4054-83f7-9e753c83e8ad-service-ca-bundle podName:b9a3e094-4012-4054-83f7-9e753c83e8ad nodeName:}" failed. No retries permitted until 2026-07-13 09:13:10.369394987 +0000 UTC m=+274.249350232 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b9a3e094-4012-4054-83f7-9e753c83e8ad-service-ca-bundle") pod "router-default-77b8c6656d-wxkr8" (UID: "b9a3e094-4012-4054-83f7-9e753c83e8ad") : configmap references non-existent config key: service-ca.crt Jul 13 09:12:54.371529 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:54.371505 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9a3e094-4012-4054-83f7-9e753c83e8ad-metrics-certs\") pod \"router-default-77b8c6656d-wxkr8\" (UID: \"b9a3e094-4012-4054-83f7-9e753c83e8ad\") " pod="openshift-ingress/router-default-77b8c6656d-wxkr8" Jul 13 09:12:54.435166 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:54.435143 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:54.469700 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:54.469675 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-registry-tls\") pod \"image-registry-68c5fcfc4f-v2f5q\" (UID: \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\") " pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" Jul 13 09:12:54.471708 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:54.471685 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-registry-tls\") pod \"image-registry-68c5fcfc4f-v2f5q\" (UID: \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\") " pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" Jul 13 09:12:54.558358 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:54.558340 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" Jul 13 09:12:54.653383 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:54.653357 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:54.699424 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:54.699395 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-68c5fcfc4f-v2f5q"] Jul 13 09:12:54.703837 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:12:54.703811 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0a4a73b_5bd3_4ff2_bd69_8a68bc640539.slice/crio-f919abd462f81eaa25983f83d30470e322dd3f48ff86edf2055221cdc7079c11 WatchSource:0}: Error finding container f919abd462f81eaa25983f83d30470e322dd3f48ff86edf2055221cdc7079c11: Status 404 returned error can't find the container with id f919abd462f81eaa25983f83d30470e322dd3f48ff86edf2055221cdc7079c11 Jul 13 09:12:54.835982 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:54.835952 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:55.407638 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:55.407609 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" event={"ID":"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539","Type":"ContainerStarted","Data":"9f7dfce5cd9c22f18db3116028b974dbb03aa5a48c8b90661dfb6493c6cf2e86"} Jul 13 09:12:55.407817 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:55.407653 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" event={"ID":"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539","Type":"ContainerStarted","Data":"f919abd462f81eaa25983f83d30470e322dd3f48ff86edf2055221cdc7079c11"} Jul 13 09:12:55.407817 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:55.407681 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" Jul 13 09:12:55.429001 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:55.428913 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" podStartSLOduration=17.428899137 podStartE2EDuration="17.428899137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:12:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:12:55.427999611 +0000 UTC m=+259.307954890" watchObservedRunningTime="2026-07-13 09:12:55.428899137 +0000 UTC m=+259.308854403" Jul 13 09:12:55.837292 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:55.837265 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:57.114979 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:57.114948 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:57.646466 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:57.646424 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:58.231580 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:58.231553 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:58.430019 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:58.429986 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:58.501392 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:58.501334 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:58.637957 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:58.637937 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:58.830383 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:58.830362 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:59.707486 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:59.707451 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c913a302-9335-4e64-bffc-22ce094e431b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nllgk\" (UID: \"c913a302-9335-4e64-bffc-22ce094e431b\") " pod="openshift-insights/insights-runtime-extractor-nllgk" Jul 13 09:12:59.707875 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:59.707494 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c913a302-9335-4e64-bffc-22ce094e431b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nllgk\" (UID: \"c913a302-9335-4e64-bffc-22ce094e431b\") " pod="openshift-insights/insights-runtime-extractor-nllgk" Jul 13 09:12:59.707961 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:59.707939 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c913a302-9335-4e64-bffc-22ce094e431b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nllgk\" (UID: \"c913a302-9335-4e64-bffc-22ce094e431b\") " pod="openshift-insights/insights-runtime-extractor-nllgk" Jul 13 09:12:59.709783 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:59.709761 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c913a302-9335-4e64-bffc-22ce094e431b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nllgk\" (UID: \"c913a302-9335-4e64-bffc-22ce094e431b\") " pod="openshift-insights/insights-runtime-extractor-nllgk" Jul 13 09:12:59.824564 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:59.824537 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nllgk" Jul 13 09:12:59.870472 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:59.870441 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:12:59.948328 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:12:59.948304 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nllgk"] Jul 13 09:12:59.953268 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:12:59.953241 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc913a302_9335_4e64_bffc_22ce094e431b.slice/crio-63709ffaa215ca7ce131e0314031e69abf040c656f7caf200756996ce2658d99 WatchSource:0}: Error finding container 63709ffaa215ca7ce131e0314031e69abf040c656f7caf200756996ce2658d99: Status 404 returned error can't find the container with id 63709ffaa215ca7ce131e0314031e69abf040c656f7caf200756996ce2658d99 Jul 13 09:13:00.033160 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:00.033109 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:00.239300 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:00.239272 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:00.428582 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:00.428548 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nllgk" event={"ID":"c913a302-9335-4e64-bffc-22ce094e431b","Type":"ContainerStarted","Data":"63709ffaa215ca7ce131e0314031e69abf040c656f7caf200756996ce2658d99"} Jul 13 09:13:00.428997 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:00.428941 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:01.528490 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:01.528456 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:01.629629 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:01.629596 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:01.703347 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:01.703313 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:01.827313 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:01.827263 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:02.196259 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:02.196157 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:02.246968 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:02.246934 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:02.600472 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:02.600446 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:02.658671 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:02.658643 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:02.995810 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:02.995694 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:03.063762 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:03.063723 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:03.213512 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:03.213479 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:03.933166 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:03.933097 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:04.228687 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:04.228620 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:04.329095 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:04.329071 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:04.441476 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:04.441452 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nllgk" event={"ID":"c913a302-9335-4e64-bffc-22ce094e431b","Type":"ContainerStarted","Data":"9b23eb339a58ed65648fd80f44e559ed23ddfc9fac85666422946f6b8bf350de"} Jul 13 09:13:04.441568 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:04.441500 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nllgk" event={"ID":"c913a302-9335-4e64-bffc-22ce094e431b","Type":"ContainerStarted","Data":"d8e96d71233206f08725bc4785c32993394993b46555a0c8f8a2415739d6cb18"} Jul 13 09:13:06.597882 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:06.597851 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:06.702817 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:06.702796 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:07.456121 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:07.456089 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nllgk" event={"ID":"c913a302-9335-4e64-bffc-22ce094e431b","Type":"ContainerStarted","Data":"559c8e5779a8bd089f72b50728f071a5bda331b694c8b11097d4ccb9d529b827"} Jul 13 09:13:07.474219 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:07.474174 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-nllgk" podStartSLOduration=17.917722037 podStartE2EDuration="24.47415791s" totalImagesPullingTime="6.556435873s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 09:12:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:13:07.473227462 +0000 UTC m=+271.353182721" watchObservedRunningTime="2026-07-13 09:13:07.47415791 +0000 UTC m=+271.354113176" Jul 13 09:13:09.475757 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:09.475725 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-68c5fcfc4f-v2f5q"] Jul 13 09:13:09.480022 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:09.479994 2552 patch_prober.go:28] interesting pod/image-registry-68c5fcfc4f-v2f5q container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 13 09:13:09.480151 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:09.480043 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" podUID="c0a4a73b-5bd3-4ff2-bd69-8a68bc640539" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 09:13:09.630947 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:09.630906 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:09.732485 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:09.732424 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:10.389613 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:10.389583 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9a3e094-4012-4054-83f7-9e753c83e8ad-service-ca-bundle\") pod \"router-default-77b8c6656d-wxkr8\" (UID: \"b9a3e094-4012-4054-83f7-9e753c83e8ad\") " pod="openshift-ingress/router-default-77b8c6656d-wxkr8" Jul 13 09:13:10.390145 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:10.390126 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9a3e094-4012-4054-83f7-9e753c83e8ad-service-ca-bundle\") pod \"router-default-77b8c6656d-wxkr8\" (UID: \"b9a3e094-4012-4054-83f7-9e753c83e8ad\") " pod="openshift-ingress/router-default-77b8c6656d-wxkr8" Jul 13 09:13:10.666409 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:10.666349 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-77b8c6656d-wxkr8" Jul 13 09:13:10.790077 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:10.790047 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-77b8c6656d-wxkr8"] Jul 13 09:13:10.792704 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:13:10.792678 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9a3e094_4012_4054_83f7_9e753c83e8ad.slice/crio-db808fbeb8844a11032c37bd46c1a4fdfb32959f851f8525c1b82269e81b4925 WatchSource:0}: Error finding container db808fbeb8844a11032c37bd46c1a4fdfb32959f851f8525c1b82269e81b4925: Status 404 returned error can't find the container with id db808fbeb8844a11032c37bd46c1a4fdfb32959f851f8525c1b82269e81b4925 Jul 13 09:13:11.469978 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:11.469940 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-77b8c6656d-wxkr8" event={"ID":"b9a3e094-4012-4054-83f7-9e753c83e8ad","Type":"ContainerStarted","Data":"6549f6dea6185cb8a2798092e7966e050114985cca3aa08443c4064f52d36a8c"} Jul 13 09:13:11.470127 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:11.470021 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-77b8c6656d-wxkr8" event={"ID":"b9a3e094-4012-4054-83f7-9e753c83e8ad","Type":"ContainerStarted","Data":"db808fbeb8844a11032c37bd46c1a4fdfb32959f851f8525c1b82269e81b4925"} Jul 13 09:13:11.491604 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:11.491561 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-77b8c6656d-wxkr8" podStartSLOduration=33.491549089 podStartE2EDuration="33.491549089s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:12:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:13:11.490996695 +0000 UTC m=+275.370951963" watchObservedRunningTime="2026-07-13 09:13:11.491549089 +0000 UTC m=+275.371504356" Jul 13 09:13:11.667360 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:11.667334 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-77b8c6656d-wxkr8" Jul 13 09:13:11.669690 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:11.669672 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-77b8c6656d-wxkr8" Jul 13 09:13:12.472936 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:12.472906 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-77b8c6656d-wxkr8" Jul 13 09:13:12.474184 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:12.474160 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-77b8c6656d-wxkr8" Jul 13 09:13:13.182696 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:13.182661 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:13.286300 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:13.286269 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:13.341708 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:13.341684 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bnqz9"] Jul 13 09:13:13.344914 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:13.344898 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bnqz9" Jul 13 09:13:13.347615 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:13.347595 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 13 09:13:13.347774 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:13.347751 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-bzf87\"" Jul 13 09:13:13.354895 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:13.354868 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bnqz9"] Jul 13 09:13:13.373770 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:13.373742 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:13.511146 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:13.511091 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c02bf867-897b-4144-bcf6-de76c08b030d-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-bnqz9\" (UID: \"c02bf867-897b-4144-bcf6-de76c08b030d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bnqz9" Jul 13 09:13:13.611419 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:13.611393 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c02bf867-897b-4144-bcf6-de76c08b030d-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-bnqz9\" (UID: \"c02bf867-897b-4144-bcf6-de76c08b030d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bnqz9" Jul 13 09:13:13.611572 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:13:13.611548 2552 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 13 09:13:13.611657 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:13:13.611643 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c02bf867-897b-4144-bcf6-de76c08b030d-tls-certificates podName:c02bf867-897b-4144-bcf6-de76c08b030d nodeName:}" failed. No retries permitted until 2026-07-13 09:13:14.111622632 +0000 UTC m=+277.991577878 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/c02bf867-897b-4144-bcf6-de76c08b030d-tls-certificates") pod "prometheus-operator-admission-webhook-77d98bc786-bnqz9" (UID: "c02bf867-897b-4144-bcf6-de76c08b030d") : secret "prometheus-operator-admission-webhook-tls" not found Jul 13 09:13:13.953843 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:13.953818 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:14.114773 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:14.114739 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c02bf867-897b-4144-bcf6-de76c08b030d-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-bnqz9\" (UID: \"c02bf867-897b-4144-bcf6-de76c08b030d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bnqz9" Jul 13 09:13:14.117040 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:14.117023 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c02bf867-897b-4144-bcf6-de76c08b030d-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-bnqz9\" (UID: \"c02bf867-897b-4144-bcf6-de76c08b030d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bnqz9" Jul 13 09:13:14.254071 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:14.254008 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bnqz9" Jul 13 09:13:14.374894 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:14.374866 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bnqz9"] Jul 13 09:13:14.377655 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:13:14.377632 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc02bf867_897b_4144_bcf6_de76c08b030d.slice/crio-bb52e52beb7dd790a37cd2193e293b96e2c4dbb401a24e24f374a6a33952a7eb WatchSource:0}: Error finding container bb52e52beb7dd790a37cd2193e293b96e2c4dbb401a24e24f374a6a33952a7eb: Status 404 returned error can't find the container with id bb52e52beb7dd790a37cd2193e293b96e2c4dbb401a24e24f374a6a33952a7eb Jul 13 09:13:14.479766 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:14.479736 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bnqz9" event={"ID":"c02bf867-897b-4144-bcf6-de76c08b030d","Type":"ContainerStarted","Data":"bb52e52beb7dd790a37cd2193e293b96e2c4dbb401a24e24f374a6a33952a7eb"} Jul 13 09:13:16.494740 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:16.494692 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bnqz9" event={"ID":"c02bf867-897b-4144-bcf6-de76c08b030d","Type":"ContainerStarted","Data":"c554fa1eb292c41fab82d7925a2ee7aefae50af1b6de0f5eb996d03e141448c4"} Jul 13 09:13:16.495203 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:16.494941 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bnqz9" Jul 13 09:13:16.499848 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:16.499823 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bnqz9" Jul 13 09:13:16.511673 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:16.511628 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bnqz9" podStartSLOduration=2.458206177 podStartE2EDuration="3.51161146s" totalImagesPullingTime="1.053405283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:13:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:13:16.510795271 +0000 UTC m=+280.390750537" watchObservedRunningTime="2026-07-13 09:13:16.51161146 +0000 UTC m=+280.391566729" Jul 13 09:13:17.398405 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.398375 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:17.423865 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.423839 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-crsd7"] Jul 13 09:13:17.426907 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.426892 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-crsd7" Jul 13 09:13:17.429530 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.429507 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 13 09:13:17.429661 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.429510 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 13 09:13:17.430437 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.430417 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-92cj2\"" Jul 13 09:13:17.430539 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.430521 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 13 09:13:17.430588 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.430550 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 13 09:13:17.430632 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.430598 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 13 09:13:17.439779 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.439716 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-crsd7"] Jul 13 09:13:17.450248 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.450229 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7787895976-9pm2h"] Jul 13 09:13:17.453006 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.452992 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7787895976-9pm2h" Jul 13 09:13:17.455594 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.455566 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 13 09:13:17.455730 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.455707 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4gxx5\"" Jul 13 09:13:17.455844 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.455829 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 13 09:13:17.455972 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.455954 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 13 09:13:17.456035 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.455980 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 13 09:13:17.456035 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.456011 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 13 09:13:17.456133 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.456101 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 13 09:13:17.456264 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.456201 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 13 09:13:17.464961 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.464942 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7787895976-9pm2h"] Jul 13 09:13:17.484934 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.484914 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:17.533713 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.533695 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86ebac58-f50b-4e17-8153-b9e3bfbed54e-metrics-client-ca\") pod \"prometheus-operator-64785b6675-crsd7\" (UID: \"86ebac58-f50b-4e17-8153-b9e3bfbed54e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-crsd7" Jul 13 09:13:17.534012 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.533735 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/86ebac58-f50b-4e17-8153-b9e3bfbed54e-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-crsd7\" (UID: \"86ebac58-f50b-4e17-8153-b9e3bfbed54e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-crsd7" Jul 13 09:13:17.534012 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.533761 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/86ebac58-f50b-4e17-8153-b9e3bfbed54e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-crsd7\" (UID: \"86ebac58-f50b-4e17-8153-b9e3bfbed54e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-crsd7" Jul 13 09:13:17.534012 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.533805 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pc2x\" (UniqueName: \"kubernetes.io/projected/86ebac58-f50b-4e17-8153-b9e3bfbed54e-kube-api-access-9pc2x\") pod \"prometheus-operator-64785b6675-crsd7\" (UID: \"86ebac58-f50b-4e17-8153-b9e3bfbed54e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-crsd7" Jul 13 09:13:17.634823 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.634791 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/86ebac58-f50b-4e17-8153-b9e3bfbed54e-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-crsd7\" (UID: \"86ebac58-f50b-4e17-8153-b9e3bfbed54e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-crsd7" Jul 13 09:13:17.634823 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.634829 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5f8d3e22-5af7-4150-9f56-65e55d2e6713-console-config\") pod \"console-7787895976-9pm2h\" (UID: \"5f8d3e22-5af7-4150-9f56-65e55d2e6713\") " pod="openshift-console/console-7787895976-9pm2h" Jul 13 09:13:17.634952 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.634847 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5f8d3e22-5af7-4150-9f56-65e55d2e6713-oauth-serving-cert\") pod \"console-7787895976-9pm2h\" (UID: \"5f8d3e22-5af7-4150-9f56-65e55d2e6713\") " pod="openshift-console/console-7787895976-9pm2h" Jul 13 09:13:17.634952 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.634880 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/86ebac58-f50b-4e17-8153-b9e3bfbed54e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-crsd7\" (UID: \"86ebac58-f50b-4e17-8153-b9e3bfbed54e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-crsd7" Jul 13 09:13:17.634952 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:13:17.634941 2552 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 13 09:13:17.635042 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:13:17.634996 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86ebac58-f50b-4e17-8153-b9e3bfbed54e-prometheus-operator-tls podName:86ebac58-f50b-4e17-8153-b9e3bfbed54e nodeName:}" failed. No retries permitted until 2026-07-13 09:13:18.134983866 +0000 UTC m=+282.014939115 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/86ebac58-f50b-4e17-8153-b9e3bfbed54e-prometheus-operator-tls") pod "prometheus-operator-64785b6675-crsd7" (UID: "86ebac58-f50b-4e17-8153-b9e3bfbed54e") : secret "prometheus-operator-tls" not found Jul 13 09:13:17.635125 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.635053 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pc2x\" (UniqueName: \"kubernetes.io/projected/86ebac58-f50b-4e17-8153-b9e3bfbed54e-kube-api-access-9pc2x\") pod \"prometheus-operator-64785b6675-crsd7\" (UID: \"86ebac58-f50b-4e17-8153-b9e3bfbed54e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-crsd7" Jul 13 09:13:17.635125 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.635094 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5f8d3e22-5af7-4150-9f56-65e55d2e6713-console-serving-cert\") pod \"console-7787895976-9pm2h\" (UID: \"5f8d3e22-5af7-4150-9f56-65e55d2e6713\") " pod="openshift-console/console-7787895976-9pm2h" Jul 13 09:13:17.635193 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.635162 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5f8d3e22-5af7-4150-9f56-65e55d2e6713-service-ca\") pod \"console-7787895976-9pm2h\" (UID: \"5f8d3e22-5af7-4150-9f56-65e55d2e6713\") " pod="openshift-console/console-7787895976-9pm2h" Jul 13 09:13:17.635261 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.635246 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86ebac58-f50b-4e17-8153-b9e3bfbed54e-metrics-client-ca\") pod \"prometheus-operator-64785b6675-crsd7\" (UID: \"86ebac58-f50b-4e17-8153-b9e3bfbed54e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-crsd7" Jul 13 09:13:17.635303 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.635275 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5f8d3e22-5af7-4150-9f56-65e55d2e6713-console-oauth-config\") pod \"console-7787895976-9pm2h\" (UID: \"5f8d3e22-5af7-4150-9f56-65e55d2e6713\") " pod="openshift-console/console-7787895976-9pm2h" Jul 13 09:13:17.635339 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.635299 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nw6t\" (UniqueName: \"kubernetes.io/projected/5f8d3e22-5af7-4150-9f56-65e55d2e6713-kube-api-access-6nw6t\") pod \"console-7787895976-9pm2h\" (UID: \"5f8d3e22-5af7-4150-9f56-65e55d2e6713\") " pod="openshift-console/console-7787895976-9pm2h" Jul 13 09:13:17.635809 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.635785 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86ebac58-f50b-4e17-8153-b9e3bfbed54e-metrics-client-ca\") pod \"prometheus-operator-64785b6675-crsd7\" (UID: \"86ebac58-f50b-4e17-8153-b9e3bfbed54e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-crsd7" Jul 13 09:13:17.637222 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.637192 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/86ebac58-f50b-4e17-8153-b9e3bfbed54e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-crsd7\" (UID: \"86ebac58-f50b-4e17-8153-b9e3bfbed54e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-crsd7" Jul 13 09:13:17.643267 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.643250 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pc2x\" (UniqueName: \"kubernetes.io/projected/86ebac58-f50b-4e17-8153-b9e3bfbed54e-kube-api-access-9pc2x\") pod \"prometheus-operator-64785b6675-crsd7\" (UID: \"86ebac58-f50b-4e17-8153-b9e3bfbed54e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-crsd7" Jul 13 09:13:17.736428 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.736375 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5f8d3e22-5af7-4150-9f56-65e55d2e6713-console-config\") pod \"console-7787895976-9pm2h\" (UID: \"5f8d3e22-5af7-4150-9f56-65e55d2e6713\") " pod="openshift-console/console-7787895976-9pm2h" Jul 13 09:13:17.736428 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.736405 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5f8d3e22-5af7-4150-9f56-65e55d2e6713-oauth-serving-cert\") pod \"console-7787895976-9pm2h\" (UID: \"5f8d3e22-5af7-4150-9f56-65e55d2e6713\") " pod="openshift-console/console-7787895976-9pm2h" Jul 13 09:13:17.736428 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.736429 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5f8d3e22-5af7-4150-9f56-65e55d2e6713-console-serving-cert\") pod \"console-7787895976-9pm2h\" (UID: \"5f8d3e22-5af7-4150-9f56-65e55d2e6713\") " pod="openshift-console/console-7787895976-9pm2h" Jul 13 09:13:17.736583 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.736454 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5f8d3e22-5af7-4150-9f56-65e55d2e6713-service-ca\") pod \"console-7787895976-9pm2h\" (UID: \"5f8d3e22-5af7-4150-9f56-65e55d2e6713\") " pod="openshift-console/console-7787895976-9pm2h" Jul 13 09:13:17.736583 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.736492 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5f8d3e22-5af7-4150-9f56-65e55d2e6713-console-oauth-config\") pod \"console-7787895976-9pm2h\" (UID: \"5f8d3e22-5af7-4150-9f56-65e55d2e6713\") " pod="openshift-console/console-7787895976-9pm2h" Jul 13 09:13:17.736680 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.736608 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nw6t\" (UniqueName: \"kubernetes.io/projected/5f8d3e22-5af7-4150-9f56-65e55d2e6713-kube-api-access-6nw6t\") pod \"console-7787895976-9pm2h\" (UID: \"5f8d3e22-5af7-4150-9f56-65e55d2e6713\") " pod="openshift-console/console-7787895976-9pm2h" Jul 13 09:13:17.737065 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.737046 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5f8d3e22-5af7-4150-9f56-65e55d2e6713-console-config\") pod \"console-7787895976-9pm2h\" (UID: \"5f8d3e22-5af7-4150-9f56-65e55d2e6713\") " pod="openshift-console/console-7787895976-9pm2h" Jul 13 09:13:17.737148 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.737120 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5f8d3e22-5af7-4150-9f56-65e55d2e6713-service-ca\") pod \"console-7787895976-9pm2h\" (UID: \"5f8d3e22-5af7-4150-9f56-65e55d2e6713\") " pod="openshift-console/console-7787895976-9pm2h" Jul 13 09:13:17.737148 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.737120 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5f8d3e22-5af7-4150-9f56-65e55d2e6713-oauth-serving-cert\") pod \"console-7787895976-9pm2h\" (UID: \"5f8d3e22-5af7-4150-9f56-65e55d2e6713\") " pod="openshift-console/console-7787895976-9pm2h" Jul 13 09:13:17.738550 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.738527 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5f8d3e22-5af7-4150-9f56-65e55d2e6713-console-oauth-config\") pod \"console-7787895976-9pm2h\" (UID: \"5f8d3e22-5af7-4150-9f56-65e55d2e6713\") " pod="openshift-console/console-7787895976-9pm2h" Jul 13 09:13:17.738732 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.738717 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5f8d3e22-5af7-4150-9f56-65e55d2e6713-console-serving-cert\") pod \"console-7787895976-9pm2h\" (UID: \"5f8d3e22-5af7-4150-9f56-65e55d2e6713\") " pod="openshift-console/console-7787895976-9pm2h" Jul 13 09:13:17.744871 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.744852 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nw6t\" (UniqueName: \"kubernetes.io/projected/5f8d3e22-5af7-4150-9f56-65e55d2e6713-kube-api-access-6nw6t\") pod \"console-7787895976-9pm2h\" (UID: \"5f8d3e22-5af7-4150-9f56-65e55d2e6713\") " pod="openshift-console/console-7787895976-9pm2h" Jul 13 09:13:17.761840 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.761822 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7787895976-9pm2h" Jul 13 09:13:17.788937 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.788892 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:17.881528 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:17.881376 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7787895976-9pm2h"] Jul 13 09:13:17.883821 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:13:17.883791 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f8d3e22_5af7_4150_9f56_65e55d2e6713.slice/crio-534560b8b633e6422472c6a629a06e716f4d8916dfd20e51de5f206a5a5db6bf WatchSource:0}: Error finding container 534560b8b633e6422472c6a629a06e716f4d8916dfd20e51de5f206a5a5db6bf: Status 404 returned error can't find the container with id 534560b8b633e6422472c6a629a06e716f4d8916dfd20e51de5f206a5a5db6bf Jul 13 09:13:18.139963 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:18.139938 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/86ebac58-f50b-4e17-8153-b9e3bfbed54e-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-crsd7\" (UID: \"86ebac58-f50b-4e17-8153-b9e3bfbed54e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-crsd7" Jul 13 09:13:18.142194 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:18.142174 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/86ebac58-f50b-4e17-8153-b9e3bfbed54e-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-crsd7\" (UID: \"86ebac58-f50b-4e17-8153-b9e3bfbed54e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-crsd7" Jul 13 09:13:18.337285 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:18.337255 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-crsd7" Jul 13 09:13:18.480007 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:18.479976 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-crsd7"] Jul 13 09:13:18.483268 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:13:18.483238 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86ebac58_f50b_4e17_8153_b9e3bfbed54e.slice/crio-f9d586b7b0cc2ca7083d6aa498bf9da10f10a5c31f9dfbabf7973bcada951e9f WatchSource:0}: Error finding container f9d586b7b0cc2ca7083d6aa498bf9da10f10a5c31f9dfbabf7973bcada951e9f: Status 404 returned error can't find the container with id f9d586b7b0cc2ca7083d6aa498bf9da10f10a5c31f9dfbabf7973bcada951e9f Jul 13 09:13:18.502007 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:18.501975 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-crsd7" event={"ID":"86ebac58-f50b-4e17-8153-b9e3bfbed54e","Type":"ContainerStarted","Data":"f9d586b7b0cc2ca7083d6aa498bf9da10f10a5c31f9dfbabf7973bcada951e9f"} Jul 13 09:13:18.503333 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:18.503305 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7787895976-9pm2h" event={"ID":"5f8d3e22-5af7-4150-9f56-65e55d2e6713","Type":"ContainerStarted","Data":"534560b8b633e6422472c6a629a06e716f4d8916dfd20e51de5f206a5a5db6bf"} Jul 13 09:13:18.628452 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:18.628427 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:19.122895 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:13:19.122857 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-85jbg" podUID="08c6cfc3-a46b-447e-b470-a2f19a4ddcce" Jul 13 09:13:19.482773 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:19.482697 2552 patch_prober.go:28] interesting pod/image-registry-68c5fcfc4f-v2f5q container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 13 09:13:19.482773 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:19.482754 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" podUID="c0a4a73b-5bd3-4ff2-bd69-8a68bc640539" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 09:13:19.507268 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:19.507241 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-85jbg" Jul 13 09:13:21.135306 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:21.135199 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:21.515553 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:21.515452 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7787895976-9pm2h" event={"ID":"5f8d3e22-5af7-4150-9f56-65e55d2e6713","Type":"ContainerStarted","Data":"576ca4df9340acef8f4adf4fa84e94d24b6596e9172ecebe8411a6894ad6501a"} Jul 13 09:13:21.517149 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:21.517121 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-crsd7" event={"ID":"86ebac58-f50b-4e17-8153-b9e3bfbed54e","Type":"ContainerStarted","Data":"3c8907537e04027345de5c3eea029de0d36547e3d31290111188d17652645a70"} Jul 13 09:13:21.517273 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:21.517171 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-crsd7" event={"ID":"86ebac58-f50b-4e17-8153-b9e3bfbed54e","Type":"ContainerStarted","Data":"8532b66b954f592aa006faecfdb97ff644ce70465f6fb966b9c23e73f5f11ba2"} Jul 13 09:13:21.533906 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:21.533862 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7787895976-9pm2h" podStartSLOduration=1.575196167 podStartE2EDuration="4.533847666s" totalImagesPullingTime="2.958651499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:13:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:13:21.53328003 +0000 UTC m=+285.413235298" watchObservedRunningTime="2026-07-13 09:13:21.533847666 +0000 UTC m=+285.413802932" Jul 13 09:13:21.549690 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:21.549651 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-crsd7" podStartSLOduration=2.198841029 podStartE2EDuration="4.549636246s" totalImagesPullingTime="2.350795217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:13:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:13:21.548812689 +0000 UTC m=+285.428767950" watchObservedRunningTime="2026-07-13 09:13:21.549636246 +0000 UTC m=+285.429591519" Jul 13 09:13:22.373904 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:22.373869 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-metrics-tls\") pod \"dns-default-85jbg\" (UID: \"08c6cfc3-a46b-447e-b470-a2f19a4ddcce\") " pod="openshift-dns/dns-default-85jbg" Jul 13 09:13:22.376137 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:22.376116 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08c6cfc3-a46b-447e-b470-a2f19a4ddcce-metrics-tls\") pod \"dns-default-85jbg\" (UID: \"08c6cfc3-a46b-447e-b470-a2f19a4ddcce\") " pod="openshift-dns/dns-default-85jbg" Jul 13 09:13:22.511203 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:22.511180 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jbvrj\"" Jul 13 09:13:22.518415 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:22.518396 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-85jbg" Jul 13 09:13:22.640132 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:22.640112 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-85jbg"] Jul 13 09:13:22.642134 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:13:22.642109 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08c6cfc3_a46b_447e_b470_a2f19a4ddcce.slice/crio-0dc8f373cf7e839c251d500b5c125fee13616e0044a9ca06f85aa4d137441235 WatchSource:0}: Error finding container 0dc8f373cf7e839c251d500b5c125fee13616e0044a9ca06f85aa4d137441235: Status 404 returned error can't find the container with id 0dc8f373cf7e839c251d500b5c125fee13616e0044a9ca06f85aa4d137441235 Jul 13 09:13:23.526049 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.526013 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-85jbg" event={"ID":"08c6cfc3-a46b-447e-b470-a2f19a4ddcce","Type":"ContainerStarted","Data":"0dc8f373cf7e839c251d500b5c125fee13616e0044a9ca06f85aa4d137441235"} Jul 13 09:13:23.544429 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.544401 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:23.743134 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.743106 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:23.793408 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.793285 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-lmn8p"] Jul 13 09:13:23.797147 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.797080 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lmn8p" Jul 13 09:13:23.801095 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.799807 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-r5rh6"] Jul 13 09:13:23.801095 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.800150 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-tj69d\"" Jul 13 09:13:23.801095 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.800399 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 13 09:13:23.801095 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.800600 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 13 09:13:23.803823 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.803128 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-r5rh6" Jul 13 09:13:23.811652 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.810864 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-lmn8p"] Jul 13 09:13:23.811652 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.811033 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-cmhg8\"" Jul 13 09:13:23.811652 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.811227 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 13 09:13:23.815050 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.814271 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 13 09:13:23.815778 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.815618 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 13 09:13:23.830127 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.830105 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-djfbm"] Jul 13 09:13:23.835810 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.835780 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-djfbm" Jul 13 09:13:23.838303 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.838281 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 13 09:13:23.838514 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.838497 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-nhq5x\"" Jul 13 09:13:23.838647 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.838594 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 13 09:13:23.838807 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.838786 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 13 09:13:23.840745 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.840718 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-djfbm"] Jul 13 09:13:23.842702 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.842660 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:23.884100 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.884072 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5def036b-13e7-4089-bfe0-3847346da725-node-exporter-accelerators-collector-config\") pod \"node-exporter-r5rh6\" (UID: \"5def036b-13e7-4089-bfe0-3847346da725\") " pod="openshift-monitoring/node-exporter-r5rh6" Jul 13 09:13:23.884290 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.884119 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j6ln\" (UniqueName: \"kubernetes.io/projected/e21df076-b466-44be-afdc-ee35d2749a00-kube-api-access-7j6ln\") pod \"openshift-state-metrics-84bd68b896-lmn8p\" (UID: \"e21df076-b466-44be-afdc-ee35d2749a00\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lmn8p" Jul 13 09:13:23.884290 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.884173 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgcwx\" (UniqueName: \"kubernetes.io/projected/5def036b-13e7-4089-bfe0-3847346da725-kube-api-access-kgcwx\") pod \"node-exporter-r5rh6\" (UID: \"5def036b-13e7-4089-bfe0-3847346da725\") " pod="openshift-monitoring/node-exporter-r5rh6" Jul 13 09:13:23.884290 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.884236 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/88805853-a812-47a7-81a0-9f96bf4d16ae-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-djfbm\" (UID: \"88805853-a812-47a7-81a0-9f96bf4d16ae\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-djfbm" Jul 13 09:13:23.884290 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.884271 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/88805853-a812-47a7-81a0-9f96bf4d16ae-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-djfbm\" (UID: \"88805853-a812-47a7-81a0-9f96bf4d16ae\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-djfbm" Jul 13 09:13:23.884492 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.884305 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/88805853-a812-47a7-81a0-9f96bf4d16ae-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-djfbm\" (UID: \"88805853-a812-47a7-81a0-9f96bf4d16ae\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-djfbm" Jul 13 09:13:23.884492 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.884363 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e21df076-b466-44be-afdc-ee35d2749a00-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-lmn8p\" (UID: \"e21df076-b466-44be-afdc-ee35d2749a00\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lmn8p" Jul 13 09:13:23.884492 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.884390 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5def036b-13e7-4089-bfe0-3847346da725-root\") pod \"node-exporter-r5rh6\" (UID: \"5def036b-13e7-4089-bfe0-3847346da725\") " pod="openshift-monitoring/node-exporter-r5rh6" Jul 13 09:13:23.884492 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.884436 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5def036b-13e7-4089-bfe0-3847346da725-metrics-client-ca\") pod \"node-exporter-r5rh6\" (UID: \"5def036b-13e7-4089-bfe0-3847346da725\") " pod="openshift-monitoring/node-exporter-r5rh6" Jul 13 09:13:23.884492 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.884467 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e21df076-b466-44be-afdc-ee35d2749a00-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-lmn8p\" (UID: \"e21df076-b466-44be-afdc-ee35d2749a00\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lmn8p" Jul 13 09:13:23.884492 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.884492 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5def036b-13e7-4089-bfe0-3847346da725-sys\") pod \"node-exporter-r5rh6\" (UID: \"5def036b-13e7-4089-bfe0-3847346da725\") " pod="openshift-monitoring/node-exporter-r5rh6" Jul 13 09:13:23.884767 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.884513 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e21df076-b466-44be-afdc-ee35d2749a00-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-lmn8p\" (UID: \"e21df076-b466-44be-afdc-ee35d2749a00\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lmn8p" Jul 13 09:13:23.884767 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.884535 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5def036b-13e7-4089-bfe0-3847346da725-node-exporter-wtmp\") pod \"node-exporter-r5rh6\" (UID: \"5def036b-13e7-4089-bfe0-3847346da725\") " pod="openshift-monitoring/node-exporter-r5rh6" Jul 13 09:13:23.884767 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.884592 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5def036b-13e7-4089-bfe0-3847346da725-node-exporter-tls\") pod \"node-exporter-r5rh6\" (UID: \"5def036b-13e7-4089-bfe0-3847346da725\") " pod="openshift-monitoring/node-exporter-r5rh6" Jul 13 09:13:23.884767 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.884619 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/88805853-a812-47a7-81a0-9f96bf4d16ae-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-djfbm\" (UID: \"88805853-a812-47a7-81a0-9f96bf4d16ae\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-djfbm" Jul 13 09:13:23.884767 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.884644 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5def036b-13e7-4089-bfe0-3847346da725-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r5rh6\" (UID: \"5def036b-13e7-4089-bfe0-3847346da725\") " pod="openshift-monitoring/node-exporter-r5rh6" Jul 13 09:13:23.884767 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.884716 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8h69\" (UniqueName: \"kubernetes.io/projected/88805853-a812-47a7-81a0-9f96bf4d16ae-kube-api-access-l8h69\") pod \"kube-state-metrics-785778ff74-djfbm\" (UID: \"88805853-a812-47a7-81a0-9f96bf4d16ae\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-djfbm" Jul 13 09:13:23.884767 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.884743 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5def036b-13e7-4089-bfe0-3847346da725-node-exporter-textfile\") pod \"node-exporter-r5rh6\" (UID: \"5def036b-13e7-4089-bfe0-3847346da725\") " pod="openshift-monitoring/node-exporter-r5rh6" Jul 13 09:13:23.884767 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.884760 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/88805853-a812-47a7-81a0-9f96bf4d16ae-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-djfbm\" (UID: \"88805853-a812-47a7-81a0-9f96bf4d16ae\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-djfbm" Jul 13 09:13:23.940119 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.940094 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:23.986157 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.985519 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5def036b-13e7-4089-bfe0-3847346da725-metrics-client-ca\") pod \"node-exporter-r5rh6\" (UID: \"5def036b-13e7-4089-bfe0-3847346da725\") " pod="openshift-monitoring/node-exporter-r5rh6" Jul 13 09:13:23.986157 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.985563 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e21df076-b466-44be-afdc-ee35d2749a00-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-lmn8p\" (UID: \"e21df076-b466-44be-afdc-ee35d2749a00\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lmn8p" Jul 13 09:13:23.986157 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.985593 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5def036b-13e7-4089-bfe0-3847346da725-sys\") pod \"node-exporter-r5rh6\" (UID: \"5def036b-13e7-4089-bfe0-3847346da725\") " pod="openshift-monitoring/node-exporter-r5rh6" Jul 13 09:13:23.986157 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.985626 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e21df076-b466-44be-afdc-ee35d2749a00-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-lmn8p\" (UID: \"e21df076-b466-44be-afdc-ee35d2749a00\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lmn8p" Jul 13 09:13:23.986157 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.985651 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5def036b-13e7-4089-bfe0-3847346da725-node-exporter-wtmp\") pod \"node-exporter-r5rh6\" (UID: \"5def036b-13e7-4089-bfe0-3847346da725\") " pod="openshift-monitoring/node-exporter-r5rh6" Jul 13 09:13:23.986157 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.985681 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5def036b-13e7-4089-bfe0-3847346da725-node-exporter-tls\") pod \"node-exporter-r5rh6\" (UID: \"5def036b-13e7-4089-bfe0-3847346da725\") " pod="openshift-monitoring/node-exporter-r5rh6" Jul 13 09:13:23.986157 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.985733 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/88805853-a812-47a7-81a0-9f96bf4d16ae-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-djfbm\" (UID: \"88805853-a812-47a7-81a0-9f96bf4d16ae\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-djfbm" Jul 13 09:13:23.986157 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.985778 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5def036b-13e7-4089-bfe0-3847346da725-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r5rh6\" (UID: \"5def036b-13e7-4089-bfe0-3847346da725\") " pod="openshift-monitoring/node-exporter-r5rh6" Jul 13 09:13:23.986157 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.985817 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8h69\" (UniqueName: \"kubernetes.io/projected/88805853-a812-47a7-81a0-9f96bf4d16ae-kube-api-access-l8h69\") pod \"kube-state-metrics-785778ff74-djfbm\" (UID: \"88805853-a812-47a7-81a0-9f96bf4d16ae\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-djfbm" Jul 13 09:13:23.986157 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.985827 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5def036b-13e7-4089-bfe0-3847346da725-node-exporter-wtmp\") pod \"node-exporter-r5rh6\" (UID: \"5def036b-13e7-4089-bfe0-3847346da725\") " pod="openshift-monitoring/node-exporter-r5rh6" Jul 13 09:13:23.986157 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.985849 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5def036b-13e7-4089-bfe0-3847346da725-node-exporter-textfile\") pod \"node-exporter-r5rh6\" (UID: \"5def036b-13e7-4089-bfe0-3847346da725\") " pod="openshift-monitoring/node-exporter-r5rh6" Jul 13 09:13:23.986157 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.985871 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5def036b-13e7-4089-bfe0-3847346da725-sys\") pod \"node-exporter-r5rh6\" (UID: \"5def036b-13e7-4089-bfe0-3847346da725\") " pod="openshift-monitoring/node-exporter-r5rh6" Jul 13 09:13:23.986157 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.985877 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/88805853-a812-47a7-81a0-9f96bf4d16ae-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-djfbm\" (UID: \"88805853-a812-47a7-81a0-9f96bf4d16ae\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-djfbm" Jul 13 09:13:23.986157 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.985921 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5def036b-13e7-4089-bfe0-3847346da725-node-exporter-accelerators-collector-config\") pod \"node-exporter-r5rh6\" (UID: \"5def036b-13e7-4089-bfe0-3847346da725\") " pod="openshift-monitoring/node-exporter-r5rh6" Jul 13 09:13:23.986157 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.985952 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j6ln\" (UniqueName: \"kubernetes.io/projected/e21df076-b466-44be-afdc-ee35d2749a00-kube-api-access-7j6ln\") pod \"openshift-state-metrics-84bd68b896-lmn8p\" (UID: \"e21df076-b466-44be-afdc-ee35d2749a00\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lmn8p" Jul 13 09:13:23.986157 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.985977 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgcwx\" (UniqueName: \"kubernetes.io/projected/5def036b-13e7-4089-bfe0-3847346da725-kube-api-access-kgcwx\") pod \"node-exporter-r5rh6\" (UID: \"5def036b-13e7-4089-bfe0-3847346da725\") " pod="openshift-monitoring/node-exporter-r5rh6" Jul 13 09:13:23.987024 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.986002 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/88805853-a812-47a7-81a0-9f96bf4d16ae-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-djfbm\" (UID: \"88805853-a812-47a7-81a0-9f96bf4d16ae\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-djfbm" Jul 13 09:13:23.987024 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.986027 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/88805853-a812-47a7-81a0-9f96bf4d16ae-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-djfbm\" (UID: \"88805853-a812-47a7-81a0-9f96bf4d16ae\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-djfbm" Jul 13 09:13:23.987024 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.986062 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/88805853-a812-47a7-81a0-9f96bf4d16ae-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-djfbm\" (UID: \"88805853-a812-47a7-81a0-9f96bf4d16ae\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-djfbm" Jul 13 09:13:23.987024 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.986109 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e21df076-b466-44be-afdc-ee35d2749a00-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-lmn8p\" (UID: \"e21df076-b466-44be-afdc-ee35d2749a00\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lmn8p" Jul 13 09:13:23.987024 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.986131 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5def036b-13e7-4089-bfe0-3847346da725-root\") pod \"node-exporter-r5rh6\" (UID: \"5def036b-13e7-4089-bfe0-3847346da725\") " pod="openshift-monitoring/node-exporter-r5rh6" Jul 13 09:13:23.987024 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.986232 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5def036b-13e7-4089-bfe0-3847346da725-root\") pod \"node-exporter-r5rh6\" (UID: \"5def036b-13e7-4089-bfe0-3847346da725\") " pod="openshift-monitoring/node-exporter-r5rh6" Jul 13 09:13:23.987024 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.986259 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5def036b-13e7-4089-bfe0-3847346da725-metrics-client-ca\") pod \"node-exporter-r5rh6\" (UID: \"5def036b-13e7-4089-bfe0-3847346da725\") " pod="openshift-monitoring/node-exporter-r5rh6" Jul 13 09:13:23.987024 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.986651 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e21df076-b466-44be-afdc-ee35d2749a00-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-lmn8p\" (UID: \"e21df076-b466-44be-afdc-ee35d2749a00\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lmn8p" Jul 13 09:13:23.987024 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.986732 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5def036b-13e7-4089-bfe0-3847346da725-node-exporter-textfile\") pod \"node-exporter-r5rh6\" (UID: \"5def036b-13e7-4089-bfe0-3847346da725\") " pod="openshift-monitoring/node-exporter-r5rh6" Jul 13 09:13:23.987024 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.986939 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/88805853-a812-47a7-81a0-9f96bf4d16ae-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-djfbm\" (UID: \"88805853-a812-47a7-81a0-9f96bf4d16ae\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-djfbm" Jul 13 09:13:23.987024 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:13:23.986950 2552 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 13 09:13:23.987024 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.986984 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/88805853-a812-47a7-81a0-9f96bf4d16ae-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-djfbm\" (UID: \"88805853-a812-47a7-81a0-9f96bf4d16ae\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-djfbm" Jul 13 09:13:23.987024 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:13:23.987002 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88805853-a812-47a7-81a0-9f96bf4d16ae-kube-state-metrics-tls podName:88805853-a812-47a7-81a0-9f96bf4d16ae nodeName:}" failed. No retries permitted until 2026-07-13 09:13:24.486985531 +0000 UTC m=+288.366940779 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/88805853-a812-47a7-81a0-9f96bf4d16ae-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-djfbm" (UID: "88805853-a812-47a7-81a0-9f96bf4d16ae") : secret "kube-state-metrics-tls" not found Jul 13 09:13:23.987673 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.987641 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/88805853-a812-47a7-81a0-9f96bf4d16ae-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-djfbm\" (UID: \"88805853-a812-47a7-81a0-9f96bf4d16ae\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-djfbm" Jul 13 09:13:23.988107 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.988082 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5def036b-13e7-4089-bfe0-3847346da725-node-exporter-accelerators-collector-config\") pod \"node-exporter-r5rh6\" (UID: \"5def036b-13e7-4089-bfe0-3847346da725\") " pod="openshift-monitoring/node-exporter-r5rh6" Jul 13 09:13:23.988830 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.988790 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5def036b-13e7-4089-bfe0-3847346da725-node-exporter-tls\") pod \"node-exporter-r5rh6\" (UID: \"5def036b-13e7-4089-bfe0-3847346da725\") " pod="openshift-monitoring/node-exporter-r5rh6" Jul 13 09:13:23.989777 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.989751 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e21df076-b466-44be-afdc-ee35d2749a00-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-lmn8p\" (UID: \"e21df076-b466-44be-afdc-ee35d2749a00\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lmn8p" Jul 13 09:13:23.990053 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.990028 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/88805853-a812-47a7-81a0-9f96bf4d16ae-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-djfbm\" (UID: \"88805853-a812-47a7-81a0-9f96bf4d16ae\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-djfbm" Jul 13 09:13:23.990730 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.990675 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e21df076-b466-44be-afdc-ee35d2749a00-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-lmn8p\" (UID: \"e21df076-b466-44be-afdc-ee35d2749a00\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lmn8p" Jul 13 09:13:23.990836 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.990822 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5def036b-13e7-4089-bfe0-3847346da725-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r5rh6\" (UID: \"5def036b-13e7-4089-bfe0-3847346da725\") " pod="openshift-monitoring/node-exporter-r5rh6" Jul 13 09:13:23.995840 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.995798 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8h69\" (UniqueName: \"kubernetes.io/projected/88805853-a812-47a7-81a0-9f96bf4d16ae-kube-api-access-l8h69\") pod \"kube-state-metrics-785778ff74-djfbm\" (UID: \"88805853-a812-47a7-81a0-9f96bf4d16ae\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-djfbm" Jul 13 09:13:23.996662 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.996631 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j6ln\" (UniqueName: \"kubernetes.io/projected/e21df076-b466-44be-afdc-ee35d2749a00-kube-api-access-7j6ln\") pod \"openshift-state-metrics-84bd68b896-lmn8p\" (UID: \"e21df076-b466-44be-afdc-ee35d2749a00\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lmn8p" Jul 13 09:13:23.998268 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:23.998223 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgcwx\" (UniqueName: \"kubernetes.io/projected/5def036b-13e7-4089-bfe0-3847346da725-kube-api-access-kgcwx\") pod \"node-exporter-r5rh6\" (UID: \"5def036b-13e7-4089-bfe0-3847346da725\") " pod="openshift-monitoring/node-exporter-r5rh6" Jul 13 09:13:24.040255 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:24.040163 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:24.117983 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:24.117948 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lmn8p" Jul 13 09:13:24.133568 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:24.133546 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-r5rh6" Jul 13 09:13:24.140335 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:24.140306 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:24.241126 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:24.241093 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:24.243688 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:13:24.243659 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5def036b_13e7_4089_bfe0_3847346da725.slice/crio-d1f9823d15455eb4a0c916a63de0f1e9035c45bc62f4515474acc7b1c44d1cd0 WatchSource:0}: Error finding container d1f9823d15455eb4a0c916a63de0f1e9035c45bc62f4515474acc7b1c44d1cd0: Status 404 returned error can't find the container with id d1f9823d15455eb4a0c916a63de0f1e9035c45bc62f4515474acc7b1c44d1cd0 Jul 13 09:13:24.341649 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:24.341616 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:24.363597 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:24.363437 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-lmn8p"] Jul 13 09:13:24.365336 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:13:24.365303 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode21df076_b466_44be_afdc_ee35d2749a00.slice/crio-81039a32f005409bfa6efa3549492594cd0181020120d80bdd8583f6623946b1 WatchSource:0}: Error finding container 81039a32f005409bfa6efa3549492594cd0181020120d80bdd8583f6623946b1: Status 404 returned error can't find the container with id 81039a32f005409bfa6efa3549492594cd0181020120d80bdd8583f6623946b1 Jul 13 09:13:24.489707 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:24.489681 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/88805853-a812-47a7-81a0-9f96bf4d16ae-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-djfbm\" (UID: \"88805853-a812-47a7-81a0-9f96bf4d16ae\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-djfbm" Jul 13 09:13:24.492163 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:24.492142 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/88805853-a812-47a7-81a0-9f96bf4d16ae-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-djfbm\" (UID: \"88805853-a812-47a7-81a0-9f96bf4d16ae\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-djfbm" Jul 13 09:13:24.532036 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:24.532009 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-85jbg" event={"ID":"08c6cfc3-a46b-447e-b470-a2f19a4ddcce","Type":"ContainerStarted","Data":"eeef2926174e92fadc2231a81f90908b365033f526485496ea5b663a37b38091"} Jul 13 09:13:24.532389 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:24.532054 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-85jbg" event={"ID":"08c6cfc3-a46b-447e-b470-a2f19a4ddcce","Type":"ContainerStarted","Data":"e4bb2e9a92d84b898278ce92e0662989998b4499c2407e1bf9a8ae0c72bcc8c3"} Jul 13 09:13:24.532389 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:24.532122 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-85jbg" Jul 13 09:13:24.533193 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:24.533167 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r5rh6" event={"ID":"5def036b-13e7-4089-bfe0-3847346da725","Type":"ContainerStarted","Data":"d1f9823d15455eb4a0c916a63de0f1e9035c45bc62f4515474acc7b1c44d1cd0"} Jul 13 09:13:24.534814 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:24.534790 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lmn8p" event={"ID":"e21df076-b466-44be-afdc-ee35d2749a00","Type":"ContainerStarted","Data":"fdd69fea8919bf4f236828b0f65ada65f1bae1c7a8e5828dc417d222d75bdc4a"} Jul 13 09:13:24.534928 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:24.534846 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lmn8p" event={"ID":"e21df076-b466-44be-afdc-ee35d2749a00","Type":"ContainerStarted","Data":"522eb1ac5ecbd37e6175d9c5df4bb6dcb49e08b2c061ff511646fc26d108da7b"} Jul 13 09:13:24.534928 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:24.534877 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lmn8p" event={"ID":"e21df076-b466-44be-afdc-ee35d2749a00","Type":"ContainerStarted","Data":"81039a32f005409bfa6efa3549492594cd0181020120d80bdd8583f6623946b1"} Jul 13 09:13:24.550818 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:24.550779 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-85jbg" podStartSLOduration=250.909999532 podStartE2EDuration="4m12.550766477s" totalImagesPullingTime="1.640766945s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:09:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:13:24.550055919 +0000 UTC m=+288.430011198" watchObservedRunningTime="2026-07-13 09:13:24.550766477 +0000 UTC m=+288.430721744" Jul 13 09:13:24.749000 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:24.748809 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-djfbm" Jul 13 09:13:24.765638 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:24.765609 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:24.857356 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:24.857327 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:24.902139 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:24.902108 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-djfbm"] Jul 13 09:13:24.905287 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:13:24.905261 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88805853_a812_47a7_81a0_9f96bf4d16ae.slice/crio-16e4b192e24a5efe2ff729b7517b3bb56b0ad6df406a2963766b55601bd363a5 WatchSource:0}: Error finding container 16e4b192e24a5efe2ff729b7517b3bb56b0ad6df406a2963766b55601bd363a5: Status 404 returned error can't find the container with id 16e4b192e24a5efe2ff729b7517b3bb56b0ad6df406a2963766b55601bd363a5 Jul 13 09:13:24.967827 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:24.967788 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:25.539563 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:25.539538 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-djfbm" event={"ID":"88805853-a812-47a7-81a0-9f96bf4d16ae","Type":"ContainerStarted","Data":"16e4b192e24a5efe2ff729b7517b3bb56b0ad6df406a2963766b55601bd363a5"} Jul 13 09:13:25.541078 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:25.541057 2552 generic.go:356] "Generic (PLEG): container finished" podID="5def036b-13e7-4089-bfe0-3847346da725" containerID="8f513dbaa425b219c7effe247065f768d3f43b70e9958f360030adcc299edaac" exitCode=0 Jul 13 09:13:25.541169 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:25.541139 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r5rh6" event={"ID":"5def036b-13e7-4089-bfe0-3847346da725","Type":"ContainerDied","Data":"8f513dbaa425b219c7effe247065f768d3f43b70e9958f360030adcc299edaac"} Jul 13 09:13:26.548594 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:26.548570 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r5rh6" event={"ID":"5def036b-13e7-4089-bfe0-3847346da725","Type":"ContainerStarted","Data":"18138ee166a2136b1bdd8b50ffe1170191da127725685349b4b80b12b50d62d2"} Jul 13 09:13:26.548973 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:26.548617 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r5rh6" event={"ID":"5def036b-13e7-4089-bfe0-3847346da725","Type":"ContainerStarted","Data":"4f84135e2c1f1b5ca471bae4d2a579436c02292fa8db1851dbd8ef4a8d205b9f"} Jul 13 09:13:26.550888 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:26.550863 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lmn8p" event={"ID":"e21df076-b466-44be-afdc-ee35d2749a00","Type":"ContainerStarted","Data":"a037ef058feb2da44aae647fb91694cecc55cea95dd891a686a67c4b749b6ff4"} Jul 13 09:13:26.553474 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:26.553402 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-djfbm" event={"ID":"88805853-a812-47a7-81a0-9f96bf4d16ae","Type":"ContainerStarted","Data":"8fbb2b00422b92b3a9bddfc0d318b7236a3ce1b8ade13fa5252bd4f8102e6abf"} Jul 13 09:13:26.553474 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:26.553449 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-djfbm" event={"ID":"88805853-a812-47a7-81a0-9f96bf4d16ae","Type":"ContainerStarted","Data":"d66be1d3af1f166f807f4a0f339e4b6e340bc404e6d38734fc59a6e54504d1d0"} Jul 13 09:13:26.571356 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:26.571313 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-r5rh6" podStartSLOduration=2.555939799 podStartE2EDuration="3.571300055s" totalImagesPullingTime="946.851513ms" totalInitContainerRuntime="68.508743ms" isStatefulPod=true podCreationTimestamp="2026-07-13 09:13:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:13:26.569424956 +0000 UTC m=+290.449380224" watchObservedRunningTime="2026-07-13 09:13:26.571300055 +0000 UTC m=+290.451255300" Jul 13 09:13:26.588241 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:26.588184 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lmn8p" podStartSLOduration=2.577366687 podStartE2EDuration="3.588170194s" totalImagesPullingTime="1.010803507s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:13:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:13:26.587750314 +0000 UTC m=+290.467705582" watchObservedRunningTime="2026-07-13 09:13:26.588170194 +0000 UTC m=+290.468125460" Jul 13 09:13:26.729464 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:26.729376 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:26.830559 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:26.830524 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:26.931939 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:26.931898 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:27.558943 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:27.558905 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-djfbm" event={"ID":"88805853-a812-47a7-81a0-9f96bf4d16ae","Type":"ContainerStarted","Data":"8bbbe33dd2fc4ff3104a2c83d4c32e7c0e412b554f67f88456b0a561706ee356"} Jul 13 09:13:27.578394 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:27.578350 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-djfbm" podStartSLOduration=3.108402343 podStartE2EDuration="4.578335623s" totalImagesPullingTime="1.46993328s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:13:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:13:27.577951216 +0000 UTC m=+291.457906480" watchObservedRunningTime="2026-07-13 09:13:27.578335623 +0000 UTC m=+291.458290890" Jul 13 09:13:27.762219 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:27.762182 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7787895976-9pm2h" Jul 13 09:13:27.762318 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:27.762265 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7787895976-9pm2h" Jul 13 09:13:27.763488 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:27.763465 2552 patch_prober.go:28] interesting pod/console-7787895976-9pm2h container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.16:8443/health\": dial tcp 10.133.0.16:8443: connect: connection refused" start-of-body= Jul 13 09:13:27.763577 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:27.763506 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-7787895976-9pm2h" podUID="5f8d3e22-5af7-4150-9f56-65e55d2e6713" containerName="console" probeResult="failure" output="Get \"https://10.133.0.16:8443/health\": dial tcp 10.133.0.16:8443: connect: connection refused" Jul 13 09:13:28.545694 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:28.545662 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:28.599349 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:28.599315 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-wjjsm"] Jul 13 09:13:28.602701 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:28.602676 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wjjsm" Jul 13 09:13:28.605663 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:28.605644 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 13 09:13:28.605769 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:28.605644 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-58cpw\"" Jul 13 09:13:28.613330 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:28.613301 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-wjjsm"] Jul 13 09:13:28.624381 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:28.624357 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fkbl\" (UniqueName: \"kubernetes.io/projected/d8e5e06f-6b18-4e06-be34-bc4df8d4bf87-kube-api-access-8fkbl\") pod \"monitoring-plugin-5f68df7f58-wjjsm\" (UID: \"d8e5e06f-6b18-4e06-be34-bc4df8d4bf87\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wjjsm" Jul 13 09:13:28.624519 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:28.624500 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d8e5e06f-6b18-4e06-be34-bc4df8d4bf87-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-wjjsm\" (UID: \"d8e5e06f-6b18-4e06-be34-bc4df8d4bf87\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wjjsm" Jul 13 09:13:28.628139 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:28.628119 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:28.632064 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:28.632041 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7787895976-9pm2h"] Jul 13 09:13:28.725150 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:28.725126 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fkbl\" (UniqueName: \"kubernetes.io/projected/d8e5e06f-6b18-4e06-be34-bc4df8d4bf87-kube-api-access-8fkbl\") pod \"monitoring-plugin-5f68df7f58-wjjsm\" (UID: \"d8e5e06f-6b18-4e06-be34-bc4df8d4bf87\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wjjsm" Jul 13 09:13:28.725276 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:28.725194 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d8e5e06f-6b18-4e06-be34-bc4df8d4bf87-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-wjjsm\" (UID: \"d8e5e06f-6b18-4e06-be34-bc4df8d4bf87\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wjjsm" Jul 13 09:13:28.725320 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:13:28.725300 2552 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 13 09:13:28.725357 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:13:28.725351 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8e5e06f-6b18-4e06-be34-bc4df8d4bf87-monitoring-plugin-cert podName:d8e5e06f-6b18-4e06-be34-bc4df8d4bf87 nodeName:}" failed. No retries permitted until 2026-07-13 09:13:29.225333842 +0000 UTC m=+293.105289091 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/d8e5e06f-6b18-4e06-be34-bc4df8d4bf87-monitoring-plugin-cert") pod "monitoring-plugin-5f68df7f58-wjjsm" (UID: "d8e5e06f-6b18-4e06-be34-bc4df8d4bf87") : secret "monitoring-plugin-cert" not found Jul 13 09:13:28.733868 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:28.733848 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fkbl\" (UniqueName: \"kubernetes.io/projected/d8e5e06f-6b18-4e06-be34-bc4df8d4bf87-kube-api-access-8fkbl\") pod \"monitoring-plugin-5f68df7f58-wjjsm\" (UID: \"d8e5e06f-6b18-4e06-be34-bc4df8d4bf87\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wjjsm" Jul 13 09:13:28.899449 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:28.899423 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:29.228379 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:29.228301 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d8e5e06f-6b18-4e06-be34-bc4df8d4bf87-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-wjjsm\" (UID: \"d8e5e06f-6b18-4e06-be34-bc4df8d4bf87\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wjjsm" Jul 13 09:13:29.230801 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:29.230775 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d8e5e06f-6b18-4e06-be34-bc4df8d4bf87-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-wjjsm\" (UID: \"d8e5e06f-6b18-4e06-be34-bc4df8d4bf87\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wjjsm" Jul 13 09:13:29.486728 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:29.486647 2552 patch_prober.go:28] interesting pod/image-registry-68c5fcfc4f-v2f5q container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 13 09:13:29.486728 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:29.486716 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" podUID="c0a4a73b-5bd3-4ff2-bd69-8a68bc640539" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 09:13:29.512876 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:29.512805 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wjjsm" Jul 13 09:13:29.648920 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:29.648897 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-wjjsm"] Jul 13 09:13:29.651167 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:13:29.651139 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8e5e06f_6b18_4e06_be34_bc4df8d4bf87.slice/crio-b5781fd9051b9de5d738588d1d4124ccc75bed933df33b2f16948a26278f5648 WatchSource:0}: Error finding container b5781fd9051b9de5d738588d1d4124ccc75bed933df33b2f16948a26278f5648: Status 404 returned error can't find the container with id b5781fd9051b9de5d738588d1d4124ccc75bed933df33b2f16948a26278f5648 Jul 13 09:13:30.007981 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:30.007923 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:30.573805 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:30.573720 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wjjsm" event={"ID":"d8e5e06f-6b18-4e06-be34-bc4df8d4bf87","Type":"ContainerStarted","Data":"b5781fd9051b9de5d738588d1d4124ccc75bed933df33b2f16948a26278f5648"} Jul 13 09:13:31.579069 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:31.579035 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wjjsm" event={"ID":"d8e5e06f-6b18-4e06-be34-bc4df8d4bf87","Type":"ContainerStarted","Data":"32005c0e929994559dde60e0cb7be1432f55d477b7724d1bfe1402a2a470191e"} Jul 13 09:13:31.579515 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:31.579282 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wjjsm" Jul 13 09:13:31.584078 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:31.584056 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wjjsm" Jul 13 09:13:31.597396 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:31.597355 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wjjsm" podStartSLOduration=2.250363495 podStartE2EDuration="3.597342775s" totalImagesPullingTime="1.34697928s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:13:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:13:31.596550222 +0000 UTC m=+295.476505490" watchObservedRunningTime="2026-07-13 09:13:31.597342775 +0000 UTC m=+295.477298055" Jul 13 09:13:31.661009 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:13:31.660982 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-nsd2z" podUID="a24643eb-b486-4118-9d57-77c6b07ef957" Jul 13 09:13:32.817609 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:32.817572 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:32.914372 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:32.914327 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:34.495304 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:34.495265 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" podUID="c0a4a73b-5bd3-4ff2-bd69-8a68bc640539" containerName="registry" containerID="cri-o://9f7dfce5cd9c22f18db3116028b974dbb03aa5a48c8b90661dfb6493c6cf2e86" gracePeriod=30 Jul 13 09:13:34.543605 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:34.543584 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-85jbg" Jul 13 09:13:34.733043 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:34.733014 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" Jul 13 09:13:34.773446 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:34.773385 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-registry-certificates\") pod \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\" (UID: \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\") " Jul 13 09:13:34.773446 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:34.773419 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zgspk\" (UniqueName: \"kubernetes.io/projected/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-kube-api-access-zgspk\") pod \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\" (UID: \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\") " Jul 13 09:13:34.773626 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:34.773450 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-trusted-ca\") pod \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\" (UID: \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\") " Jul 13 09:13:34.773626 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:34.773476 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-installation-pull-secrets\") pod \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\" (UID: \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\") " Jul 13 09:13:34.773626 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:34.773506 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-bound-sa-token\") pod \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\" (UID: \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\") " Jul 13 09:13:34.773626 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:34.773523 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-ca-trust-extracted\") pod \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\" (UID: \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\") " Jul 13 09:13:34.773626 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:34.773536 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-registry-tls\") pod \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\" (UID: \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\") " Jul 13 09:13:34.773626 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:34.773599 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-image-registry-private-configuration\") pod \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\" (UID: \"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539\") " Jul 13 09:13:34.774284 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:34.774056 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "c0a4a73b-5bd3-4ff2-bd69-8a68bc640539" (UID: "c0a4a73b-5bd3-4ff2-bd69-8a68bc640539"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 09:13:34.774638 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:34.774530 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "c0a4a73b-5bd3-4ff2-bd69-8a68bc640539" (UID: "c0a4a73b-5bd3-4ff2-bd69-8a68bc640539"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 09:13:34.778077 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:34.777896 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "c0a4a73b-5bd3-4ff2-bd69-8a68bc640539" (UID: "c0a4a73b-5bd3-4ff2-bd69-8a68bc640539"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 09:13:34.778077 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:34.777985 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c0a4a73b-5bd3-4ff2-bd69-8a68bc640539" (UID: "c0a4a73b-5bd3-4ff2-bd69-8a68bc640539"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:13:34.778077 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:34.778050 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "c0a4a73b-5bd3-4ff2-bd69-8a68bc640539" (UID: "c0a4a73b-5bd3-4ff2-bd69-8a68bc640539"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:13:34.781526 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:34.781199 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-kube-api-access-zgspk" (OuterVolumeSpecName: "kube-api-access-zgspk") pod "c0a4a73b-5bd3-4ff2-bd69-8a68bc640539" (UID: "c0a4a73b-5bd3-4ff2-bd69-8a68bc640539"). InnerVolumeSpecName "kube-api-access-zgspk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:13:34.781887 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:34.781705 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "c0a4a73b-5bd3-4ff2-bd69-8a68bc640539" (UID: "c0a4a73b-5bd3-4ff2-bd69-8a68bc640539"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 09:13:34.784715 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:34.784691 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "c0a4a73b-5bd3-4ff2-bd69-8a68bc640539" (UID: "c0a4a73b-5bd3-4ff2-bd69-8a68bc640539"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:13:34.875511 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:34.875489 2552 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-trusted-ca\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:13:34.875511 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:34.875510 2552 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-installation-pull-secrets\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:13:34.875646 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:34.875519 2552 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-bound-sa-token\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:13:34.875646 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:34.875528 2552 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-ca-trust-extracted\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:13:34.875646 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:34.875535 2552 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-registry-tls\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:13:34.875646 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:34.875544 2552 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-image-registry-private-configuration\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:13:34.875646 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:34.875552 2552 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-registry-certificates\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:13:34.875646 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:34.875560 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zgspk\" (UniqueName: \"kubernetes.io/projected/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539-kube-api-access-zgspk\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:13:35.091288 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:35.091261 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:35.192863 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:35.192834 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:35.265797 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:35.265774 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:35.595454 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:35.595430 2552 generic.go:356] "Generic (PLEG): container finished" podID="c0a4a73b-5bd3-4ff2-bd69-8a68bc640539" containerID="9f7dfce5cd9c22f18db3116028b974dbb03aa5a48c8b90661dfb6493c6cf2e86" exitCode=0 Jul 13 09:13:35.595836 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:35.595482 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" Jul 13 09:13:35.595836 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:35.595512 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" event={"ID":"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539","Type":"ContainerDied","Data":"9f7dfce5cd9c22f18db3116028b974dbb03aa5a48c8b90661dfb6493c6cf2e86"} Jul 13 09:13:35.595836 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:35.595582 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68c5fcfc4f-v2f5q" event={"ID":"c0a4a73b-5bd3-4ff2-bd69-8a68bc640539","Type":"ContainerDied","Data":"f919abd462f81eaa25983f83d30470e322dd3f48ff86edf2055221cdc7079c11"} Jul 13 09:13:35.595836 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:35.595605 2552 scope.go:117] "RemoveContainer" containerID="9f7dfce5cd9c22f18db3116028b974dbb03aa5a48c8b90661dfb6493c6cf2e86" Jul 13 09:13:35.606320 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:35.606296 2552 scope.go:117] "RemoveContainer" containerID="9f7dfce5cd9c22f18db3116028b974dbb03aa5a48c8b90661dfb6493c6cf2e86" Jul 13 09:13:35.606549 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:13:35.606528 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f7dfce5cd9c22f18db3116028b974dbb03aa5a48c8b90661dfb6493c6cf2e86\": container with ID starting with 9f7dfce5cd9c22f18db3116028b974dbb03aa5a48c8b90661dfb6493c6cf2e86 not found: ID does not exist" containerID="9f7dfce5cd9c22f18db3116028b974dbb03aa5a48c8b90661dfb6493c6cf2e86" Jul 13 09:13:35.606599 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:35.606556 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f7dfce5cd9c22f18db3116028b974dbb03aa5a48c8b90661dfb6493c6cf2e86"} err="failed to get container status \"9f7dfce5cd9c22f18db3116028b974dbb03aa5a48c8b90661dfb6493c6cf2e86\": rpc error: code = NotFound desc = could not find container \"9f7dfce5cd9c22f18db3116028b974dbb03aa5a48c8b90661dfb6493c6cf2e86\": container with ID starting with 9f7dfce5cd9c22f18db3116028b974dbb03aa5a48c8b90661dfb6493c6cf2e86 not found: ID does not exist" Jul 13 09:13:35.621244 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:35.621203 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-68c5fcfc4f-v2f5q"] Jul 13 09:13:35.624925 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:35.624906 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-68c5fcfc4f-v2f5q"] Jul 13 09:13:36.341325 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:36.341294 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:36.543638 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:36.543614 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gp6mv_47250d22-56b6-4503-af3f-36a9e79691b2/ovn-acl-logging/0.log" Jul 13 09:13:36.544694 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:36.544670 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gp6mv_47250d22-56b6-4503-af3f-36a9e79691b2/ovn-acl-logging/0.log" Jul 13 09:13:36.547450 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:36.547432 2552 kubelet.go:1648] "Image garbage collection succeeded" Jul 13 09:13:36.598564 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:36.598511 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:36.655849 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:36.655826 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:36.663761 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:36.663659 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0a4a73b-5bd3-4ff2-bd69-8a68bc640539" path="/var/lib/kubelet/pods/c0a4a73b-5bd3-4ff2-bd69-8a68bc640539/volumes" Jul 13 09:13:39.718919 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:39.718886 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:39.818558 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:39.818532 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:40.504432 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:40.504401 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7b86fc6955-24pwl"] Jul 13 09:13:40.504776 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:40.504760 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0a4a73b-5bd3-4ff2-bd69-8a68bc640539" containerName="registry" Jul 13 09:13:40.504854 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:40.504778 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0a4a73b-5bd3-4ff2-bd69-8a68bc640539" containerName="registry" Jul 13 09:13:40.504854 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:40.504833 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0a4a73b-5bd3-4ff2-bd69-8a68bc640539" containerName="registry" Jul 13 09:13:40.507497 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:40.507481 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b86fc6955-24pwl" Jul 13 09:13:40.515626 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:40.515603 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 13 09:13:40.516889 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:40.516866 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b86fc6955-24pwl"] Jul 13 09:13:40.613369 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:40.613349 2552 generic.go:356] "Generic (PLEG): container finished" podID="768513db-9c85-4e82-be90-b33e0ca71c94" containerID="4fc565df5251dec8b492f5a1ef4ebd6d6744eb1c6a5a343efda0e42c2baba10d" exitCode=255 Jul 13 09:13:40.613469 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:40.613375 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dd667957b-j9grk" event={"ID":"768513db-9c85-4e82-be90-b33e0ca71c94","Type":"ContainerDied","Data":"4fc565df5251dec8b492f5a1ef4ebd6d6744eb1c6a5a343efda0e42c2baba10d"} Jul 13 09:13:40.613469 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:40.613404 2552 scope.go:117] "RemoveContainer" containerID="ff11a8fecdeaf95fda4a224e858924c9c0d7ee68af32bd505d8a58084f4277ba" Jul 13 09:13:40.619301 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:40.619286 2552 scope.go:117] "RemoveContainer" containerID="4fc565df5251dec8b492f5a1ef4ebd6d6744eb1c6a5a343efda0e42c2baba10d" Jul 13 09:13:40.619450 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:13:40.619434 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-5dd667957b-j9grk_open-cluster-management-agent-addon(768513db-9c85-4e82-be90-b33e0ca71c94)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dd667957b-j9grk" podUID="768513db-9c85-4e82-be90-b33e0ca71c94" Jul 13 09:13:40.621451 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:40.621426 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/431de774-1fc9-40a4-a15f-e3e0f7ee9276-trusted-ca-bundle\") pod \"console-7b86fc6955-24pwl\" (UID: \"431de774-1fc9-40a4-a15f-e3e0f7ee9276\") " pod="openshift-console/console-7b86fc6955-24pwl" Jul 13 09:13:40.621524 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:40.621471 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/431de774-1fc9-40a4-a15f-e3e0f7ee9276-oauth-serving-cert\") pod \"console-7b86fc6955-24pwl\" (UID: \"431de774-1fc9-40a4-a15f-e3e0f7ee9276\") " pod="openshift-console/console-7b86fc6955-24pwl" Jul 13 09:13:40.621524 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:40.621495 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl8x5\" (UniqueName: \"kubernetes.io/projected/431de774-1fc9-40a4-a15f-e3e0f7ee9276-kube-api-access-kl8x5\") pod \"console-7b86fc6955-24pwl\" (UID: \"431de774-1fc9-40a4-a15f-e3e0f7ee9276\") " pod="openshift-console/console-7b86fc6955-24pwl" Jul 13 09:13:40.621605 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:40.621512 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/431de774-1fc9-40a4-a15f-e3e0f7ee9276-console-oauth-config\") pod \"console-7b86fc6955-24pwl\" (UID: \"431de774-1fc9-40a4-a15f-e3e0f7ee9276\") " pod="openshift-console/console-7b86fc6955-24pwl" Jul 13 09:13:40.621605 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:40.621540 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/431de774-1fc9-40a4-a15f-e3e0f7ee9276-console-config\") pod \"console-7b86fc6955-24pwl\" (UID: \"431de774-1fc9-40a4-a15f-e3e0f7ee9276\") " pod="openshift-console/console-7b86fc6955-24pwl" Jul 13 09:13:40.621605 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:40.621558 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/431de774-1fc9-40a4-a15f-e3e0f7ee9276-console-serving-cert\") pod \"console-7b86fc6955-24pwl\" (UID: \"431de774-1fc9-40a4-a15f-e3e0f7ee9276\") " pod="openshift-console/console-7b86fc6955-24pwl" Jul 13 09:13:40.621605 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:40.621574 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/431de774-1fc9-40a4-a15f-e3e0f7ee9276-service-ca\") pod \"console-7b86fc6955-24pwl\" (UID: \"431de774-1fc9-40a4-a15f-e3e0f7ee9276\") " pod="openshift-console/console-7b86fc6955-24pwl" Jul 13 09:13:40.637528 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:40.637508 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:40.722053 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:40.722016 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/431de774-1fc9-40a4-a15f-e3e0f7ee9276-service-ca\") pod \"console-7b86fc6955-24pwl\" (UID: \"431de774-1fc9-40a4-a15f-e3e0f7ee9276\") " pod="openshift-console/console-7b86fc6955-24pwl" Jul 13 09:13:40.722398 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:40.722076 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/431de774-1fc9-40a4-a15f-e3e0f7ee9276-trusted-ca-bundle\") pod \"console-7b86fc6955-24pwl\" (UID: \"431de774-1fc9-40a4-a15f-e3e0f7ee9276\") " pod="openshift-console/console-7b86fc6955-24pwl" Jul 13 09:13:40.722398 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:40.722124 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/431de774-1fc9-40a4-a15f-e3e0f7ee9276-oauth-serving-cert\") pod \"console-7b86fc6955-24pwl\" (UID: \"431de774-1fc9-40a4-a15f-e3e0f7ee9276\") " pod="openshift-console/console-7b86fc6955-24pwl" Jul 13 09:13:40.722398 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:40.722155 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl8x5\" (UniqueName: \"kubernetes.io/projected/431de774-1fc9-40a4-a15f-e3e0f7ee9276-kube-api-access-kl8x5\") pod \"console-7b86fc6955-24pwl\" (UID: \"431de774-1fc9-40a4-a15f-e3e0f7ee9276\") " pod="openshift-console/console-7b86fc6955-24pwl" Jul 13 09:13:40.722398 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:40.722186 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/431de774-1fc9-40a4-a15f-e3e0f7ee9276-console-oauth-config\") pod \"console-7b86fc6955-24pwl\" (UID: \"431de774-1fc9-40a4-a15f-e3e0f7ee9276\") " pod="openshift-console/console-7b86fc6955-24pwl" Jul 13 09:13:40.722398 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:40.722200 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/431de774-1fc9-40a4-a15f-e3e0f7ee9276-console-config\") pod \"console-7b86fc6955-24pwl\" (UID: \"431de774-1fc9-40a4-a15f-e3e0f7ee9276\") " pod="openshift-console/console-7b86fc6955-24pwl" Jul 13 09:13:40.722398 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:40.722243 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/431de774-1fc9-40a4-a15f-e3e0f7ee9276-console-serving-cert\") pod \"console-7b86fc6955-24pwl\" (UID: \"431de774-1fc9-40a4-a15f-e3e0f7ee9276\") " pod="openshift-console/console-7b86fc6955-24pwl" Jul 13 09:13:40.722825 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:40.722799 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/431de774-1fc9-40a4-a15f-e3e0f7ee9276-service-ca\") pod \"console-7b86fc6955-24pwl\" (UID: \"431de774-1fc9-40a4-a15f-e3e0f7ee9276\") " pod="openshift-console/console-7b86fc6955-24pwl" Jul 13 09:13:40.722992 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:40.722965 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/431de774-1fc9-40a4-a15f-e3e0f7ee9276-oauth-serving-cert\") pod \"console-7b86fc6955-24pwl\" (UID: \"431de774-1fc9-40a4-a15f-e3e0f7ee9276\") " pod="openshift-console/console-7b86fc6955-24pwl" Jul 13 09:13:40.722992 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:40.722980 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/431de774-1fc9-40a4-a15f-e3e0f7ee9276-trusted-ca-bundle\") pod \"console-7b86fc6955-24pwl\" (UID: \"431de774-1fc9-40a4-a15f-e3e0f7ee9276\") " pod="openshift-console/console-7b86fc6955-24pwl" Jul 13 09:13:40.723126 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:40.723049 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/431de774-1fc9-40a4-a15f-e3e0f7ee9276-console-config\") pod \"console-7b86fc6955-24pwl\" (UID: \"431de774-1fc9-40a4-a15f-e3e0f7ee9276\") " pod="openshift-console/console-7b86fc6955-24pwl" Jul 13 09:13:40.724629 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:40.724609 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/431de774-1fc9-40a4-a15f-e3e0f7ee9276-console-serving-cert\") pod \"console-7b86fc6955-24pwl\" (UID: \"431de774-1fc9-40a4-a15f-e3e0f7ee9276\") " pod="openshift-console/console-7b86fc6955-24pwl" Jul 13 09:13:40.724748 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:40.724729 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/431de774-1fc9-40a4-a15f-e3e0f7ee9276-console-oauth-config\") pod \"console-7b86fc6955-24pwl\" (UID: \"431de774-1fc9-40a4-a15f-e3e0f7ee9276\") " pod="openshift-console/console-7b86fc6955-24pwl" Jul 13 09:13:40.731204 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:40.731182 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl8x5\" (UniqueName: \"kubernetes.io/projected/431de774-1fc9-40a4-a15f-e3e0f7ee9276-kube-api-access-kl8x5\") pod \"console-7b86fc6955-24pwl\" (UID: \"431de774-1fc9-40a4-a15f-e3e0f7ee9276\") " pod="openshift-console/console-7b86fc6955-24pwl" Jul 13 09:13:40.766293 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:40.766270 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:40.816622 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:40.816598 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b86fc6955-24pwl" Jul 13 09:13:40.943461 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:40.943426 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b86fc6955-24pwl"] Jul 13 09:13:40.950097 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:13:40.950071 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod431de774_1fc9_40a4_a15f_e3e0f7ee9276.slice/crio-5a2cf60423b999d7e15f7d184119da26a8d1d3144d901bc5bdba3443b3e4c404 WatchSource:0}: Error finding container 5a2cf60423b999d7e15f7d184119da26a8d1d3144d901bc5bdba3443b3e4c404: Status 404 returned error can't find the container with id 5a2cf60423b999d7e15f7d184119da26a8d1d3144d901bc5bdba3443b3e4c404 Jul 13 09:13:41.619100 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:41.619064 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b86fc6955-24pwl" event={"ID":"431de774-1fc9-40a4-a15f-e3e0f7ee9276","Type":"ContainerStarted","Data":"326b016d24b1b07822612afc6d19802b215dae01fab066e89052e27ea3f7919d"} Jul 13 09:13:41.619304 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:41.619123 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b86fc6955-24pwl" event={"ID":"431de774-1fc9-40a4-a15f-e3e0f7ee9276","Type":"ContainerStarted","Data":"5a2cf60423b999d7e15f7d184119da26a8d1d3144d901bc5bdba3443b3e4c404"} Jul 13 09:13:41.641456 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:41.641409 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7b86fc6955-24pwl" podStartSLOduration=1.6413925 podStartE2EDuration="1.6413925s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:13:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:13:41.639960354 +0000 UTC m=+305.519915637" watchObservedRunningTime="2026-07-13 09:13:41.6413925 +0000 UTC m=+305.521347767" Jul 13 09:13:42.130460 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:42.130435 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:42.228975 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:42.228951 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:42.375404 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:42.375379 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:42.439173 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:42.439110 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:42.619186 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:42.619160 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:42.720110 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:42.720046 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:43.682093 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:43.682057 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:43.781936 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:43.781903 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:44.660281 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:44.660248 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsd2z" Jul 13 09:13:47.316013 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:47.315986 2552 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dd667957b-j9grk" Jul 13 09:13:47.316396 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:47.316271 2552 scope.go:117] "RemoveContainer" containerID="4fc565df5251dec8b492f5a1ef4ebd6d6744eb1c6a5a343efda0e42c2baba10d" Jul 13 09:13:47.316439 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:13:47.316394 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-5dd667957b-j9grk_open-cluster-management-agent-addon(768513db-9c85-4e82-be90-b33e0ca71c94)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dd667957b-j9grk" podUID="768513db-9c85-4e82-be90-b33e0ca71c94" Jul 13 09:13:48.799523 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:48.799488 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:48.870016 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:48.869989 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:49.821506 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:49.821467 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:49.896235 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:49.896184 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:50.081986 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:50.081964 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:50.817252 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:50.817224 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7b86fc6955-24pwl" Jul 13 09:13:50.817416 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:50.817277 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7b86fc6955-24pwl" Jul 13 09:13:50.821930 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:50.821907 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7b86fc6955-24pwl" Jul 13 09:13:51.666564 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:51.666530 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7b86fc6955-24pwl" Jul 13 09:13:51.841562 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:51.841531 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:51.942064 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:51.941997 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:52.015884 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:52.015863 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:52.685115 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:52.685084 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:52.747875 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:52.747847 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:13:53.652048 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:53.651999 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7787895976-9pm2h" podUID="5f8d3e22-5af7-4150-9f56-65e55d2e6713" containerName="console" containerID="cri-o://576ca4df9340acef8f4adf4fa84e94d24b6596e9172ecebe8411a6894ad6501a" gracePeriod=15 Jul 13 09:13:53.903462 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:53.903407 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7787895976-9pm2h" Jul 13 09:13:54.016305 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:54.016273 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5f8d3e22-5af7-4150-9f56-65e55d2e6713-console-config\") pod \"5f8d3e22-5af7-4150-9f56-65e55d2e6713\" (UID: \"5f8d3e22-5af7-4150-9f56-65e55d2e6713\") " Jul 13 09:13:54.016409 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:54.016324 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5f8d3e22-5af7-4150-9f56-65e55d2e6713-oauth-serving-cert\") pod \"5f8d3e22-5af7-4150-9f56-65e55d2e6713\" (UID: \"5f8d3e22-5af7-4150-9f56-65e55d2e6713\") " Jul 13 09:13:54.016409 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:54.016354 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5f8d3e22-5af7-4150-9f56-65e55d2e6713-console-serving-cert\") pod \"5f8d3e22-5af7-4150-9f56-65e55d2e6713\" (UID: \"5f8d3e22-5af7-4150-9f56-65e55d2e6713\") " Jul 13 09:13:54.016409 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:54.016370 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5f8d3e22-5af7-4150-9f56-65e55d2e6713-service-ca\") pod \"5f8d3e22-5af7-4150-9f56-65e55d2e6713\" (UID: \"5f8d3e22-5af7-4150-9f56-65e55d2e6713\") " Jul 13 09:13:54.016409 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:54.016407 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5f8d3e22-5af7-4150-9f56-65e55d2e6713-console-oauth-config\") pod \"5f8d3e22-5af7-4150-9f56-65e55d2e6713\" (UID: \"5f8d3e22-5af7-4150-9f56-65e55d2e6713\") " Jul 13 09:13:54.016664 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:54.016457 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6nw6t\" (UniqueName: \"kubernetes.io/projected/5f8d3e22-5af7-4150-9f56-65e55d2e6713-kube-api-access-6nw6t\") pod \"5f8d3e22-5af7-4150-9f56-65e55d2e6713\" (UID: \"5f8d3e22-5af7-4150-9f56-65e55d2e6713\") " Jul 13 09:13:54.016735 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:54.016707 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f8d3e22-5af7-4150-9f56-65e55d2e6713-console-config" (OuterVolumeSpecName: "console-config") pod "5f8d3e22-5af7-4150-9f56-65e55d2e6713" (UID: "5f8d3e22-5af7-4150-9f56-65e55d2e6713"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 09:13:54.016787 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:54.016766 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f8d3e22-5af7-4150-9f56-65e55d2e6713-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5f8d3e22-5af7-4150-9f56-65e55d2e6713" (UID: "5f8d3e22-5af7-4150-9f56-65e55d2e6713"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 09:13:54.016839 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:54.016796 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f8d3e22-5af7-4150-9f56-65e55d2e6713-service-ca" (OuterVolumeSpecName: "service-ca") pod "5f8d3e22-5af7-4150-9f56-65e55d2e6713" (UID: "5f8d3e22-5af7-4150-9f56-65e55d2e6713"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 09:13:54.018642 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:54.018606 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f8d3e22-5af7-4150-9f56-65e55d2e6713-kube-api-access-6nw6t" (OuterVolumeSpecName: "kube-api-access-6nw6t") pod "5f8d3e22-5af7-4150-9f56-65e55d2e6713" (UID: "5f8d3e22-5af7-4150-9f56-65e55d2e6713"). InnerVolumeSpecName "kube-api-access-6nw6t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:13:54.018642 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:54.018622 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f8d3e22-5af7-4150-9f56-65e55d2e6713-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5f8d3e22-5af7-4150-9f56-65e55d2e6713" (UID: "5f8d3e22-5af7-4150-9f56-65e55d2e6713"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 09:13:54.018771 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:54.018667 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f8d3e22-5af7-4150-9f56-65e55d2e6713-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5f8d3e22-5af7-4150-9f56-65e55d2e6713" (UID: "5f8d3e22-5af7-4150-9f56-65e55d2e6713"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 09:13:54.117142 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:54.117111 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nw6t\" (UniqueName: \"kubernetes.io/projected/5f8d3e22-5af7-4150-9f56-65e55d2e6713-kube-api-access-6nw6t\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:13:54.117142 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:54.117139 2552 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5f8d3e22-5af7-4150-9f56-65e55d2e6713-console-config\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:13:54.117142 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:54.117156 2552 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5f8d3e22-5af7-4150-9f56-65e55d2e6713-oauth-serving-cert\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:13:54.117336 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:54.117166 2552 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5f8d3e22-5af7-4150-9f56-65e55d2e6713-console-serving-cert\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:13:54.117336 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:54.117174 2552 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5f8d3e22-5af7-4150-9f56-65e55d2e6713-service-ca\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:13:54.117336 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:54.117182 2552 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5f8d3e22-5af7-4150-9f56-65e55d2e6713-console-oauth-config\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:13:54.669732 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:54.669699 2552 generic.go:356] "Generic (PLEG): container finished" podID="5f8d3e22-5af7-4150-9f56-65e55d2e6713" containerID="576ca4df9340acef8f4adf4fa84e94d24b6596e9172ecebe8411a6894ad6501a" exitCode=0 Jul 13 09:13:54.670172 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:54.669746 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7787895976-9pm2h" Jul 13 09:13:54.670172 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:54.669763 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7787895976-9pm2h" event={"ID":"5f8d3e22-5af7-4150-9f56-65e55d2e6713","Type":"ContainerDied","Data":"576ca4df9340acef8f4adf4fa84e94d24b6596e9172ecebe8411a6894ad6501a"} Jul 13 09:13:54.670172 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:54.669800 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7787895976-9pm2h" event={"ID":"5f8d3e22-5af7-4150-9f56-65e55d2e6713","Type":"ContainerDied","Data":"534560b8b633e6422472c6a629a06e716f4d8916dfd20e51de5f206a5a5db6bf"} Jul 13 09:13:54.670172 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:54.669819 2552 scope.go:117] "RemoveContainer" containerID="576ca4df9340acef8f4adf4fa84e94d24b6596e9172ecebe8411a6894ad6501a" Jul 13 09:13:54.678035 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:54.678018 2552 scope.go:117] "RemoveContainer" containerID="576ca4df9340acef8f4adf4fa84e94d24b6596e9172ecebe8411a6894ad6501a" Jul 13 09:13:54.678329 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:13:54.678310 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"576ca4df9340acef8f4adf4fa84e94d24b6596e9172ecebe8411a6894ad6501a\": container with ID starting with 576ca4df9340acef8f4adf4fa84e94d24b6596e9172ecebe8411a6894ad6501a not found: ID does not exist" containerID="576ca4df9340acef8f4adf4fa84e94d24b6596e9172ecebe8411a6894ad6501a" Jul 13 09:13:54.678400 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:54.678337 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"576ca4df9340acef8f4adf4fa84e94d24b6596e9172ecebe8411a6894ad6501a"} err="failed to get container status \"576ca4df9340acef8f4adf4fa84e94d24b6596e9172ecebe8411a6894ad6501a\": rpc error: code = NotFound desc = could not find container \"576ca4df9340acef8f4adf4fa84e94d24b6596e9172ecebe8411a6894ad6501a\": container with ID starting with 576ca4df9340acef8f4adf4fa84e94d24b6596e9172ecebe8411a6894ad6501a not found: ID does not exist" Jul 13 09:13:54.687729 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:54.687708 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7787895976-9pm2h"] Jul 13 09:13:54.691419 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:54.691400 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7787895976-9pm2h"] Jul 13 09:13:56.664113 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:56.664080 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f8d3e22-5af7-4150-9f56-65e55d2e6713" path="/var/lib/kubelet/pods/5f8d3e22-5af7-4150-9f56-65e55d2e6713/volumes" Jul 13 09:13:57.666099 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:57.666077 2552 scope.go:117] "RemoveContainer" containerID="4fc565df5251dec8b492f5a1ef4ebd6d6744eb1c6a5a343efda0e42c2baba10d" Jul 13 09:13:58.685811 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:13:58.685783 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dd667957b-j9grk" event={"ID":"768513db-9c85-4e82-be90-b33e0ca71c94","Type":"ContainerStarted","Data":"38135ee2624fee34186b8aaa65c914036ab8743aea5a1ec40debc9e523175c5b"} Jul 13 09:14:06.603273 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:06.603244 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:14:06.672134 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:06.672107 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:14:39.905981 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:39.903955 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:14:39.998161 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:39.998130 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:14:47.014678 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:47.014647 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:14:47.115976 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:47.115946 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:14:48.951923 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:48.951893 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-678945cf5c-7lrjp"] Jul 13 09:14:48.952299 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:48.952137 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f8d3e22-5af7-4150-9f56-65e55d2e6713" containerName="console" Jul 13 09:14:48.952299 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:48.952148 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f8d3e22-5af7-4150-9f56-65e55d2e6713" containerName="console" Jul 13 09:14:48.952299 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:48.952206 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f8d3e22-5af7-4150-9f56-65e55d2e6713" containerName="console" Jul 13 09:14:48.956293 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:48.956273 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-678945cf5c-7lrjp" Jul 13 09:14:48.966230 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:48.966180 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-678945cf5c-7lrjp"] Jul 13 09:14:49.015886 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:49.015863 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:14:49.071162 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:49.071128 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:14:49.092247 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:49.092201 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-console-config\") pod \"console-678945cf5c-7lrjp\" (UID: \"af6e4f34-3bd7-46f2-95dc-f99cc52699f3\") " pod="openshift-console/console-678945cf5c-7lrjp" Jul 13 09:14:49.092345 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:49.092267 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-console-oauth-config\") pod \"console-678945cf5c-7lrjp\" (UID: \"af6e4f34-3bd7-46f2-95dc-f99cc52699f3\") " pod="openshift-console/console-678945cf5c-7lrjp" Jul 13 09:14:49.092345 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:49.092301 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-service-ca\") pod \"console-678945cf5c-7lrjp\" (UID: \"af6e4f34-3bd7-46f2-95dc-f99cc52699f3\") " pod="openshift-console/console-678945cf5c-7lrjp" Jul 13 09:14:49.092345 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:49.092332 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99lmc\" (UniqueName: \"kubernetes.io/projected/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-kube-api-access-99lmc\") pod \"console-678945cf5c-7lrjp\" (UID: \"af6e4f34-3bd7-46f2-95dc-f99cc52699f3\") " pod="openshift-console/console-678945cf5c-7lrjp" Jul 13 09:14:49.092458 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:49.092405 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-console-serving-cert\") pod \"console-678945cf5c-7lrjp\" (UID: \"af6e4f34-3bd7-46f2-95dc-f99cc52699f3\") " pod="openshift-console/console-678945cf5c-7lrjp" Jul 13 09:14:49.092458 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:49.092445 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-oauth-serving-cert\") pod \"console-678945cf5c-7lrjp\" (UID: \"af6e4f34-3bd7-46f2-95dc-f99cc52699f3\") " pod="openshift-console/console-678945cf5c-7lrjp" Jul 13 09:14:49.092527 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:49.092474 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-trusted-ca-bundle\") pod \"console-678945cf5c-7lrjp\" (UID: \"af6e4f34-3bd7-46f2-95dc-f99cc52699f3\") " pod="openshift-console/console-678945cf5c-7lrjp" Jul 13 09:14:49.193456 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:49.193432 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-trusted-ca-bundle\") pod \"console-678945cf5c-7lrjp\" (UID: \"af6e4f34-3bd7-46f2-95dc-f99cc52699f3\") " pod="openshift-console/console-678945cf5c-7lrjp" Jul 13 09:14:49.193537 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:49.193467 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-console-config\") pod \"console-678945cf5c-7lrjp\" (UID: \"af6e4f34-3bd7-46f2-95dc-f99cc52699f3\") " pod="openshift-console/console-678945cf5c-7lrjp" Jul 13 09:14:49.193537 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:49.193490 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-console-oauth-config\") pod \"console-678945cf5c-7lrjp\" (UID: \"af6e4f34-3bd7-46f2-95dc-f99cc52699f3\") " pod="openshift-console/console-678945cf5c-7lrjp" Jul 13 09:14:49.193537 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:49.193509 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-service-ca\") pod \"console-678945cf5c-7lrjp\" (UID: \"af6e4f34-3bd7-46f2-95dc-f99cc52699f3\") " pod="openshift-console/console-678945cf5c-7lrjp" Jul 13 09:14:49.193638 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:49.193558 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99lmc\" (UniqueName: \"kubernetes.io/projected/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-kube-api-access-99lmc\") pod \"console-678945cf5c-7lrjp\" (UID: \"af6e4f34-3bd7-46f2-95dc-f99cc52699f3\") " pod="openshift-console/console-678945cf5c-7lrjp" Jul 13 09:14:49.193638 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:49.193594 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-console-serving-cert\") pod \"console-678945cf5c-7lrjp\" (UID: \"af6e4f34-3bd7-46f2-95dc-f99cc52699f3\") " pod="openshift-console/console-678945cf5c-7lrjp" Jul 13 09:14:49.193638 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:49.193629 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-oauth-serving-cert\") pod \"console-678945cf5c-7lrjp\" (UID: \"af6e4f34-3bd7-46f2-95dc-f99cc52699f3\") " pod="openshift-console/console-678945cf5c-7lrjp" Jul 13 09:14:49.194149 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:49.194126 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-console-config\") pod \"console-678945cf5c-7lrjp\" (UID: \"af6e4f34-3bd7-46f2-95dc-f99cc52699f3\") " pod="openshift-console/console-678945cf5c-7lrjp" Jul 13 09:14:49.194352 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:49.194329 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-service-ca\") pod \"console-678945cf5c-7lrjp\" (UID: \"af6e4f34-3bd7-46f2-95dc-f99cc52699f3\") " pod="openshift-console/console-678945cf5c-7lrjp" Jul 13 09:14:49.194424 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:49.194348 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-oauth-serving-cert\") pod \"console-678945cf5c-7lrjp\" (UID: \"af6e4f34-3bd7-46f2-95dc-f99cc52699f3\") " pod="openshift-console/console-678945cf5c-7lrjp" Jul 13 09:14:49.194424 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:49.194394 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-trusted-ca-bundle\") pod \"console-678945cf5c-7lrjp\" (UID: \"af6e4f34-3bd7-46f2-95dc-f99cc52699f3\") " pod="openshift-console/console-678945cf5c-7lrjp" Jul 13 09:14:49.196137 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:49.196114 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-console-oauth-config\") pod \"console-678945cf5c-7lrjp\" (UID: \"af6e4f34-3bd7-46f2-95dc-f99cc52699f3\") " pod="openshift-console/console-678945cf5c-7lrjp" Jul 13 09:14:49.196227 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:49.196194 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-console-serving-cert\") pod \"console-678945cf5c-7lrjp\" (UID: \"af6e4f34-3bd7-46f2-95dc-f99cc52699f3\") " pod="openshift-console/console-678945cf5c-7lrjp" Jul 13 09:14:49.202347 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:49.202296 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99lmc\" (UniqueName: \"kubernetes.io/projected/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-kube-api-access-99lmc\") pod \"console-678945cf5c-7lrjp\" (UID: \"af6e4f34-3bd7-46f2-95dc-f99cc52699f3\") " pod="openshift-console/console-678945cf5c-7lrjp" Jul 13 09:14:49.265959 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:49.265937 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-678945cf5c-7lrjp" Jul 13 09:14:49.393815 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:49.393781 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-678945cf5c-7lrjp"] Jul 13 09:14:49.396426 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:14:49.396399 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf6e4f34_3bd7_46f2_95dc_f99cc52699f3.slice/crio-42683b82c92e70f0eeeaeab33eb7622e897de793195f3a79bc3f8e574be03359 WatchSource:0}: Error finding container 42683b82c92e70f0eeeaeab33eb7622e897de793195f3a79bc3f8e574be03359: Status 404 returned error can't find the container with id 42683b82c92e70f0eeeaeab33eb7622e897de793195f3a79bc3f8e574be03359 Jul 13 09:14:49.860526 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:49.860497 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-678945cf5c-7lrjp" event={"ID":"af6e4f34-3bd7-46f2-95dc-f99cc52699f3","Type":"ContainerStarted","Data":"2ac03e5941a052876ee137f1bae559e0150ac08b7e2ef089ab62ce3c614319f6"} Jul 13 09:14:49.860676 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:49.860542 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-678945cf5c-7lrjp" event={"ID":"af6e4f34-3bd7-46f2-95dc-f99cc52699f3","Type":"ContainerStarted","Data":"42683b82c92e70f0eeeaeab33eb7622e897de793195f3a79bc3f8e574be03359"} Jul 13 09:14:49.880941 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:49.880894 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-678945cf5c-7lrjp" podStartSLOduration=1.880879376 podStartE2EDuration="1.880879376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:14:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:14:49.879925909 +0000 UTC m=+373.759881200" watchObservedRunningTime="2026-07-13 09:14:49.880879376 +0000 UTC m=+373.760834642" Jul 13 09:14:50.705445 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:50.705413 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a24643eb-b486-4118-9d57-77c6b07ef957-metrics-certs\") pod \"network-metrics-daemon-nsd2z\" (UID: \"a24643eb-b486-4118-9d57-77c6b07ef957\") " pod="openshift-multus/network-metrics-daemon-nsd2z" Jul 13 09:14:50.707803 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:50.707783 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a24643eb-b486-4118-9d57-77c6b07ef957-metrics-certs\") pod \"network-metrics-daemon-nsd2z\" (UID: \"a24643eb-b486-4118-9d57-77c6b07ef957\") " pod="openshift-multus/network-metrics-daemon-nsd2z" Jul 13 09:14:50.962996 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:50.962933 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zlq2v\"" Jul 13 09:14:50.971247 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:50.971206 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsd2z" Jul 13 09:14:51.091284 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:51.091255 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nsd2z"] Jul 13 09:14:51.094088 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:14:51.094046 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda24643eb_b486_4118_9d57_77c6b07ef957.slice/crio-4c037f7fc0255321832646237045712d775ef36c6ce19b5f8cd7da3cf9f1cc4b WatchSource:0}: Error finding container 4c037f7fc0255321832646237045712d775ef36c6ce19b5f8cd7da3cf9f1cc4b: Status 404 returned error can't find the container with id 4c037f7fc0255321832646237045712d775ef36c6ce19b5f8cd7da3cf9f1cc4b Jul 13 09:14:51.096316 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:51.096300 2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 09:14:51.770095 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:51.770066 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:14:51.870137 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:51.870093 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nsd2z" event={"ID":"a24643eb-b486-4118-9d57-77c6b07ef957","Type":"ContainerStarted","Data":"4c037f7fc0255321832646237045712d775ef36c6ce19b5f8cd7da3cf9f1cc4b"} Jul 13 09:14:51.870928 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:51.870886 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:14:52.875554 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:52.875518 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nsd2z" event={"ID":"a24643eb-b486-4118-9d57-77c6b07ef957","Type":"ContainerStarted","Data":"de5bdad2b1b0bbe0f30b6b0f3b9984a7c34ff6b94c33196f3d65689f764d8bf6"} Jul 13 09:14:52.876011 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:52.875574 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nsd2z" event={"ID":"a24643eb-b486-4118-9d57-77c6b07ef957","Type":"ContainerStarted","Data":"7d02d26cd9d481afb5e5ba9851a6706ed42218cfe74435cb54995c88a97b82af"} Jul 13 09:14:52.892462 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:52.892404 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-nsd2z" podStartSLOduration=374.958865601 podStartE2EDuration="6m15.892388855s" totalImagesPullingTime="933.523254ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:08:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:14:52.891032511 +0000 UTC m=+376.770987779" watchObservedRunningTime="2026-07-13 09:14:52.892388855 +0000 UTC m=+376.772344122" Jul 13 09:14:52.983038 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:52.983007 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:14:53.083591 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:53.083565 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:14:59.266294 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:59.266253 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-678945cf5c-7lrjp" Jul 13 09:14:59.266733 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:59.266319 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-678945cf5c-7lrjp" Jul 13 09:14:59.271353 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:59.271330 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-678945cf5c-7lrjp" Jul 13 09:14:59.904118 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:59.904090 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-678945cf5c-7lrjp" Jul 13 09:14:59.956273 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:14:59.956242 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b86fc6955-24pwl"] Jul 13 09:15:00.064865 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:00.064815 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:15:00.120120 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:00.120094 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:15:00.191181 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:00.191112 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:15:01.479615 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:01.479585 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:15:01.561095 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:01.561069 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:15:06.604285 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:06.604258 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:15:06.706059 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:06.706030 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:15:24.979781 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:24.979733 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7b86fc6955-24pwl" podUID="431de774-1fc9-40a4-a15f-e3e0f7ee9276" containerName="console" containerID="cri-o://326b016d24b1b07822612afc6d19802b215dae01fab066e89052e27ea3f7919d" gracePeriod=15 Jul 13 09:15:25.226253 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:25.226228 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b86fc6955-24pwl" Jul 13 09:15:25.341098 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:25.341072 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/431de774-1fc9-40a4-a15f-e3e0f7ee9276-console-oauth-config\") pod \"431de774-1fc9-40a4-a15f-e3e0f7ee9276\" (UID: \"431de774-1fc9-40a4-a15f-e3e0f7ee9276\") " Jul 13 09:15:25.341237 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:25.341112 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/431de774-1fc9-40a4-a15f-e3e0f7ee9276-console-serving-cert\") pod \"431de774-1fc9-40a4-a15f-e3e0f7ee9276\" (UID: \"431de774-1fc9-40a4-a15f-e3e0f7ee9276\") " Jul 13 09:15:25.341237 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:25.341146 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/431de774-1fc9-40a4-a15f-e3e0f7ee9276-trusted-ca-bundle\") pod \"431de774-1fc9-40a4-a15f-e3e0f7ee9276\" (UID: \"431de774-1fc9-40a4-a15f-e3e0f7ee9276\") " Jul 13 09:15:25.341237 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:25.341162 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/431de774-1fc9-40a4-a15f-e3e0f7ee9276-service-ca\") pod \"431de774-1fc9-40a4-a15f-e3e0f7ee9276\" (UID: \"431de774-1fc9-40a4-a15f-e3e0f7ee9276\") " Jul 13 09:15:25.341237 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:25.341191 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/431de774-1fc9-40a4-a15f-e3e0f7ee9276-oauth-serving-cert\") pod \"431de774-1fc9-40a4-a15f-e3e0f7ee9276\" (UID: \"431de774-1fc9-40a4-a15f-e3e0f7ee9276\") " Jul 13 09:15:25.341237 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:25.341207 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kl8x5\" (UniqueName: \"kubernetes.io/projected/431de774-1fc9-40a4-a15f-e3e0f7ee9276-kube-api-access-kl8x5\") pod \"431de774-1fc9-40a4-a15f-e3e0f7ee9276\" (UID: \"431de774-1fc9-40a4-a15f-e3e0f7ee9276\") " Jul 13 09:15:25.341487 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:25.341257 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/431de774-1fc9-40a4-a15f-e3e0f7ee9276-console-config\") pod \"431de774-1fc9-40a4-a15f-e3e0f7ee9276\" (UID: \"431de774-1fc9-40a4-a15f-e3e0f7ee9276\") " Jul 13 09:15:25.341592 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:25.341563 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/431de774-1fc9-40a4-a15f-e3e0f7ee9276-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "431de774-1fc9-40a4-a15f-e3e0f7ee9276" (UID: "431de774-1fc9-40a4-a15f-e3e0f7ee9276"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 09:15:25.341716 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:25.341637 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/431de774-1fc9-40a4-a15f-e3e0f7ee9276-service-ca" (OuterVolumeSpecName: "service-ca") pod "431de774-1fc9-40a4-a15f-e3e0f7ee9276" (UID: "431de774-1fc9-40a4-a15f-e3e0f7ee9276"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 09:15:25.341716 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:25.341654 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/431de774-1fc9-40a4-a15f-e3e0f7ee9276-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "431de774-1fc9-40a4-a15f-e3e0f7ee9276" (UID: "431de774-1fc9-40a4-a15f-e3e0f7ee9276"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 09:15:25.341716 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:25.341678 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/431de774-1fc9-40a4-a15f-e3e0f7ee9276-console-config" (OuterVolumeSpecName: "console-config") pod "431de774-1fc9-40a4-a15f-e3e0f7ee9276" (UID: "431de774-1fc9-40a4-a15f-e3e0f7ee9276"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 09:15:25.343291 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:25.343254 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/431de774-1fc9-40a4-a15f-e3e0f7ee9276-kube-api-access-kl8x5" (OuterVolumeSpecName: "kube-api-access-kl8x5") pod "431de774-1fc9-40a4-a15f-e3e0f7ee9276" (UID: "431de774-1fc9-40a4-a15f-e3e0f7ee9276"). InnerVolumeSpecName "kube-api-access-kl8x5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:15:25.343431 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:25.343407 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/431de774-1fc9-40a4-a15f-e3e0f7ee9276-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "431de774-1fc9-40a4-a15f-e3e0f7ee9276" (UID: "431de774-1fc9-40a4-a15f-e3e0f7ee9276"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 09:15:25.343473 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:25.343418 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/431de774-1fc9-40a4-a15f-e3e0f7ee9276-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "431de774-1fc9-40a4-a15f-e3e0f7ee9276" (UID: "431de774-1fc9-40a4-a15f-e3e0f7ee9276"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 09:15:25.441877 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:25.441856 2552 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/431de774-1fc9-40a4-a15f-e3e0f7ee9276-console-oauth-config\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:15:25.441877 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:25.441876 2552 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/431de774-1fc9-40a4-a15f-e3e0f7ee9276-console-serving-cert\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:15:25.441997 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:25.441885 2552 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/431de774-1fc9-40a4-a15f-e3e0f7ee9276-trusted-ca-bundle\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:15:25.441997 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:25.441894 2552 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/431de774-1fc9-40a4-a15f-e3e0f7ee9276-service-ca\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:15:25.441997 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:25.441903 2552 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/431de774-1fc9-40a4-a15f-e3e0f7ee9276-oauth-serving-cert\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:15:25.441997 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:25.441913 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kl8x5\" (UniqueName: \"kubernetes.io/projected/431de774-1fc9-40a4-a15f-e3e0f7ee9276-kube-api-access-kl8x5\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:15:25.441997 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:25.441921 2552 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/431de774-1fc9-40a4-a15f-e3e0f7ee9276-console-config\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:15:25.996595 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:25.996566 2552 generic.go:356] "Generic (PLEG): container finished" podID="431de774-1fc9-40a4-a15f-e3e0f7ee9276" containerID="326b016d24b1b07822612afc6d19802b215dae01fab066e89052e27ea3f7919d" exitCode=0 Jul 13 09:15:25.996949 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:25.996615 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b86fc6955-24pwl" Jul 13 09:15:25.996949 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:25.996623 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b86fc6955-24pwl" event={"ID":"431de774-1fc9-40a4-a15f-e3e0f7ee9276","Type":"ContainerDied","Data":"326b016d24b1b07822612afc6d19802b215dae01fab066e89052e27ea3f7919d"} Jul 13 09:15:25.996949 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:25.996665 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b86fc6955-24pwl" event={"ID":"431de774-1fc9-40a4-a15f-e3e0f7ee9276","Type":"ContainerDied","Data":"5a2cf60423b999d7e15f7d184119da26a8d1d3144d901bc5bdba3443b3e4c404"} Jul 13 09:15:25.996949 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:25.996685 2552 scope.go:117] "RemoveContainer" containerID="326b016d24b1b07822612afc6d19802b215dae01fab066e89052e27ea3f7919d" Jul 13 09:15:26.006648 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:26.006628 2552 scope.go:117] "RemoveContainer" containerID="326b016d24b1b07822612afc6d19802b215dae01fab066e89052e27ea3f7919d" Jul 13 09:15:26.006910 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:15:26.006889 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"326b016d24b1b07822612afc6d19802b215dae01fab066e89052e27ea3f7919d\": container with ID starting with 326b016d24b1b07822612afc6d19802b215dae01fab066e89052e27ea3f7919d not found: ID does not exist" containerID="326b016d24b1b07822612afc6d19802b215dae01fab066e89052e27ea3f7919d" Jul 13 09:15:26.006964 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:26.006918 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"326b016d24b1b07822612afc6d19802b215dae01fab066e89052e27ea3f7919d"} err="failed to get container status \"326b016d24b1b07822612afc6d19802b215dae01fab066e89052e27ea3f7919d\": rpc error: code = NotFound desc = could not find container \"326b016d24b1b07822612afc6d19802b215dae01fab066e89052e27ea3f7919d\": container with ID starting with 326b016d24b1b07822612afc6d19802b215dae01fab066e89052e27ea3f7919d not found: ID does not exist" Jul 13 09:15:26.019194 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:26.019171 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b86fc6955-24pwl"] Jul 13 09:15:26.023295 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:26.023274 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7b86fc6955-24pwl"] Jul 13 09:15:26.665165 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:26.665131 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="431de774-1fc9-40a4-a15f-e3e0f7ee9276" path="/var/lib/kubelet/pods/431de774-1fc9-40a4-a15f-e3e0f7ee9276/volumes" Jul 13 09:15:44.669489 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:44.669455 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:15:44.728395 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:44.728367 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:15:54.417509 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:54.417474 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:15:54.523788 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:54.523762 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:15:55.005227 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:55.005186 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:15:55.118782 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:55.118748 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6m6j"] Jul 13 09:15:55.119092 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:55.119076 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="431de774-1fc9-40a4-a15f-e3e0f7ee9276" containerName="console" Jul 13 09:15:55.119182 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:55.119095 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="431de774-1fc9-40a4-a15f-e3e0f7ee9276" containerName="console" Jul 13 09:15:55.119182 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:55.119177 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="431de774-1fc9-40a4-a15f-e3e0f7ee9276" containerName="console" Jul 13 09:15:55.123479 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:55.123458 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6m6j" Jul 13 09:15:55.126303 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:55.126285 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 13 09:15:55.126395 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:55.126307 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-77zzn\"" Jul 13 09:15:55.128355 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:55.128334 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 13 09:15:55.134627 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:55.134606 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6m6j"] Jul 13 09:15:55.246424 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:55.246399 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a44db9bd-1992-44ff-ab0b-9ebc178ff2ed-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6m6j\" (UID: \"a44db9bd-1992-44ff-ab0b-9ebc178ff2ed\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6m6j" Jul 13 09:15:55.246538 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:55.246436 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a44db9bd-1992-44ff-ab0b-9ebc178ff2ed-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6m6j\" (UID: \"a44db9bd-1992-44ff-ab0b-9ebc178ff2ed\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6m6j" Jul 13 09:15:55.246538 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:55.246467 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wl75z\" (UniqueName: \"kubernetes.io/projected/a44db9bd-1992-44ff-ab0b-9ebc178ff2ed-kube-api-access-wl75z\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6m6j\" (UID: \"a44db9bd-1992-44ff-ab0b-9ebc178ff2ed\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6m6j" Jul 13 09:15:55.346770 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:55.346749 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a44db9bd-1992-44ff-ab0b-9ebc178ff2ed-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6m6j\" (UID: \"a44db9bd-1992-44ff-ab0b-9ebc178ff2ed\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6m6j" Jul 13 09:15:55.346934 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:55.346784 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a44db9bd-1992-44ff-ab0b-9ebc178ff2ed-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6m6j\" (UID: \"a44db9bd-1992-44ff-ab0b-9ebc178ff2ed\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6m6j" Jul 13 09:15:55.346934 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:55.346805 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl75z\" (UniqueName: \"kubernetes.io/projected/a44db9bd-1992-44ff-ab0b-9ebc178ff2ed-kube-api-access-wl75z\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6m6j\" (UID: \"a44db9bd-1992-44ff-ab0b-9ebc178ff2ed\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6m6j" Jul 13 09:15:55.347173 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:55.347154 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a44db9bd-1992-44ff-ab0b-9ebc178ff2ed-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6m6j\" (UID: \"a44db9bd-1992-44ff-ab0b-9ebc178ff2ed\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6m6j" Jul 13 09:15:55.347173 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:55.347165 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a44db9bd-1992-44ff-ab0b-9ebc178ff2ed-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6m6j\" (UID: \"a44db9bd-1992-44ff-ab0b-9ebc178ff2ed\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6m6j" Jul 13 09:15:55.356145 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:55.356125 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wl75z\" (UniqueName: \"kubernetes.io/projected/a44db9bd-1992-44ff-ab0b-9ebc178ff2ed-kube-api-access-wl75z\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6m6j\" (UID: \"a44db9bd-1992-44ff-ab0b-9ebc178ff2ed\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6m6j" Jul 13 09:15:55.432096 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:55.432077 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6m6j" Jul 13 09:15:55.554404 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:55.554369 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6m6j"] Jul 13 09:15:55.556855 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:15:55.556822 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda44db9bd_1992_44ff_ab0b_9ebc178ff2ed.slice/crio-9733fdd21f4c33db4d493b9c4f854208acbc0e75852eeb72b78c65585615f2ef WatchSource:0}: Error finding container 9733fdd21f4c33db4d493b9c4f854208acbc0e75852eeb72b78c65585615f2ef: Status 404 returned error can't find the container with id 9733fdd21f4c33db4d493b9c4f854208acbc0e75852eeb72b78c65585615f2ef Jul 13 09:15:56.107442 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:15:56.107396 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6m6j" event={"ID":"a44db9bd-1992-44ff-ab0b-9ebc178ff2ed","Type":"ContainerStarted","Data":"9733fdd21f4c33db4d493b9c4f854208acbc0e75852eeb72b78c65585615f2ef"} Jul 13 09:16:03.134326 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:03.134288 2552 generic.go:356] "Generic (PLEG): container finished" podID="a44db9bd-1992-44ff-ab0b-9ebc178ff2ed" containerID="f6dbf4ab1e6084aa2ed4744f66490a52b275a690153eee75b87c4a7b7865a078" exitCode=0 Jul 13 09:16:03.134701 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:03.134372 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6m6j" event={"ID":"a44db9bd-1992-44ff-ab0b-9ebc178ff2ed","Type":"ContainerDied","Data":"f6dbf4ab1e6084aa2ed4744f66490a52b275a690153eee75b87c4a7b7865a078"} Jul 13 09:16:06.148045 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:06.148013 2552 generic.go:356] "Generic (PLEG): container finished" podID="a44db9bd-1992-44ff-ab0b-9ebc178ff2ed" containerID="7c859fdb977b8424b67763c93a86808147652208a1b17fb1603229722cc336ed" exitCode=0 Jul 13 09:16:06.148413 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:06.148101 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6m6j" event={"ID":"a44db9bd-1992-44ff-ab0b-9ebc178ff2ed","Type":"ContainerDied","Data":"7c859fdb977b8424b67763c93a86808147652208a1b17fb1603229722cc336ed"} Jul 13 09:16:15.187272 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:15.187243 2552 generic.go:356] "Generic (PLEG): container finished" podID="a44db9bd-1992-44ff-ab0b-9ebc178ff2ed" containerID="8ea1ee404af07fa877710f8e1f8bee6166d9ad1063b1ea6c7cd5370ccda23d31" exitCode=0 Jul 13 09:16:15.187623 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:15.187288 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6m6j" event={"ID":"a44db9bd-1992-44ff-ab0b-9ebc178ff2ed","Type":"ContainerDied","Data":"8ea1ee404af07fa877710f8e1f8bee6166d9ad1063b1ea6c7cd5370ccda23d31"} Jul 13 09:16:16.317848 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:16.317826 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6m6j" Jul 13 09:16:16.410670 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:16.410646 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wl75z\" (UniqueName: \"kubernetes.io/projected/a44db9bd-1992-44ff-ab0b-9ebc178ff2ed-kube-api-access-wl75z\") pod \"a44db9bd-1992-44ff-ab0b-9ebc178ff2ed\" (UID: \"a44db9bd-1992-44ff-ab0b-9ebc178ff2ed\") " Jul 13 09:16:16.410809 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:16.410691 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a44db9bd-1992-44ff-ab0b-9ebc178ff2ed-bundle\") pod \"a44db9bd-1992-44ff-ab0b-9ebc178ff2ed\" (UID: \"a44db9bd-1992-44ff-ab0b-9ebc178ff2ed\") " Jul 13 09:16:16.410809 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:16.410711 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a44db9bd-1992-44ff-ab0b-9ebc178ff2ed-util\") pod \"a44db9bd-1992-44ff-ab0b-9ebc178ff2ed\" (UID: \"a44db9bd-1992-44ff-ab0b-9ebc178ff2ed\") " Jul 13 09:16:16.411331 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:16.411307 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a44db9bd-1992-44ff-ab0b-9ebc178ff2ed-bundle" (OuterVolumeSpecName: "bundle") pod "a44db9bd-1992-44ff-ab0b-9ebc178ff2ed" (UID: "a44db9bd-1992-44ff-ab0b-9ebc178ff2ed"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:16:16.412790 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:16.412767 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a44db9bd-1992-44ff-ab0b-9ebc178ff2ed-kube-api-access-wl75z" (OuterVolumeSpecName: "kube-api-access-wl75z") pod "a44db9bd-1992-44ff-ab0b-9ebc178ff2ed" (UID: "a44db9bd-1992-44ff-ab0b-9ebc178ff2ed"). InnerVolumeSpecName "kube-api-access-wl75z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:16:16.415877 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:16.415857 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a44db9bd-1992-44ff-ab0b-9ebc178ff2ed-util" (OuterVolumeSpecName: "util") pod "a44db9bd-1992-44ff-ab0b-9ebc178ff2ed" (UID: "a44db9bd-1992-44ff-ab0b-9ebc178ff2ed"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:16:16.511504 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:16.511436 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wl75z\" (UniqueName: \"kubernetes.io/projected/a44db9bd-1992-44ff-ab0b-9ebc178ff2ed-kube-api-access-wl75z\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:16:16.511504 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:16.511467 2552 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a44db9bd-1992-44ff-ab0b-9ebc178ff2ed-bundle\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:16:16.511504 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:16.511480 2552 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a44db9bd-1992-44ff-ab0b-9ebc178ff2ed-util\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:16:17.197549 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:17.197523 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6m6j" Jul 13 09:16:17.197708 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:17.197518 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6m6j" event={"ID":"a44db9bd-1992-44ff-ab0b-9ebc178ff2ed","Type":"ContainerDied","Data":"9733fdd21f4c33db4d493b9c4f854208acbc0e75852eeb72b78c65585615f2ef"} Jul 13 09:16:17.197708 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:17.197672 2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9733fdd21f4c33db4d493b9c4f854208acbc0e75852eeb72b78c65585615f2ef" Jul 13 09:16:20.944079 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:20.944048 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:16:22.325345 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:22.325312 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:16:22.826292 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:22.826259 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-frt4c"] Jul 13 09:16:22.826527 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:22.826516 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a44db9bd-1992-44ff-ab0b-9ebc178ff2ed" containerName="pull" Jul 13 09:16:22.826571 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:22.826528 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="a44db9bd-1992-44ff-ab0b-9ebc178ff2ed" containerName="pull" Jul 13 09:16:22.826571 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:22.826539 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a44db9bd-1992-44ff-ab0b-9ebc178ff2ed" containerName="util" Jul 13 09:16:22.826571 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:22.826545 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="a44db9bd-1992-44ff-ab0b-9ebc178ff2ed" containerName="util" Jul 13 09:16:22.826571 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:22.826553 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a44db9bd-1992-44ff-ab0b-9ebc178ff2ed" containerName="extract" Jul 13 09:16:22.826571 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:22.826558 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="a44db9bd-1992-44ff-ab0b-9ebc178ff2ed" containerName="extract" Jul 13 09:16:22.826714 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:22.826607 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="a44db9bd-1992-44ff-ab0b-9ebc178ff2ed" containerName="extract" Jul 13 09:16:22.829747 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:22.829728 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-frt4c" Jul 13 09:16:22.832546 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:22.832517 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-tddjp\"" Jul 13 09:16:22.832650 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:22.832518 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 13 09:16:22.832650 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:22.832561 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 13 09:16:22.844447 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:22.844430 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-frt4c"] Jul 13 09:16:22.953715 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:22.953688 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/74f19d73-e4f7-4174-bf16-e695d975e658-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-frt4c\" (UID: \"74f19d73-e4f7-4174-bf16-e695d975e658\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-frt4c" Jul 13 09:16:22.953803 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:22.953729 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-559fg\" (UniqueName: \"kubernetes.io/projected/74f19d73-e4f7-4174-bf16-e695d975e658-kube-api-access-559fg\") pod \"cert-manager-operator-controller-manager-55c444bf86-frt4c\" (UID: \"74f19d73-e4f7-4174-bf16-e695d975e658\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-frt4c" Jul 13 09:16:23.054249 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:23.054205 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-559fg\" (UniqueName: \"kubernetes.io/projected/74f19d73-e4f7-4174-bf16-e695d975e658-kube-api-access-559fg\") pod \"cert-manager-operator-controller-manager-55c444bf86-frt4c\" (UID: \"74f19d73-e4f7-4174-bf16-e695d975e658\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-frt4c" Jul 13 09:16:23.054336 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:23.054276 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/74f19d73-e4f7-4174-bf16-e695d975e658-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-frt4c\" (UID: \"74f19d73-e4f7-4174-bf16-e695d975e658\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-frt4c" Jul 13 09:16:23.054569 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:23.054554 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/74f19d73-e4f7-4174-bf16-e695d975e658-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-frt4c\" (UID: \"74f19d73-e4f7-4174-bf16-e695d975e658\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-frt4c" Jul 13 09:16:23.062520 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:23.062491 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-559fg\" (UniqueName: \"kubernetes.io/projected/74f19d73-e4f7-4174-bf16-e695d975e658-kube-api-access-559fg\") pod \"cert-manager-operator-controller-manager-55c444bf86-frt4c\" (UID: \"74f19d73-e4f7-4174-bf16-e695d975e658\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-frt4c" Jul 13 09:16:23.138398 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:23.138344 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-frt4c" Jul 13 09:16:23.271013 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:23.270982 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-frt4c"] Jul 13 09:16:23.274668 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:16:23.274644 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74f19d73_e4f7_4174_bf16_e695d975e658.slice/crio-fc5fecfad8c7792ef094e2fd515897745c3b3a5a1eb99cdbcbf321dcd473687e WatchSource:0}: Error finding container fc5fecfad8c7792ef094e2fd515897745c3b3a5a1eb99cdbcbf321dcd473687e: Status 404 returned error can't find the container with id fc5fecfad8c7792ef094e2fd515897745c3b3a5a1eb99cdbcbf321dcd473687e Jul 13 09:16:24.228640 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:24.228609 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-frt4c" event={"ID":"74f19d73-e4f7-4174-bf16-e695d975e658","Type":"ContainerStarted","Data":"fc5fecfad8c7792ef094e2fd515897745c3b3a5a1eb99cdbcbf321dcd473687e"} Jul 13 09:16:24.569590 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:24.569499 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:16:24.670462 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:24.670435 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:16:25.235263 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:25.235201 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-frt4c" event={"ID":"74f19d73-e4f7-4174-bf16-e695d975e658","Type":"ContainerStarted","Data":"d5f19909b440fde0427ad8d2cf5221a5fbd664093cd7d8e42f692519b30650c6"} Jul 13 09:16:25.654131 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:25.654100 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:16:25.757251 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:25.757197 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:16:26.316229 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:26.316185 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:16:26.419462 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:26.419430 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:16:26.650145 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:26.650086 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-frt4c" podStartSLOduration=3.286037815 podStartE2EDuration="4.65006979s" totalImagesPullingTime="1.364031975s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:16:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:16:25.261249018 +0000 UTC m=+469.141204281" watchObservedRunningTime="2026-07-13 09:16:26.65006979 +0000 UTC m=+470.530025061" Jul 13 09:16:26.650861 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:26.650843 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsf5w4"] Jul 13 09:16:26.654330 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:26.654314 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsf5w4" Jul 13 09:16:26.656982 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:26.656962 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 13 09:16:26.657979 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:26.657957 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 13 09:16:26.658089 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:26.657978 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-77zzn\"" Jul 13 09:16:26.666189 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:26.666166 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsf5w4"] Jul 13 09:16:26.784354 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:26.784320 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/430dce33-ddac-4e75-9e0e-57bafc72bf24-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsf5w4\" (UID: \"430dce33-ddac-4e75-9e0e-57bafc72bf24\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsf5w4" Jul 13 09:16:26.784517 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:26.784360 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nct8\" (UniqueName: \"kubernetes.io/projected/430dce33-ddac-4e75-9e0e-57bafc72bf24-kube-api-access-4nct8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsf5w4\" (UID: \"430dce33-ddac-4e75-9e0e-57bafc72bf24\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsf5w4" Jul 13 09:16:26.784517 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:26.784387 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/430dce33-ddac-4e75-9e0e-57bafc72bf24-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsf5w4\" (UID: \"430dce33-ddac-4e75-9e0e-57bafc72bf24\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsf5w4" Jul 13 09:16:26.884941 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:26.884904 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/430dce33-ddac-4e75-9e0e-57bafc72bf24-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsf5w4\" (UID: \"430dce33-ddac-4e75-9e0e-57bafc72bf24\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsf5w4" Jul 13 09:16:26.885107 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:26.884948 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nct8\" (UniqueName: \"kubernetes.io/projected/430dce33-ddac-4e75-9e0e-57bafc72bf24-kube-api-access-4nct8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsf5w4\" (UID: \"430dce33-ddac-4e75-9e0e-57bafc72bf24\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsf5w4" Jul 13 09:16:26.885107 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:26.884998 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/430dce33-ddac-4e75-9e0e-57bafc72bf24-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsf5w4\" (UID: \"430dce33-ddac-4e75-9e0e-57bafc72bf24\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsf5w4" Jul 13 09:16:26.885467 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:26.885445 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/430dce33-ddac-4e75-9e0e-57bafc72bf24-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsf5w4\" (UID: \"430dce33-ddac-4e75-9e0e-57bafc72bf24\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsf5w4" Jul 13 09:16:26.885508 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:26.885456 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/430dce33-ddac-4e75-9e0e-57bafc72bf24-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsf5w4\" (UID: \"430dce33-ddac-4e75-9e0e-57bafc72bf24\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsf5w4" Jul 13 09:16:26.893308 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:26.893284 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nct8\" (UniqueName: \"kubernetes.io/projected/430dce33-ddac-4e75-9e0e-57bafc72bf24-kube-api-access-4nct8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsf5w4\" (UID: \"430dce33-ddac-4e75-9e0e-57bafc72bf24\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsf5w4" Jul 13 09:16:26.964164 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:26.964094 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsf5w4" Jul 13 09:16:27.094742 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:27.094720 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsf5w4"] Jul 13 09:16:27.096808 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:16:27.096782 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod430dce33_ddac_4e75_9e0e_57bafc72bf24.slice/crio-3a349e3eb111967ad8ae68971b683d33e3bb77314bb5d9f0c6fc0eaea1f98381 WatchSource:0}: Error finding container 3a349e3eb111967ad8ae68971b683d33e3bb77314bb5d9f0c6fc0eaea1f98381: Status 404 returned error can't find the container with id 3a349e3eb111967ad8ae68971b683d33e3bb77314bb5d9f0c6fc0eaea1f98381 Jul 13 09:16:27.244386 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:27.244317 2552 generic.go:356] "Generic (PLEG): container finished" podID="430dce33-ddac-4e75-9e0e-57bafc72bf24" containerID="0a715d55196c6737da422eb497777c85af3420c402ddba37bbfb31c9586c35eb" exitCode=0 Jul 13 09:16:27.244386 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:27.244351 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsf5w4" event={"ID":"430dce33-ddac-4e75-9e0e-57bafc72bf24","Type":"ContainerDied","Data":"0a715d55196c6737da422eb497777c85af3420c402ddba37bbfb31c9586c35eb"} Jul 13 09:16:27.244590 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:27.244410 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsf5w4" event={"ID":"430dce33-ddac-4e75-9e0e-57bafc72bf24","Type":"ContainerStarted","Data":"3a349e3eb111967ad8ae68971b683d33e3bb77314bb5d9f0c6fc0eaea1f98381"} Jul 13 09:16:28.312963 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:28.312886 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:16:28.731123 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:28.731086 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:16:29.254533 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:29.254501 2552 generic.go:356] "Generic (PLEG): container finished" podID="430dce33-ddac-4e75-9e0e-57bafc72bf24" containerID="5f4040b068fe62dee1ea02b26ae4614422f57e89b552808b4acbddb6c3440035" exitCode=0 Jul 13 09:16:29.254646 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:29.254558 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsf5w4" event={"ID":"430dce33-ddac-4e75-9e0e-57bafc72bf24","Type":"ContainerDied","Data":"5f4040b068fe62dee1ea02b26ae4614422f57e89b552808b4acbddb6c3440035"} Jul 13 09:16:29.421788 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:29.421756 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-db9bl"] Jul 13 09:16:29.424804 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:29.424788 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-db9bl" Jul 13 09:16:29.428223 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:29.428193 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-xkqxz\"" Jul 13 09:16:29.428344 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:29.428254 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 13 09:16:29.428344 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:29.428262 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 13 09:16:29.436408 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:29.436381 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-db9bl"] Jul 13 09:16:29.608428 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:29.608391 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4d35db5e-b100-4c72-bce4-5cad56516025-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-db9bl\" (UID: \"4d35db5e-b100-4c72-bce4-5cad56516025\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-db9bl" Jul 13 09:16:29.608597 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:29.608445 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n785z\" (UniqueName: \"kubernetes.io/projected/4d35db5e-b100-4c72-bce4-5cad56516025-kube-api-access-n785z\") pod \"cert-manager-cainjector-559cdd8c87-db9bl\" (UID: \"4d35db5e-b100-4c72-bce4-5cad56516025\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-db9bl" Jul 13 09:16:29.709489 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:29.709393 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4d35db5e-b100-4c72-bce4-5cad56516025-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-db9bl\" (UID: \"4d35db5e-b100-4c72-bce4-5cad56516025\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-db9bl" Jul 13 09:16:29.709489 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:29.709449 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n785z\" (UniqueName: \"kubernetes.io/projected/4d35db5e-b100-4c72-bce4-5cad56516025-kube-api-access-n785z\") pod \"cert-manager-cainjector-559cdd8c87-db9bl\" (UID: \"4d35db5e-b100-4c72-bce4-5cad56516025\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-db9bl" Jul 13 09:16:29.717835 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:29.717800 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4d35db5e-b100-4c72-bce4-5cad56516025-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-db9bl\" (UID: \"4d35db5e-b100-4c72-bce4-5cad56516025\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-db9bl" Jul 13 09:16:29.717936 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:29.717867 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n785z\" (UniqueName: \"kubernetes.io/projected/4d35db5e-b100-4c72-bce4-5cad56516025-kube-api-access-n785z\") pod \"cert-manager-cainjector-559cdd8c87-db9bl\" (UID: \"4d35db5e-b100-4c72-bce4-5cad56516025\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-db9bl" Jul 13 09:16:29.750347 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:29.750320 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-db9bl" Jul 13 09:16:29.883164 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:29.883138 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-db9bl"] Jul 13 09:16:29.885639 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:16:29.885613 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4d35db5e_b100_4c72_bce4_5cad56516025.slice/crio-c9988e644627496b032b97cc0cbaf7b00f22e5f8abdf5375dc1460a65c260486 WatchSource:0}: Error finding container c9988e644627496b032b97cc0cbaf7b00f22e5f8abdf5375dc1460a65c260486: Status 404 returned error can't find the container with id c9988e644627496b032b97cc0cbaf7b00f22e5f8abdf5375dc1460a65c260486 Jul 13 09:16:30.260867 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:30.260829 2552 generic.go:356] "Generic (PLEG): container finished" podID="430dce33-ddac-4e75-9e0e-57bafc72bf24" containerID="6074f279bf7d99e5914863bb1642a879e1fa1ba2421df00edd9471415435227d" exitCode=0 Jul 13 09:16:30.261035 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:30.260902 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsf5w4" event={"ID":"430dce33-ddac-4e75-9e0e-57bafc72bf24","Type":"ContainerDied","Data":"6074f279bf7d99e5914863bb1642a879e1fa1ba2421df00edd9471415435227d"} Jul 13 09:16:30.262034 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:30.262010 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-db9bl" event={"ID":"4d35db5e-b100-4c72-bce4-5cad56516025","Type":"ContainerStarted","Data":"c9988e644627496b032b97cc0cbaf7b00f22e5f8abdf5375dc1460a65c260486"} Jul 13 09:16:31.411463 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:31.411435 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsf5w4" Jul 13 09:16:31.525369 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:31.525279 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/430dce33-ddac-4e75-9e0e-57bafc72bf24-bundle\") pod \"430dce33-ddac-4e75-9e0e-57bafc72bf24\" (UID: \"430dce33-ddac-4e75-9e0e-57bafc72bf24\") " Jul 13 09:16:31.525369 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:31.525319 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/430dce33-ddac-4e75-9e0e-57bafc72bf24-util\") pod \"430dce33-ddac-4e75-9e0e-57bafc72bf24\" (UID: \"430dce33-ddac-4e75-9e0e-57bafc72bf24\") " Jul 13 09:16:31.525582 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:31.525394 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4nct8\" (UniqueName: \"kubernetes.io/projected/430dce33-ddac-4e75-9e0e-57bafc72bf24-kube-api-access-4nct8\") pod \"430dce33-ddac-4e75-9e0e-57bafc72bf24\" (UID: \"430dce33-ddac-4e75-9e0e-57bafc72bf24\") " Jul 13 09:16:31.525785 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:31.525733 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/430dce33-ddac-4e75-9e0e-57bafc72bf24-bundle" (OuterVolumeSpecName: "bundle") pod "430dce33-ddac-4e75-9e0e-57bafc72bf24" (UID: "430dce33-ddac-4e75-9e0e-57bafc72bf24"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:16:31.527832 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:31.527801 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/430dce33-ddac-4e75-9e0e-57bafc72bf24-kube-api-access-4nct8" (OuterVolumeSpecName: "kube-api-access-4nct8") pod "430dce33-ddac-4e75-9e0e-57bafc72bf24" (UID: "430dce33-ddac-4e75-9e0e-57bafc72bf24"). InnerVolumeSpecName "kube-api-access-4nct8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:16:31.532830 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:31.532798 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/430dce33-ddac-4e75-9e0e-57bafc72bf24-util" (OuterVolumeSpecName: "util") pod "430dce33-ddac-4e75-9e0e-57bafc72bf24" (UID: "430dce33-ddac-4e75-9e0e-57bafc72bf24"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:16:31.551505 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:31.551459 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:16:31.614517 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:31.614484 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:16:31.626567 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:31.626532 2552 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/430dce33-ddac-4e75-9e0e-57bafc72bf24-bundle\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:16:31.626567 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:31.626558 2552 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/430dce33-ddac-4e75-9e0e-57bafc72bf24-util\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:16:31.626567 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:31.626573 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4nct8\" (UniqueName: \"kubernetes.io/projected/430dce33-ddac-4e75-9e0e-57bafc72bf24-kube-api-access-4nct8\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:16:32.274657 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:32.274615 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsf5w4" event={"ID":"430dce33-ddac-4e75-9e0e-57bafc72bf24","Type":"ContainerDied","Data":"3a349e3eb111967ad8ae68971b683d33e3bb77314bb5d9f0c6fc0eaea1f98381"} Jul 13 09:16:32.274816 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:32.274664 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsf5w4" Jul 13 09:16:32.274816 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:32.274673 2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3a349e3eb111967ad8ae68971b683d33e3bb77314bb5d9f0c6fc0eaea1f98381" Jul 13 09:16:33.209287 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:33.209262 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-rtdxt"] Jul 13 09:16:33.209579 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:33.209552 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="430dce33-ddac-4e75-9e0e-57bafc72bf24" containerName="util" Jul 13 09:16:33.209579 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:33.209563 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="430dce33-ddac-4e75-9e0e-57bafc72bf24" containerName="util" Jul 13 09:16:33.209579 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:33.209574 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="430dce33-ddac-4e75-9e0e-57bafc72bf24" containerName="pull" Jul 13 09:16:33.209579 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:33.209579 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="430dce33-ddac-4e75-9e0e-57bafc72bf24" containerName="pull" Jul 13 09:16:33.209723 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:33.209586 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="430dce33-ddac-4e75-9e0e-57bafc72bf24" containerName="extract" Jul 13 09:16:33.209723 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:33.209592 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="430dce33-ddac-4e75-9e0e-57bafc72bf24" containerName="extract" Jul 13 09:16:33.209723 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:33.209645 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="430dce33-ddac-4e75-9e0e-57bafc72bf24" containerName="extract" Jul 13 09:16:33.235351 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:33.235326 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-rtdxt"] Jul 13 09:16:33.235473 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:33.235433 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-rtdxt" Jul 13 09:16:33.238322 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:33.238303 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-ktzqv\"" Jul 13 09:16:33.241856 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:33.241830 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2337456b-a84c-4603-9c6b-ccb2c843ef6b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-rtdxt\" (UID: \"2337456b-a84c-4603-9c6b-ccb2c843ef6b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rtdxt" Jul 13 09:16:33.241940 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:33.241884 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwfq7\" (UniqueName: \"kubernetes.io/projected/2337456b-a84c-4603-9c6b-ccb2c843ef6b-kube-api-access-pwfq7\") pod \"cert-manager-webhook-6486ffd997-rtdxt\" (UID: \"2337456b-a84c-4603-9c6b-ccb2c843ef6b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rtdxt" Jul 13 09:16:33.280969 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:33.280934 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-db9bl" event={"ID":"4d35db5e-b100-4c72-bce4-5cad56516025","Type":"ContainerStarted","Data":"87ae43c14d3cd502499ce0263bcd0a6f1e0b902b736ef80a85163125b452cc5f"} Jul 13 09:16:33.296442 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:33.296393 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-db9bl" podStartSLOduration=1.041198038 podStartE2EDuration="4.296374083s" totalImagesPullingTime="3.255176045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:16:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:16:33.295488639 +0000 UTC m=+477.175443908" watchObservedRunningTime="2026-07-13 09:16:33.296374083 +0000 UTC m=+477.176329351" Jul 13 09:16:33.343299 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:33.343263 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2337456b-a84c-4603-9c6b-ccb2c843ef6b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-rtdxt\" (UID: \"2337456b-a84c-4603-9c6b-ccb2c843ef6b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rtdxt" Jul 13 09:16:33.343483 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:33.343387 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwfq7\" (UniqueName: \"kubernetes.io/projected/2337456b-a84c-4603-9c6b-ccb2c843ef6b-kube-api-access-pwfq7\") pod \"cert-manager-webhook-6486ffd997-rtdxt\" (UID: \"2337456b-a84c-4603-9c6b-ccb2c843ef6b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rtdxt" Jul 13 09:16:33.352588 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:33.352553 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2337456b-a84c-4603-9c6b-ccb2c843ef6b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-rtdxt\" (UID: \"2337456b-a84c-4603-9c6b-ccb2c843ef6b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rtdxt" Jul 13 09:16:33.352719 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:33.352667 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwfq7\" (UniqueName: \"kubernetes.io/projected/2337456b-a84c-4603-9c6b-ccb2c843ef6b-kube-api-access-pwfq7\") pod \"cert-manager-webhook-6486ffd997-rtdxt\" (UID: \"2337456b-a84c-4603-9c6b-ccb2c843ef6b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rtdxt" Jul 13 09:16:33.544393 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:33.544301 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-rtdxt" Jul 13 09:16:33.697631 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:33.697606 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-rtdxt"] Jul 13 09:16:33.699960 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:16:33.699931 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2337456b_a84c_4603_9c6b_ccb2c843ef6b.slice/crio-b7f8f6985d61eda50aca87ea4a708232dfded1dd7bc3b27a446423cd141cda76 WatchSource:0}: Error finding container b7f8f6985d61eda50aca87ea4a708232dfded1dd7bc3b27a446423cd141cda76: Status 404 returned error can't find the container with id b7f8f6985d61eda50aca87ea4a708232dfded1dd7bc3b27a446423cd141cda76 Jul 13 09:16:34.022303 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:34.022270 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:16:34.286437 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:34.286404 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-rtdxt" event={"ID":"2337456b-a84c-4603-9c6b-ccb2c843ef6b","Type":"ContainerStarted","Data":"735ba97f6eba312f8350ac7eb62fff72e76479d4133aee32b6bbb95df768ef94"} Jul 13 09:16:34.286437 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:34.286443 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-rtdxt" event={"ID":"2337456b-a84c-4603-9c6b-ccb2c843ef6b","Type":"ContainerStarted","Data":"b7f8f6985d61eda50aca87ea4a708232dfded1dd7bc3b27a446423cd141cda76"} Jul 13 09:16:34.286849 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:34.286482 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-rtdxt" Jul 13 09:16:34.304148 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:34.304041 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-rtdxt" podStartSLOduration=1.30400232 podStartE2EDuration="1.30400232s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:16:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:16:34.303799358 +0000 UTC m=+478.183754650" watchObservedRunningTime="2026-07-13 09:16:34.30400232 +0000 UTC m=+478.183957590" Jul 13 09:16:36.950419 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:36.950386 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:16:37.359693 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:37.359657 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:16:37.459667 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:37.459619 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:16:40.293428 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:40.293398 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-rtdxt" Jul 13 09:16:43.484917 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:43.484850 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:16:43.586691 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:43.586638 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:16:45.106327 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:45.106276 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:16:45.652030 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:45.651999 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af748dm"] Jul 13 09:16:45.705985 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:45.705960 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af748dm"] Jul 13 09:16:45.706134 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:45.706088 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af748dm" Jul 13 09:16:45.709003 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:45.708981 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 13 09:16:45.709131 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:45.709025 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 13 09:16:45.709917 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:45.709904 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-77zzn\"" Jul 13 09:16:45.729124 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:45.729099 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec57b49d-0cf1-4c3c-8413-a3c75f9c2491-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af748dm\" (UID: \"ec57b49d-0cf1-4c3c-8413-a3c75f9c2491\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af748dm" Jul 13 09:16:45.729278 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:45.729131 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qczj\" (UniqueName: \"kubernetes.io/projected/ec57b49d-0cf1-4c3c-8413-a3c75f9c2491-kube-api-access-6qczj\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af748dm\" (UID: \"ec57b49d-0cf1-4c3c-8413-a3c75f9c2491\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af748dm" Jul 13 09:16:45.729278 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:45.729183 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec57b49d-0cf1-4c3c-8413-a3c75f9c2491-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af748dm\" (UID: \"ec57b49d-0cf1-4c3c-8413-a3c75f9c2491\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af748dm" Jul 13 09:16:45.830404 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:45.830374 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qczj\" (UniqueName: \"kubernetes.io/projected/ec57b49d-0cf1-4c3c-8413-a3c75f9c2491-kube-api-access-6qczj\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af748dm\" (UID: \"ec57b49d-0cf1-4c3c-8413-a3c75f9c2491\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af748dm" Jul 13 09:16:45.830559 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:45.830413 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec57b49d-0cf1-4c3c-8413-a3c75f9c2491-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af748dm\" (UID: \"ec57b49d-0cf1-4c3c-8413-a3c75f9c2491\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af748dm" Jul 13 09:16:45.830559 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:45.830469 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec57b49d-0cf1-4c3c-8413-a3c75f9c2491-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af748dm\" (UID: \"ec57b49d-0cf1-4c3c-8413-a3c75f9c2491\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af748dm" Jul 13 09:16:45.830756 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:45.830740 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec57b49d-0cf1-4c3c-8413-a3c75f9c2491-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af748dm\" (UID: \"ec57b49d-0cf1-4c3c-8413-a3c75f9c2491\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af748dm" Jul 13 09:16:45.830812 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:45.830778 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec57b49d-0cf1-4c3c-8413-a3c75f9c2491-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af748dm\" (UID: \"ec57b49d-0cf1-4c3c-8413-a3c75f9c2491\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af748dm" Jul 13 09:16:45.838834 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:45.838805 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qczj\" (UniqueName: \"kubernetes.io/projected/ec57b49d-0cf1-4c3c-8413-a3c75f9c2491-kube-api-access-6qczj\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af748dm\" (UID: \"ec57b49d-0cf1-4c3c-8413-a3c75f9c2491\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af748dm" Jul 13 09:16:46.015157 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:46.015083 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af748dm" Jul 13 09:16:46.177053 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:46.177013 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af748dm"] Jul 13 09:16:46.182859 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:16:46.182830 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec57b49d_0cf1_4c3c_8413_a3c75f9c2491.slice/crio-4d226a0d0ac070c930cf574108bd19d90a2960a7a244e6a83e6008a9304a0fa1 WatchSource:0}: Error finding container 4d226a0d0ac070c930cf574108bd19d90a2960a7a244e6a83e6008a9304a0fa1: Status 404 returned error can't find the container with id 4d226a0d0ac070c930cf574108bd19d90a2960a7a244e6a83e6008a9304a0fa1 Jul 13 09:16:46.335863 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:46.335835 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af748dm" event={"ID":"ec57b49d-0cf1-4c3c-8413-a3c75f9c2491","Type":"ContainerStarted","Data":"859ab7bfa55a4bdeb386a28c7fb9390dfc57c58a09005d1546f941719c17a323"} Jul 13 09:16:46.335976 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:46.335881 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af748dm" event={"ID":"ec57b49d-0cf1-4c3c-8413-a3c75f9c2491","Type":"ContainerStarted","Data":"4d226a0d0ac070c930cf574108bd19d90a2960a7a244e6a83e6008a9304a0fa1"} Jul 13 09:16:46.854173 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:46.854138 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-r9l84"] Jul 13 09:16:46.870780 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:46.870756 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-r9l84"] Jul 13 09:16:46.870900 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:46.870830 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-r9l84" Jul 13 09:16:46.873598 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:46.873581 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-bntsk\"" Jul 13 09:16:46.936951 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:46.936927 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/416f7b60-3a7a-453e-9f7d-9552e9113ba5-bound-sa-token\") pod \"cert-manager-55b4d954d5-r9l84\" (UID: \"416f7b60-3a7a-453e-9f7d-9552e9113ba5\") " pod="cert-manager/cert-manager-55b4d954d5-r9l84" Jul 13 09:16:46.937048 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:46.936968 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n82jl\" (UniqueName: \"kubernetes.io/projected/416f7b60-3a7a-453e-9f7d-9552e9113ba5-kube-api-access-n82jl\") pod \"cert-manager-55b4d954d5-r9l84\" (UID: \"416f7b60-3a7a-453e-9f7d-9552e9113ba5\") " pod="cert-manager/cert-manager-55b4d954d5-r9l84" Jul 13 09:16:47.037807 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:47.037787 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n82jl\" (UniqueName: \"kubernetes.io/projected/416f7b60-3a7a-453e-9f7d-9552e9113ba5-kube-api-access-n82jl\") pod \"cert-manager-55b4d954d5-r9l84\" (UID: \"416f7b60-3a7a-453e-9f7d-9552e9113ba5\") " pod="cert-manager/cert-manager-55b4d954d5-r9l84" Jul 13 09:16:47.037903 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:47.037840 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/416f7b60-3a7a-453e-9f7d-9552e9113ba5-bound-sa-token\") pod \"cert-manager-55b4d954d5-r9l84\" (UID: \"416f7b60-3a7a-453e-9f7d-9552e9113ba5\") " pod="cert-manager/cert-manager-55b4d954d5-r9l84" Jul 13 09:16:47.045987 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:47.045964 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/416f7b60-3a7a-453e-9f7d-9552e9113ba5-bound-sa-token\") pod \"cert-manager-55b4d954d5-r9l84\" (UID: \"416f7b60-3a7a-453e-9f7d-9552e9113ba5\") " pod="cert-manager/cert-manager-55b4d954d5-r9l84" Jul 13 09:16:47.046567 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:47.046548 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n82jl\" (UniqueName: \"kubernetes.io/projected/416f7b60-3a7a-453e-9f7d-9552e9113ba5-kube-api-access-n82jl\") pod \"cert-manager-55b4d954d5-r9l84\" (UID: \"416f7b60-3a7a-453e-9f7d-9552e9113ba5\") " pod="cert-manager/cert-manager-55b4d954d5-r9l84" Jul 13 09:16:47.181132 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:47.181074 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-r9l84" Jul 13 09:16:47.312015 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:47.311994 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-r9l84"] Jul 13 09:16:47.313752 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:16:47.313726 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod416f7b60_3a7a_453e_9f7d_9552e9113ba5.slice/crio-9cbdd53f490f60d9435df7468baac5bc185ab6ffbaa035b62908e1d6e0a025d4 WatchSource:0}: Error finding container 9cbdd53f490f60d9435df7468baac5bc185ab6ffbaa035b62908e1d6e0a025d4: Status 404 returned error can't find the container with id 9cbdd53f490f60d9435df7468baac5bc185ab6ffbaa035b62908e1d6e0a025d4 Jul 13 09:16:47.341932 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:47.341903 2552 generic.go:356] "Generic (PLEG): container finished" podID="ec57b49d-0cf1-4c3c-8413-a3c75f9c2491" containerID="859ab7bfa55a4bdeb386a28c7fb9390dfc57c58a09005d1546f941719c17a323" exitCode=0 Jul 13 09:16:47.342024 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:47.341999 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af748dm" event={"ID":"ec57b49d-0cf1-4c3c-8413-a3c75f9c2491","Type":"ContainerDied","Data":"859ab7bfa55a4bdeb386a28c7fb9390dfc57c58a09005d1546f941719c17a323"} Jul 13 09:16:47.343298 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:47.343272 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-r9l84" event={"ID":"416f7b60-3a7a-453e-9f7d-9552e9113ba5","Type":"ContainerStarted","Data":"9cbdd53f490f60d9435df7468baac5bc185ab6ffbaa035b62908e1d6e0a025d4"} Jul 13 09:16:48.351162 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:48.351121 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-r9l84" event={"ID":"416f7b60-3a7a-453e-9f7d-9552e9113ba5","Type":"ContainerStarted","Data":"aadd82f280446cf889da040eb36d6f89683704eeb92ca53806d653d09b421c2f"} Jul 13 09:16:48.368044 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:48.367990 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-r9l84" podStartSLOduration=2.367975588 podStartE2EDuration="2.367975588s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:16:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:16:48.367693484 +0000 UTC m=+492.247648749" watchObservedRunningTime="2026-07-13 09:16:48.367975588 +0000 UTC m=+492.247930856" Jul 13 09:16:49.356675 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:49.356642 2552 generic.go:356] "Generic (PLEG): container finished" podID="ec57b49d-0cf1-4c3c-8413-a3c75f9c2491" containerID="63e281e6f870b1d50a2ba83496d1ac7d5e37fe7bb7c6301ef016ddb198eade35" exitCode=0 Jul 13 09:16:49.357091 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:49.356719 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af748dm" event={"ID":"ec57b49d-0cf1-4c3c-8413-a3c75f9c2491","Type":"ContainerDied","Data":"63e281e6f870b1d50a2ba83496d1ac7d5e37fe7bb7c6301ef016ddb198eade35"} Jul 13 09:16:50.363293 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:50.363260 2552 generic.go:356] "Generic (PLEG): container finished" podID="ec57b49d-0cf1-4c3c-8413-a3c75f9c2491" containerID="82223f3b8660a7de346e68bb17c8c648fbfdc3b318579d873792645f9c484cf0" exitCode=0 Jul 13 09:16:50.363686 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:50.363354 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af748dm" event={"ID":"ec57b49d-0cf1-4c3c-8413-a3c75f9c2491","Type":"ContainerDied","Data":"82223f3b8660a7de346e68bb17c8c648fbfdc3b318579d873792645f9c484cf0"} Jul 13 09:16:51.493292 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:51.493268 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af748dm" Jul 13 09:16:51.571805 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:51.571774 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec57b49d-0cf1-4c3c-8413-a3c75f9c2491-bundle\") pod \"ec57b49d-0cf1-4c3c-8413-a3c75f9c2491\" (UID: \"ec57b49d-0cf1-4c3c-8413-a3c75f9c2491\") " Jul 13 09:16:51.571933 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:51.571850 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qczj\" (UniqueName: \"kubernetes.io/projected/ec57b49d-0cf1-4c3c-8413-a3c75f9c2491-kube-api-access-6qczj\") pod \"ec57b49d-0cf1-4c3c-8413-a3c75f9c2491\" (UID: \"ec57b49d-0cf1-4c3c-8413-a3c75f9c2491\") " Jul 13 09:16:51.571933 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:51.571908 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec57b49d-0cf1-4c3c-8413-a3c75f9c2491-util\") pod \"ec57b49d-0cf1-4c3c-8413-a3c75f9c2491\" (UID: \"ec57b49d-0cf1-4c3c-8413-a3c75f9c2491\") " Jul 13 09:16:51.572592 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:51.572565 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec57b49d-0cf1-4c3c-8413-a3c75f9c2491-bundle" (OuterVolumeSpecName: "bundle") pod "ec57b49d-0cf1-4c3c-8413-a3c75f9c2491" (UID: "ec57b49d-0cf1-4c3c-8413-a3c75f9c2491"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:16:51.573916 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:51.573893 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec57b49d-0cf1-4c3c-8413-a3c75f9c2491-kube-api-access-6qczj" (OuterVolumeSpecName: "kube-api-access-6qczj") pod "ec57b49d-0cf1-4c3c-8413-a3c75f9c2491" (UID: "ec57b49d-0cf1-4c3c-8413-a3c75f9c2491"). InnerVolumeSpecName "kube-api-access-6qczj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:16:51.672545 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:51.672465 2552 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec57b49d-0cf1-4c3c-8413-a3c75f9c2491-bundle\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:16:51.672545 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:51.672495 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qczj\" (UniqueName: \"kubernetes.io/projected/ec57b49d-0cf1-4c3c-8413-a3c75f9c2491-kube-api-access-6qczj\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:16:51.675660 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:51.675636 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec57b49d-0cf1-4c3c-8413-a3c75f9c2491-util" (OuterVolumeSpecName: "util") pod "ec57b49d-0cf1-4c3c-8413-a3c75f9c2491" (UID: "ec57b49d-0cf1-4c3c-8413-a3c75f9c2491"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:16:51.773197 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:51.773169 2552 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec57b49d-0cf1-4c3c-8413-a3c75f9c2491-util\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:16:52.374163 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:52.374136 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af748dm" Jul 13 09:16:52.374349 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:52.374118 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af748dm" event={"ID":"ec57b49d-0cf1-4c3c-8413-a3c75f9c2491","Type":"ContainerDied","Data":"4d226a0d0ac070c930cf574108bd19d90a2960a7a244e6a83e6008a9304a0fa1"} Jul 13 09:16:52.374349 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:16:52.374263 2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4d226a0d0ac070c930cf574108bd19d90a2960a7a244e6a83e6008a9304a0fa1" Jul 13 09:17:02.355005 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:02.354959 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:17:02.453939 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:02.453906 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:17:02.975504 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:02.975470 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:17:03.145226 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.145184 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:17:03.273692 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.273607 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:17:03.296473 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.296433 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4"] Jul 13 09:17:03.296781 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.296764 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec57b49d-0cf1-4c3c-8413-a3c75f9c2491" containerName="pull" Jul 13 09:17:03.296876 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.296783 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec57b49d-0cf1-4c3c-8413-a3c75f9c2491" containerName="pull" Jul 13 09:17:03.296876 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.296816 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec57b49d-0cf1-4c3c-8413-a3c75f9c2491" containerName="util" Jul 13 09:17:03.296876 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.296825 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec57b49d-0cf1-4c3c-8413-a3c75f9c2491" containerName="util" Jul 13 09:17:03.296876 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.296837 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec57b49d-0cf1-4c3c-8413-a3c75f9c2491" containerName="extract" Jul 13 09:17:03.296876 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.296846 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec57b49d-0cf1-4c3c-8413-a3c75f9c2491" containerName="extract" Jul 13 09:17:03.297120 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.296925 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec57b49d-0cf1-4c3c-8413-a3c75f9c2491" containerName="extract" Jul 13 09:17:03.299956 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.299933 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4" Jul 13 09:17:03.302915 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.302891 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 13 09:17:03.303042 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.302939 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 13 09:17:03.303151 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.303138 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 13 09:17:03.303457 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.303444 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7j5hk\"" Jul 13 09:17:03.304674 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.304656 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 13 09:17:03.318173 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.318146 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4"] Jul 13 09:17:03.365412 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.365382 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c7e7373d-bf27-43fd-ace3-776f08e8c726-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4\" (UID: \"c7e7373d-bf27-43fd-ace3-776f08e8c726\") " pod="opendatahub/opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4" Jul 13 09:17:03.365800 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.365426 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c7e7373d-bf27-43fd-ace3-776f08e8c726-manifests\") pod \"opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4\" (UID: \"c7e7373d-bf27-43fd-ace3-776f08e8c726\") " pod="opendatahub/opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4" Jul 13 09:17:03.365800 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.365452 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh4n7\" (UniqueName: \"kubernetes.io/projected/c7e7373d-bf27-43fd-ace3-776f08e8c726-kube-api-access-xh4n7\") pod \"opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4\" (UID: \"c7e7373d-bf27-43fd-ace3-776f08e8c726\") " pod="opendatahub/opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4" Jul 13 09:17:03.365800 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.365475 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c7e7373d-bf27-43fd-ace3-776f08e8c726-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4\" (UID: \"c7e7373d-bf27-43fd-ace3-776f08e8c726\") " pod="opendatahub/opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4" Jul 13 09:17:03.365800 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.365498 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7e7373d-bf27-43fd-ace3-776f08e8c726-tmp\") pod \"opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4\" (UID: \"c7e7373d-bf27-43fd-ace3-776f08e8c726\") " pod="opendatahub/opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4" Jul 13 09:17:03.465850 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.465816 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c7e7373d-bf27-43fd-ace3-776f08e8c726-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4\" (UID: \"c7e7373d-bf27-43fd-ace3-776f08e8c726\") " pod="opendatahub/opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4" Jul 13 09:17:03.466038 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.465869 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c7e7373d-bf27-43fd-ace3-776f08e8c726-manifests\") pod \"opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4\" (UID: \"c7e7373d-bf27-43fd-ace3-776f08e8c726\") " pod="opendatahub/opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4" Jul 13 09:17:03.466038 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.465910 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh4n7\" (UniqueName: \"kubernetes.io/projected/c7e7373d-bf27-43fd-ace3-776f08e8c726-kube-api-access-xh4n7\") pod \"opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4\" (UID: \"c7e7373d-bf27-43fd-ace3-776f08e8c726\") " pod="opendatahub/opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4" Jul 13 09:17:03.466038 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.465941 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c7e7373d-bf27-43fd-ace3-776f08e8c726-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4\" (UID: \"c7e7373d-bf27-43fd-ace3-776f08e8c726\") " pod="opendatahub/opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4" Jul 13 09:17:03.466038 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.465976 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7e7373d-bf27-43fd-ace3-776f08e8c726-tmp\") pod \"opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4\" (UID: \"c7e7373d-bf27-43fd-ace3-776f08e8c726\") " pod="opendatahub/opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4" Jul 13 09:17:03.466409 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.466377 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c7e7373d-bf27-43fd-ace3-776f08e8c726-manifests\") pod \"opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4\" (UID: \"c7e7373d-bf27-43fd-ace3-776f08e8c726\") " pod="opendatahub/opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4" Jul 13 09:17:03.468318 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.468296 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7e7373d-bf27-43fd-ace3-776f08e8c726-tmp\") pod \"opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4\" (UID: \"c7e7373d-bf27-43fd-ace3-776f08e8c726\") " pod="opendatahub/opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4" Jul 13 09:17:03.468570 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.468546 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c7e7373d-bf27-43fd-ace3-776f08e8c726-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4\" (UID: \"c7e7373d-bf27-43fd-ace3-776f08e8c726\") " pod="opendatahub/opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4" Jul 13 09:17:03.468623 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.468580 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c7e7373d-bf27-43fd-ace3-776f08e8c726-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4\" (UID: \"c7e7373d-bf27-43fd-ace3-776f08e8c726\") " pod="opendatahub/opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4" Jul 13 09:17:03.477200 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.477177 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh4n7\" (UniqueName: \"kubernetes.io/projected/c7e7373d-bf27-43fd-ace3-776f08e8c726-kube-api-access-xh4n7\") pod \"opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4\" (UID: \"c7e7373d-bf27-43fd-ace3-776f08e8c726\") " pod="opendatahub/opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4" Jul 13 09:17:03.527538 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.527458 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:17:03.611561 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.611532 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4" Jul 13 09:17:03.626607 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.626582 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:17:03.716373 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.716341 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:17:03.756492 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.756470 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4"] Jul 13 09:17:03.758699 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:17:03.758673 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7e7373d_bf27_43fd_ace3_776f08e8c726.slice/crio-192023dd63c7a0fa0a66785230b6f500e5798dea5af5101a1a64419c276310cb WatchSource:0}: Error finding container 192023dd63c7a0fa0a66785230b6f500e5798dea5af5101a1a64419c276310cb: Status 404 returned error can't find the container with id 192023dd63c7a0fa0a66785230b6f500e5798dea5af5101a1a64419c276310cb Jul 13 09:17:03.783803 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.783700 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:17:03.884243 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.884173 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:17:03.987407 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:03.987375 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:17:04.432472 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:04.432400 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4" event={"ID":"c7e7373d-bf27-43fd-ace3-776f08e8c726","Type":"ContainerStarted","Data":"192023dd63c7a0fa0a66785230b6f500e5798dea5af5101a1a64419c276310cb"} Jul 13 09:17:07.451079 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:07.451042 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4" event={"ID":"c7e7373d-bf27-43fd-ace3-776f08e8c726","Type":"ContainerStarted","Data":"4214a7c2a5cefaf4a6a9a933a14b8f61129d017c8b2bc8d3f0e27cde7bc82d7c"} Jul 13 09:17:08.456745 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:08.456711 2552 generic.go:356] "Generic (PLEG): container finished" podID="c7e7373d-bf27-43fd-ace3-776f08e8c726" containerID="4214a7c2a5cefaf4a6a9a933a14b8f61129d017c8b2bc8d3f0e27cde7bc82d7c" exitCode=0 Jul 13 09:17:08.456745 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:08.456749 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4" event={"ID":"c7e7373d-bf27-43fd-ace3-776f08e8c726","Type":"ContainerDied","Data":"4214a7c2a5cefaf4a6a9a933a14b8f61129d017c8b2bc8d3f0e27cde7bc82d7c"} Jul 13 09:17:09.463116 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:09.463076 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4" event={"ID":"c7e7373d-bf27-43fd-ace3-776f08e8c726","Type":"ContainerStarted","Data":"60671cc453d28f84812d7649c61dd6074c0f5e1264ee5af7735abf5dd273f4ee"} Jul 13 09:17:09.463601 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:09.463234 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4" Jul 13 09:17:09.498734 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:09.498685 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4" podStartSLOduration=2.056304828 podStartE2EDuration="6.498669617s" totalImagesPullingTime="3.380248266s" totalInitContainerRuntime="1.062116523s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:17:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:17:09.496061193 +0000 UTC m=+513.376016460" watchObservedRunningTime="2026-07-13 09:17:09.498669617 +0000 UTC m=+513.378624883" Jul 13 09:17:15.131177 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:15.131138 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:17:15.231628 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:15.231594 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:17:17.122896 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:17.122865 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:17:17.237406 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:17.237367 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:17:20.470390 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:20.470348 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4" Jul 13 09:17:22.658494 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:22.658451 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:17:22.850429 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:22.850396 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:17:23.296388 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:23.296353 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:17:23.460756 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:23.460714 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7d64b4d488-d2lnz"] Jul 13 09:17:23.469186 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:23.469159 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7d64b4d488-d2lnz" Jul 13 09:17:23.472395 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:23.472371 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 13 09:17:23.472580 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:23.472561 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-2dxhn\"" Jul 13 09:17:23.477954 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:23.477934 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 13 09:17:23.480588 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:23.480569 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7d64b4d488-d2lnz"] Jul 13 09:17:23.545063 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:23.545032 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d895d638-812a-437a-a2b0-5cb7a8eb4e0c-tls-certs\") pod \"kube-auth-proxy-7d64b4d488-d2lnz\" (UID: \"d895d638-812a-437a-a2b0-5cb7a8eb4e0c\") " pod="openshift-ingress/kube-auth-proxy-7d64b4d488-d2lnz" Jul 13 09:17:23.545063 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:23.545066 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgsx8\" (UniqueName: \"kubernetes.io/projected/d895d638-812a-437a-a2b0-5cb7a8eb4e0c-kube-api-access-wgsx8\") pod \"kube-auth-proxy-7d64b4d488-d2lnz\" (UID: \"d895d638-812a-437a-a2b0-5cb7a8eb4e0c\") " pod="openshift-ingress/kube-auth-proxy-7d64b4d488-d2lnz" Jul 13 09:17:23.545350 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:23.545171 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d895d638-812a-437a-a2b0-5cb7a8eb4e0c-tmp\") pod \"kube-auth-proxy-7d64b4d488-d2lnz\" (UID: \"d895d638-812a-437a-a2b0-5cb7a8eb4e0c\") " pod="openshift-ingress/kube-auth-proxy-7d64b4d488-d2lnz" Jul 13 09:17:23.585018 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:23.584988 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-gm9js"] Jul 13 09:17:23.588353 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:23.588336 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-gm9js" Jul 13 09:17:23.598537 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:17:23.598491 2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"odh-model-controller-7998c67dc4-gm9js\" is forbidden: User \"system:node:ip-10-0-129-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-99.ec2.internal' and this object" podUID="75c93628-a23e-4cb5-98eb-3134b0a39bec" pod="opendatahub/odh-model-controller-7998c67dc4-gm9js" Jul 13 09:17:23.600198 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:17:23.600165 2552 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"odh-model-controller-dockercfg-9lgvc\" is forbidden: User \"system:node:ip-10-0-129-99.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-99.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-9lgvc\"" type="*v1.Secret" Jul 13 09:17:23.601575 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:23.601556 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 13 09:17:23.645683 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:23.645647 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d895d638-812a-437a-a2b0-5cb7a8eb4e0c-tls-certs\") pod \"kube-auth-proxy-7d64b4d488-d2lnz\" (UID: \"d895d638-812a-437a-a2b0-5cb7a8eb4e0c\") " pod="openshift-ingress/kube-auth-proxy-7d64b4d488-d2lnz" Jul 13 09:17:23.645683 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:23.645684 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgsx8\" (UniqueName: \"kubernetes.io/projected/d895d638-812a-437a-a2b0-5cb7a8eb4e0c-kube-api-access-wgsx8\") pod \"kube-auth-proxy-7d64b4d488-d2lnz\" (UID: \"d895d638-812a-437a-a2b0-5cb7a8eb4e0c\") " pod="openshift-ingress/kube-auth-proxy-7d64b4d488-d2lnz" Jul 13 09:17:23.645937 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:23.645744 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d895d638-812a-437a-a2b0-5cb7a8eb4e0c-tmp\") pod \"kube-auth-proxy-7d64b4d488-d2lnz\" (UID: \"d895d638-812a-437a-a2b0-5cb7a8eb4e0c\") " pod="openshift-ingress/kube-auth-proxy-7d64b4d488-d2lnz" Jul 13 09:17:23.645937 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:17:23.645791 2552 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 13 09:17:23.645937 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:17:23.645860 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d895d638-812a-437a-a2b0-5cb7a8eb4e0c-tls-certs podName:d895d638-812a-437a-a2b0-5cb7a8eb4e0c nodeName:}" failed. No retries permitted until 2026-07-13 09:17:24.145842551 +0000 UTC m=+528.025797796 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/d895d638-812a-437a-a2b0-5cb7a8eb4e0c-tls-certs") pod "kube-auth-proxy-7d64b4d488-d2lnz" (UID: "d895d638-812a-437a-a2b0-5cb7a8eb4e0c") : secret "kube-auth-proxy-tls" not found Jul 13 09:17:23.647537 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:23.647511 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-gm9js"] Jul 13 09:17:23.648101 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:23.648075 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d895d638-812a-437a-a2b0-5cb7a8eb4e0c-tmp\") pod \"kube-auth-proxy-7d64b4d488-d2lnz\" (UID: \"d895d638-812a-437a-a2b0-5cb7a8eb4e0c\") " pod="openshift-ingress/kube-auth-proxy-7d64b4d488-d2lnz" Jul 13 09:17:23.667719 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:23.667686 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgsx8\" (UniqueName: \"kubernetes.io/projected/d895d638-812a-437a-a2b0-5cb7a8eb4e0c-kube-api-access-wgsx8\") pod \"kube-auth-proxy-7d64b4d488-d2lnz\" (UID: \"d895d638-812a-437a-a2b0-5cb7a8eb4e0c\") " pod="openshift-ingress/kube-auth-proxy-7d64b4d488-d2lnz" Jul 13 09:17:23.733488 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:23.733458 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:17:23.746708 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:23.746675 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8gv9\" (UniqueName: \"kubernetes.io/projected/75c93628-a23e-4cb5-98eb-3134b0a39bec-kube-api-access-p8gv9\") pod \"odh-model-controller-7998c67dc4-gm9js\" (UID: \"75c93628-a23e-4cb5-98eb-3134b0a39bec\") " pod="opendatahub/odh-model-controller-7998c67dc4-gm9js" Jul 13 09:17:23.746892 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:23.746804 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/75c93628-a23e-4cb5-98eb-3134b0a39bec-cert\") pod \"odh-model-controller-7998c67dc4-gm9js\" (UID: \"75c93628-a23e-4cb5-98eb-3134b0a39bec\") " pod="opendatahub/odh-model-controller-7998c67dc4-gm9js" Jul 13 09:17:23.848065 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:23.847950 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8gv9\" (UniqueName: \"kubernetes.io/projected/75c93628-a23e-4cb5-98eb-3134b0a39bec-kube-api-access-p8gv9\") pod \"odh-model-controller-7998c67dc4-gm9js\" (UID: \"75c93628-a23e-4cb5-98eb-3134b0a39bec\") " pod="opendatahub/odh-model-controller-7998c67dc4-gm9js" Jul 13 09:17:23.848065 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:23.848055 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/75c93628-a23e-4cb5-98eb-3134b0a39bec-cert\") pod \"odh-model-controller-7998c67dc4-gm9js\" (UID: \"75c93628-a23e-4cb5-98eb-3134b0a39bec\") " pod="opendatahub/odh-model-controller-7998c67dc4-gm9js" Jul 13 09:17:23.848335 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:17:23.848185 2552 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 13 09:17:23.848335 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:17:23.848259 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75c93628-a23e-4cb5-98eb-3134b0a39bec-cert podName:75c93628-a23e-4cb5-98eb-3134b0a39bec nodeName:}" failed. No retries permitted until 2026-07-13 09:17:24.348243944 +0000 UTC m=+528.228199190 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/75c93628-a23e-4cb5-98eb-3134b0a39bec-cert") pod "odh-model-controller-7998c67dc4-gm9js" (UID: "75c93628-a23e-4cb5-98eb-3134b0a39bec") : secret "odh-model-controller-webhook-cert" not found Jul 13 09:17:23.860410 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:23.860372 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:17:23.862761 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:23.862735 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8gv9\" (UniqueName: \"kubernetes.io/projected/75c93628-a23e-4cb5-98eb-3134b0a39bec-kube-api-access-p8gv9\") pod \"odh-model-controller-7998c67dc4-gm9js\" (UID: \"75c93628-a23e-4cb5-98eb-3134b0a39bec\") " pod="opendatahub/odh-model-controller-7998c67dc4-gm9js" Jul 13 09:17:23.971738 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:23.971702 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:17:24.151280 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:24.151104 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d895d638-812a-437a-a2b0-5cb7a8eb4e0c-tls-certs\") pod \"kube-auth-proxy-7d64b4d488-d2lnz\" (UID: \"d895d638-812a-437a-a2b0-5cb7a8eb4e0c\") " pod="openshift-ingress/kube-auth-proxy-7d64b4d488-d2lnz" Jul 13 09:17:24.154196 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:24.154155 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d895d638-812a-437a-a2b0-5cb7a8eb4e0c-tls-certs\") pod \"kube-auth-proxy-7d64b4d488-d2lnz\" (UID: \"d895d638-812a-437a-a2b0-5cb7a8eb4e0c\") " pod="openshift-ingress/kube-auth-proxy-7d64b4d488-d2lnz" Jul 13 09:17:24.170044 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:24.169998 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:17:24.353106 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:24.353059 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/75c93628-a23e-4cb5-98eb-3134b0a39bec-cert\") pod \"odh-model-controller-7998c67dc4-gm9js\" (UID: \"75c93628-a23e-4cb5-98eb-3134b0a39bec\") " pod="opendatahub/odh-model-controller-7998c67dc4-gm9js" Jul 13 09:17:24.355484 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:24.355465 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/75c93628-a23e-4cb5-98eb-3134b0a39bec-cert\") pod \"odh-model-controller-7998c67dc4-gm9js\" (UID: \"75c93628-a23e-4cb5-98eb-3134b0a39bec\") " pod="opendatahub/odh-model-controller-7998c67dc4-gm9js" Jul 13 09:17:24.379968 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:24.379939 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7d64b4d488-d2lnz" Jul 13 09:17:24.576298 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:24.576273 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7d64b4d488-d2lnz"] Jul 13 09:17:24.578439 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:17:24.578413 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd895d638_812a_437a_a2b0_5cb7a8eb4e0c.slice/crio-be13fe5efb5b346b2dbdd91afcf9d33e8de9386dbd7b974c835856381ba3f294 WatchSource:0}: Error finding container be13fe5efb5b346b2dbdd91afcf9d33e8de9386dbd7b974c835856381ba3f294: Status 404 returned error can't find the container with id be13fe5efb5b346b2dbdd91afcf9d33e8de9386dbd7b974c835856381ba3f294 Jul 13 09:17:24.748079 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:24.748004 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-lx9jr"] Jul 13 09:17:24.752992 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:24.752974 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-lx9jr" Jul 13 09:17:24.755779 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:24.755756 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 13 09:17:24.770518 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:24.770498 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-lx9jr"] Jul 13 09:17:24.777005 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:24.776982 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:17:24.856361 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:24.856321 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70f4fd41-cd64-4ba4-9bc4-95a4c77971e0-tls-certs\") pod \"model-serving-api-54d89bd5fc-lx9jr\" (UID: \"70f4fd41-cd64-4ba4-9bc4-95a4c77971e0\") " pod="opendatahub/model-serving-api-54d89bd5fc-lx9jr" Jul 13 09:17:24.856534 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:24.856462 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw2hk\" (UniqueName: \"kubernetes.io/projected/70f4fd41-cd64-4ba4-9bc4-95a4c77971e0-kube-api-access-nw2hk\") pod \"model-serving-api-54d89bd5fc-lx9jr\" (UID: \"70f4fd41-cd64-4ba4-9bc4-95a4c77971e0\") " pod="opendatahub/model-serving-api-54d89bd5fc-lx9jr" Jul 13 09:17:24.957477 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:24.957437 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70f4fd41-cd64-4ba4-9bc4-95a4c77971e0-tls-certs\") pod \"model-serving-api-54d89bd5fc-lx9jr\" (UID: \"70f4fd41-cd64-4ba4-9bc4-95a4c77971e0\") " pod="opendatahub/model-serving-api-54d89bd5fc-lx9jr" Jul 13 09:17:24.957651 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:24.957565 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw2hk\" (UniqueName: \"kubernetes.io/projected/70f4fd41-cd64-4ba4-9bc4-95a4c77971e0-kube-api-access-nw2hk\") pod \"model-serving-api-54d89bd5fc-lx9jr\" (UID: \"70f4fd41-cd64-4ba4-9bc4-95a4c77971e0\") " pod="opendatahub/model-serving-api-54d89bd5fc-lx9jr" Jul 13 09:17:24.957651 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:17:24.957612 2552 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 13 09:17:24.957768 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:17:24.957695 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70f4fd41-cd64-4ba4-9bc4-95a4c77971e0-tls-certs podName:70f4fd41-cd64-4ba4-9bc4-95a4c77971e0 nodeName:}" failed. No retries permitted until 2026-07-13 09:17:25.457672314 +0000 UTC m=+529.337627559 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/70f4fd41-cd64-4ba4-9bc4-95a4c77971e0-tls-certs") pod "model-serving-api-54d89bd5fc-lx9jr" (UID: "70f4fd41-cd64-4ba4-9bc4-95a4c77971e0") : secret "model-serving-api-tls" not found Jul 13 09:17:24.968698 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:24.968665 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw2hk\" (UniqueName: \"kubernetes.io/projected/70f4fd41-cd64-4ba4-9bc4-95a4c77971e0-kube-api-access-nw2hk\") pod \"model-serving-api-54d89bd5fc-lx9jr\" (UID: \"70f4fd41-cd64-4ba4-9bc4-95a4c77971e0\") " pod="opendatahub/model-serving-api-54d89bd5fc-lx9jr" Jul 13 09:17:25.015677 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.015599 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn"] Jul 13 09:17:25.019230 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.019194 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn" Jul 13 09:17:25.020075 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.020045 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-9lgvc\"" Jul 13 09:17:25.021720 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.021672 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 13 09:17:25.021720 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.021671 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:17:25.022008 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.021993 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 13 09:17:25.022008 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.022003 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 13 09:17:25.022127 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.022003 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 13 09:17:25.022186 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.022134 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-nlh4t\"" Jul 13 09:17:25.029981 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.029964 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-gm9js" Jul 13 09:17:25.041864 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.041702 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn"] Jul 13 09:17:25.093056 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.093028 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:17:25.158924 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.158887 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-local-certs\") pod \"istiod-openshift-gateway-7574459656-v6xxn\" (UID: \"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn" Jul 13 09:17:25.158924 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.158924 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-istio-token\") pod \"istiod-openshift-gateway-7574459656-v6xxn\" (UID: \"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn" Jul 13 09:17:25.159137 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.158943 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82c78\" (UniqueName: \"kubernetes.io/projected/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-kube-api-access-82c78\") pod \"istiod-openshift-gateway-7574459656-v6xxn\" (UID: \"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn" Jul 13 09:17:25.159137 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.158960 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-v6xxn\" (UID: \"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn" Jul 13 09:17:25.159137 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.159038 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-cacerts\") pod \"istiod-openshift-gateway-7574459656-v6xxn\" (UID: \"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn" Jul 13 09:17:25.159137 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.159082 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-v6xxn\" (UID: \"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn" Jul 13 09:17:25.159350 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.159189 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-v6xxn\" (UID: \"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn" Jul 13 09:17:25.170885 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.170864 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-gm9js"] Jul 13 09:17:25.172961 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:17:25.172939 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75c93628_a23e_4cb5_98eb_3134b0a39bec.slice/crio-d012103820a3caa0f0ca081509025aad401d9de03c2bf832bdc4a23aad8a69d8 WatchSource:0}: Error finding container d012103820a3caa0f0ca081509025aad401d9de03c2bf832bdc4a23aad8a69d8: Status 404 returned error can't find the container with id d012103820a3caa0f0ca081509025aad401d9de03c2bf832bdc4a23aad8a69d8 Jul 13 09:17:25.260401 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.260343 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-v6xxn\" (UID: \"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn" Jul 13 09:17:25.260635 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.260510 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-v6xxn\" (UID: \"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn" Jul 13 09:17:25.260635 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.260571 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-local-certs\") pod \"istiod-openshift-gateway-7574459656-v6xxn\" (UID: \"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn" Jul 13 09:17:25.260818 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.260764 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-istio-token\") pod \"istiod-openshift-gateway-7574459656-v6xxn\" (UID: \"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn" Jul 13 09:17:25.260876 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.260814 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82c78\" (UniqueName: \"kubernetes.io/projected/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-kube-api-access-82c78\") pod \"istiod-openshift-gateway-7574459656-v6xxn\" (UID: \"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn" Jul 13 09:17:25.260876 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.260846 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-v6xxn\" (UID: \"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn" Jul 13 09:17:25.260980 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.260903 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-cacerts\") pod \"istiod-openshift-gateway-7574459656-v6xxn\" (UID: \"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn" Jul 13 09:17:25.261100 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.261064 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-v6xxn\" (UID: \"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn" Jul 13 09:17:25.263599 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.263572 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-cacerts\") pod \"istiod-openshift-gateway-7574459656-v6xxn\" (UID: \"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn" Jul 13 09:17:25.263710 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.263659 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-local-certs\") pod \"istiod-openshift-gateway-7574459656-v6xxn\" (UID: \"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn" Jul 13 09:17:25.263710 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.263701 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-v6xxn\" (UID: \"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn" Jul 13 09:17:25.263710 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.263706 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-v6xxn\" (UID: \"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn" Jul 13 09:17:25.270200 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.270144 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82c78\" (UniqueName: \"kubernetes.io/projected/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-kube-api-access-82c78\") pod \"istiod-openshift-gateway-7574459656-v6xxn\" (UID: \"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn" Jul 13 09:17:25.270299 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.270288 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-istio-token\") pod \"istiod-openshift-gateway-7574459656-v6xxn\" (UID: \"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn" Jul 13 09:17:25.330115 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.330082 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn" Jul 13 09:17:25.462883 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.462849 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70f4fd41-cd64-4ba4-9bc4-95a4c77971e0-tls-certs\") pod \"model-serving-api-54d89bd5fc-lx9jr\" (UID: \"70f4fd41-cd64-4ba4-9bc4-95a4c77971e0\") " pod="opendatahub/model-serving-api-54d89bd5fc-lx9jr" Jul 13 09:17:25.465883 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.465855 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70f4fd41-cd64-4ba4-9bc4-95a4c77971e0-tls-certs\") pod \"model-serving-api-54d89bd5fc-lx9jr\" (UID: \"70f4fd41-cd64-4ba4-9bc4-95a4c77971e0\") " pod="opendatahub/model-serving-api-54d89bd5fc-lx9jr" Jul 13 09:17:25.483146 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.483121 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn"] Jul 13 09:17:25.485068 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:17:25.485035 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7c5b9b7_6dd3_4a22_97dd_9f1375beeede.slice/crio-3615bf03cbe580c009a79b27482190efcd3a6b0886bcb2472e7ca863e78477bc WatchSource:0}: Error finding container 3615bf03cbe580c009a79b27482190efcd3a6b0886bcb2472e7ca863e78477bc: Status 404 returned error can't find the container with id 3615bf03cbe580c009a79b27482190efcd3a6b0886bcb2472e7ca863e78477bc Jul 13 09:17:25.541353 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.541247 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn" event={"ID":"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede","Type":"ContainerStarted","Data":"3615bf03cbe580c009a79b27482190efcd3a6b0886bcb2472e7ca863e78477bc"} Jul 13 09:17:25.542615 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.542578 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-gm9js" event={"ID":"75c93628-a23e-4cb5-98eb-3134b0a39bec","Type":"ContainerStarted","Data":"d012103820a3caa0f0ca081509025aad401d9de03c2bf832bdc4a23aad8a69d8"} Jul 13 09:17:25.543820 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.543789 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7d64b4d488-d2lnz" event={"ID":"d895d638-812a-437a-a2b0-5cb7a8eb4e0c","Type":"ContainerStarted","Data":"be13fe5efb5b346b2dbdd91afcf9d33e8de9386dbd7b974c835856381ba3f294"} Jul 13 09:17:25.663563 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.663524 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-lx9jr" Jul 13 09:17:25.807476 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:25.807453 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-lx9jr"] Jul 13 09:17:25.809586 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:17:25.809558 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70f4fd41_cd64_4ba4_9bc4_95a4c77971e0.slice/crio-0e04b363ab7e67742364aea9b84872716ac014aced02face7447cda1020926b1 WatchSource:0}: Error finding container 0e04b363ab7e67742364aea9b84872716ac014aced02face7447cda1020926b1: Status 404 returned error can't find the container with id 0e04b363ab7e67742364aea9b84872716ac014aced02face7447cda1020926b1 Jul 13 09:17:26.115311 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:26.115175 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:17:26.567483 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:26.566600 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-lx9jr" event={"ID":"70f4fd41-cd64-4ba4-9bc4-95a4c77971e0","Type":"ContainerStarted","Data":"0e04b363ab7e67742364aea9b84872716ac014aced02face7447cda1020926b1"} Jul 13 09:17:30.299268 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:30.299197 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:17:30.299618 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:30.299317 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:17:30.589999 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:30.589958 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7d64b4d488-d2lnz" event={"ID":"d895d638-812a-437a-a2b0-5cb7a8eb4e0c","Type":"ContainerStarted","Data":"c6724e4f65dc91af8ab2d4c0f2a3bfffefe4f5c598ddeb01fdc8a4867408f27b"} Jul 13 09:17:30.591523 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:30.591495 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-lx9jr" event={"ID":"70f4fd41-cd64-4ba4-9bc4-95a4c77971e0","Type":"ContainerStarted","Data":"5b355982cb01c19446a3c15a0a67379043808d8b5f563d679a884bffa6ec8c04"} Jul 13 09:17:30.591662 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:30.591591 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-lx9jr" Jul 13 09:17:30.593097 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:30.593073 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn" event={"ID":"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede","Type":"ContainerStarted","Data":"ee37e456e262963c8a660f1834fa55f5a95a4e1d03d95cb4a8f4661340eb26cf"} Jul 13 09:17:30.593205 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:30.593111 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn" Jul 13 09:17:30.594601 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:30.594580 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-gm9js" event={"ID":"75c93628-a23e-4cb5-98eb-3134b0a39bec","Type":"ContainerStarted","Data":"8171f7ba334d6fcf3f0fc4a9b5fd1a51186f2d7ff3fae05e75f3de4bb5f6c98b"} Jul 13 09:17:30.594710 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:30.594695 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-gm9js" Jul 13 09:17:30.635072 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:30.634998 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7d64b4d488-d2lnz" podStartSLOduration=1.996084212 podStartE2EDuration="7.634979409s" totalImagesPullingTime="5.638895197s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:17:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:17:30.629888896 +0000 UTC m=+534.509844163" watchObservedRunningTime="2026-07-13 09:17:30.634979409 +0000 UTC m=+534.514934673" Jul 13 09:17:30.674156 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:30.674087 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-lx9jr" podStartSLOduration=2.19934272 podStartE2EDuration="6.674067131s" totalImagesPullingTime="4.474724411s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:17:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:17:30.670629073 +0000 UTC m=+534.550584341" watchObservedRunningTime="2026-07-13 09:17:30.674067131 +0000 UTC m=+534.554022398" Jul 13 09:17:30.779289 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:30.779191 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn" podStartSLOduration=1.9677787279999999 podStartE2EDuration="6.779168064s" totalImagesPullingTime="4.811389336s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:17:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:17:30.777274364 +0000 UTC m=+534.657229633" watchObservedRunningTime="2026-07-13 09:17:30.779168064 +0000 UTC m=+534.659123332" Jul 13 09:17:30.781357 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:30.781304 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-gm9js" podStartSLOduration=2.6741598250000003 podStartE2EDuration="7.781286339s" totalImagesPullingTime="5.107126514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:17:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:17:30.721133246 +0000 UTC m=+534.601088515" watchObservedRunningTime="2026-07-13 09:17:30.781286339 +0000 UTC m=+534.661241610" Jul 13 09:17:31.155791 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:31.155758 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:17:31.248832 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:31.248801 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:17:31.605655 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:31.605622 2552 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-v6xxn container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 13 09:17:31.606051 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:31.605676 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn" podUID="f7c5b9b7-6dd3-4a22-97dd-9f1375beeede" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 09:17:31.984177 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:31.984096 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25"] Jul 13 09:17:31.987888 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:31.987862 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25" Jul 13 09:17:31.991439 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:31.991419 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-st86h\"" Jul 13 09:17:32.018544 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:32.018511 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:17:32.083111 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:32.083083 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25"] Jul 13 09:17:32.123901 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:32.123864 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ccd1a61-cd7d-41c1-8236-76d04bc28b65-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wzv25\" (UID: \"5ccd1a61-cd7d-41c1-8236-76d04bc28b65\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25" Jul 13 09:17:32.123901 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:32.123904 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ccd1a61-cd7d-41c1-8236-76d04bc28b65-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wzv25\" (UID: \"5ccd1a61-cd7d-41c1-8236-76d04bc28b65\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25" Jul 13 09:17:32.124142 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:32.123927 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bbbq\" (UniqueName: \"kubernetes.io/projected/5ccd1a61-cd7d-41c1-8236-76d04bc28b65-kube-api-access-8bbbq\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wzv25\" (UID: \"5ccd1a61-cd7d-41c1-8236-76d04bc28b65\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25" Jul 13 09:17:32.124142 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:32.123978 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ccd1a61-cd7d-41c1-8236-76d04bc28b65-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wzv25\" (UID: \"5ccd1a61-cd7d-41c1-8236-76d04bc28b65\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25" Jul 13 09:17:32.124142 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:32.124032 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ccd1a61-cd7d-41c1-8236-76d04bc28b65-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wzv25\" (UID: \"5ccd1a61-cd7d-41c1-8236-76d04bc28b65\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25" Jul 13 09:17:32.124142 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:32.124056 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ccd1a61-cd7d-41c1-8236-76d04bc28b65-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wzv25\" (UID: \"5ccd1a61-cd7d-41c1-8236-76d04bc28b65\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25" Jul 13 09:17:32.124142 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:32.124081 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ccd1a61-cd7d-41c1-8236-76d04bc28b65-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wzv25\" (UID: \"5ccd1a61-cd7d-41c1-8236-76d04bc28b65\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25" Jul 13 09:17:32.124142 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:32.124125 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ccd1a61-cd7d-41c1-8236-76d04bc28b65-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wzv25\" (UID: \"5ccd1a61-cd7d-41c1-8236-76d04bc28b65\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25" Jul 13 09:17:32.124142 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:32.124139 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ccd1a61-cd7d-41c1-8236-76d04bc28b65-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wzv25\" (UID: \"5ccd1a61-cd7d-41c1-8236-76d04bc28b65\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25" Jul 13 09:17:32.131375 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:32.131339 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:17:32.225646 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:32.225606 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ccd1a61-cd7d-41c1-8236-76d04bc28b65-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wzv25\" (UID: \"5ccd1a61-cd7d-41c1-8236-76d04bc28b65\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25" Jul 13 09:17:32.225646 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:32.225646 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ccd1a61-cd7d-41c1-8236-76d04bc28b65-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wzv25\" (UID: \"5ccd1a61-cd7d-41c1-8236-76d04bc28b65\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25" Jul 13 09:17:32.225891 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:32.225689 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bbbq\" (UniqueName: \"kubernetes.io/projected/5ccd1a61-cd7d-41c1-8236-76d04bc28b65-kube-api-access-8bbbq\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wzv25\" (UID: \"5ccd1a61-cd7d-41c1-8236-76d04bc28b65\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25" Jul 13 09:17:32.225891 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:32.225714 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ccd1a61-cd7d-41c1-8236-76d04bc28b65-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wzv25\" (UID: \"5ccd1a61-cd7d-41c1-8236-76d04bc28b65\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25" Jul 13 09:17:32.225891 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:32.225731 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ccd1a61-cd7d-41c1-8236-76d04bc28b65-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wzv25\" (UID: \"5ccd1a61-cd7d-41c1-8236-76d04bc28b65\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25" Jul 13 09:17:32.225891 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:32.225759 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ccd1a61-cd7d-41c1-8236-76d04bc28b65-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wzv25\" (UID: \"5ccd1a61-cd7d-41c1-8236-76d04bc28b65\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25" Jul 13 09:17:32.225891 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:32.225797 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ccd1a61-cd7d-41c1-8236-76d04bc28b65-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wzv25\" (UID: \"5ccd1a61-cd7d-41c1-8236-76d04bc28b65\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25" Jul 13 09:17:32.225891 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:32.225846 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ccd1a61-cd7d-41c1-8236-76d04bc28b65-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wzv25\" (UID: \"5ccd1a61-cd7d-41c1-8236-76d04bc28b65\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25" Jul 13 09:17:32.225891 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:32.225869 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ccd1a61-cd7d-41c1-8236-76d04bc28b65-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wzv25\" (UID: \"5ccd1a61-cd7d-41c1-8236-76d04bc28b65\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25" Jul 13 09:17:32.226192 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:32.226066 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ccd1a61-cd7d-41c1-8236-76d04bc28b65-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wzv25\" (UID: \"5ccd1a61-cd7d-41c1-8236-76d04bc28b65\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25" Jul 13 09:17:32.226192 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:32.226137 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ccd1a61-cd7d-41c1-8236-76d04bc28b65-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wzv25\" (UID: \"5ccd1a61-cd7d-41c1-8236-76d04bc28b65\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25" Jul 13 09:17:32.226313 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:32.226239 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ccd1a61-cd7d-41c1-8236-76d04bc28b65-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wzv25\" (UID: \"5ccd1a61-cd7d-41c1-8236-76d04bc28b65\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25" Jul 13 09:17:32.226313 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:32.226267 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ccd1a61-cd7d-41c1-8236-76d04bc28b65-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wzv25\" (UID: \"5ccd1a61-cd7d-41c1-8236-76d04bc28b65\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25" Jul 13 09:17:32.226535 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:32.226514 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ccd1a61-cd7d-41c1-8236-76d04bc28b65-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wzv25\" (UID: \"5ccd1a61-cd7d-41c1-8236-76d04bc28b65\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25" Jul 13 09:17:32.228599 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:32.228575 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ccd1a61-cd7d-41c1-8236-76d04bc28b65-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wzv25\" (UID: \"5ccd1a61-cd7d-41c1-8236-76d04bc28b65\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25" Jul 13 09:17:32.229131 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:32.229100 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ccd1a61-cd7d-41c1-8236-76d04bc28b65-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wzv25\" (UID: \"5ccd1a61-cd7d-41c1-8236-76d04bc28b65\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25" Jul 13 09:17:32.238586 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:32.238526 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ccd1a61-cd7d-41c1-8236-76d04bc28b65-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wzv25\" (UID: \"5ccd1a61-cd7d-41c1-8236-76d04bc28b65\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25" Jul 13 09:17:32.238775 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:32.238752 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bbbq\" (UniqueName: \"kubernetes.io/projected/5ccd1a61-cd7d-41c1-8236-76d04bc28b65-kube-api-access-8bbbq\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wzv25\" (UID: \"5ccd1a61-cd7d-41c1-8236-76d04bc28b65\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25" Jul 13 09:17:32.261590 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:32.261530 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:17:32.298518 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:32.298482 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25" Jul 13 09:17:32.470312 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:32.470273 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25"] Jul 13 09:17:32.476601 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:17:32.476575 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ccd1a61_cd7d_41c1_8236_76d04bc28b65.slice/crio-57263e1fe61fa73122dd7f9faa046dd9a72bcf5015485716000e982d723cdf59 WatchSource:0}: Error finding container 57263e1fe61fa73122dd7f9faa046dd9a72bcf5015485716000e982d723cdf59: Status 404 returned error can't find the container with id 57263e1fe61fa73122dd7f9faa046dd9a72bcf5015485716000e982d723cdf59 Jul 13 09:17:32.606710 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:32.606671 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25" event={"ID":"5ccd1a61-cd7d-41c1-8236-76d04bc28b65","Type":"ContainerStarted","Data":"57263e1fe61fa73122dd7f9faa046dd9a72bcf5015485716000e982d723cdf59"} Jul 13 09:17:33.592876 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:33.592569 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:17:33.794467 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:33.794437 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xxjmb"] Jul 13 09:17:33.797917 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:33.797895 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xxjmb" Jul 13 09:17:33.803805 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:33.803776 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-sblwt\"" Jul 13 09:17:33.803930 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:33.803869 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 13 09:17:33.806997 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:33.806920 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:17:33.839607 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:33.839576 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xxjmb"] Jul 13 09:17:33.946697 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:33.946599 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgcln\" (UniqueName: \"kubernetes.io/projected/0c3c450a-0391-4d1c-9db9-b9c1d3215a42-kube-api-access-qgcln\") pod \"kserve-controller-manager-d6f4b65b8-xxjmb\" (UID: \"0c3c450a-0391-4d1c-9db9-b9c1d3215a42\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xxjmb" Jul 13 09:17:33.946697 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:33.946690 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c3c450a-0391-4d1c-9db9-b9c1d3215a42-cert\") pod \"kserve-controller-manager-d6f4b65b8-xxjmb\" (UID: \"0c3c450a-0391-4d1c-9db9-b9c1d3215a42\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xxjmb" Jul 13 09:17:33.962022 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:33.961946 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:17:34.047327 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:34.047295 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgcln\" (UniqueName: \"kubernetes.io/projected/0c3c450a-0391-4d1c-9db9-b9c1d3215a42-kube-api-access-qgcln\") pod \"kserve-controller-manager-d6f4b65b8-xxjmb\" (UID: \"0c3c450a-0391-4d1c-9db9-b9c1d3215a42\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xxjmb" Jul 13 09:17:34.047518 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:34.047375 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c3c450a-0391-4d1c-9db9-b9c1d3215a42-cert\") pod \"kserve-controller-manager-d6f4b65b8-xxjmb\" (UID: \"0c3c450a-0391-4d1c-9db9-b9c1d3215a42\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xxjmb" Jul 13 09:17:34.050472 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:34.050439 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c3c450a-0391-4d1c-9db9-b9c1d3215a42-cert\") pod \"kserve-controller-manager-d6f4b65b8-xxjmb\" (UID: \"0c3c450a-0391-4d1c-9db9-b9c1d3215a42\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xxjmb" Jul 13 09:17:34.071168 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:34.071116 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:17:34.071975 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:34.071948 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgcln\" (UniqueName: \"kubernetes.io/projected/0c3c450a-0391-4d1c-9db9-b9c1d3215a42-kube-api-access-qgcln\") pod \"kserve-controller-manager-d6f4b65b8-xxjmb\" (UID: \"0c3c450a-0391-4d1c-9db9-b9c1d3215a42\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xxjmb" Jul 13 09:17:34.111736 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:34.111683 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xxjmb" Jul 13 09:17:34.167675 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:34.167585 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:17:34.282827 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:34.282179 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xxjmb"] Jul 13 09:17:34.545324 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:34.545242 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:17:34.601126 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:34.601091 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn" Jul 13 09:17:34.984330 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:34.984297 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn"] Jul 13 09:17:34.984795 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:34.984513 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn" podUID="f7c5b9b7-6dd3-4a22-97dd-9f1375beeede" containerName="discovery" containerID="cri-o://ee37e456e262963c8a660f1834fa55f5a95a4e1d03d95cb4a8f4661340eb26cf" gracePeriod=30 Jul 13 09:17:35.268766 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:17:35.268687 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c3c450a_0391_4d1c_9db9_b9c1d3215a42.slice/crio-f837c8f06deb6a377ea35fae6fdccde9e706b0d317d19d26bc05b34d589fec88 WatchSource:0}: Error finding container f837c8f06deb6a377ea35fae6fdccde9e706b0d317d19d26bc05b34d589fec88: Status 404 returned error can't find the container with id f837c8f06deb6a377ea35fae6fdccde9e706b0d317d19d26bc05b34d589fec88 Jul 13 09:17:35.331697 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:35.331654 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:17:35.331807 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:35.331748 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:17:35.331807 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:35.331790 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:17:35.440736 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:35.435435 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn" Jul 13 09:17:35.563868 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:35.563772 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82c78\" (UniqueName: \"kubernetes.io/projected/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-kube-api-access-82c78\") pod \"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede\" (UID: \"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede\") " Jul 13 09:17:35.563868 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:35.563819 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-cacerts\") pod \"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede\" (UID: \"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede\") " Jul 13 09:17:35.564088 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:35.563873 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-istio-csr-dns-cert\") pod \"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede\" (UID: \"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede\") " Jul 13 09:17:35.564088 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:35.563929 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-istio-csr-ca-configmap\") pod \"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede\" (UID: \"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede\") " Jul 13 09:17:35.564088 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:35.563974 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-istio-token\") pod \"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede\" (UID: \"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede\") " Jul 13 09:17:35.564088 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:35.563996 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-istio-kubeconfig\") pod \"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede\" (UID: \"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede\") " Jul 13 09:17:35.564088 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:35.564048 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-local-certs\") pod \"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede\" (UID: \"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede\") " Jul 13 09:17:35.565458 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:35.565424 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "f7c5b9b7-6dd3-4a22-97dd-9f1375beeede" (UID: "f7c5b9b7-6dd3-4a22-97dd-9f1375beeede"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 09:17:35.566808 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:35.566772 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-kube-api-access-82c78" (OuterVolumeSpecName: "kube-api-access-82c78") pod "f7c5b9b7-6dd3-4a22-97dd-9f1375beeede" (UID: "f7c5b9b7-6dd3-4a22-97dd-9f1375beeede"). InnerVolumeSpecName "kube-api-access-82c78". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:17:35.566915 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:35.566878 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-local-certs" (OuterVolumeSpecName: "local-certs") pod "f7c5b9b7-6dd3-4a22-97dd-9f1375beeede" (UID: "f7c5b9b7-6dd3-4a22-97dd-9f1375beeede"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:17:35.566965 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:35.566935 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "f7c5b9b7-6dd3-4a22-97dd-9f1375beeede" (UID: "f7c5b9b7-6dd3-4a22-97dd-9f1375beeede"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 09:17:35.567077 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:35.567053 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-cacerts" (OuterVolumeSpecName: "cacerts") pod "f7c5b9b7-6dd3-4a22-97dd-9f1375beeede" (UID: "f7c5b9b7-6dd3-4a22-97dd-9f1375beeede"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 09:17:35.567455 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:35.567425 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-istio-token" (OuterVolumeSpecName: "istio-token") pod "f7c5b9b7-6dd3-4a22-97dd-9f1375beeede" (UID: "f7c5b9b7-6dd3-4a22-97dd-9f1375beeede"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:17:35.568274 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:35.568252 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "f7c5b9b7-6dd3-4a22-97dd-9f1375beeede" (UID: "f7c5b9b7-6dd3-4a22-97dd-9f1375beeede"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 09:17:35.631725 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:35.631692 2552 generic.go:356] "Generic (PLEG): container finished" podID="f7c5b9b7-6dd3-4a22-97dd-9f1375beeede" containerID="ee37e456e262963c8a660f1834fa55f5a95a4e1d03d95cb4a8f4661340eb26cf" exitCode=0 Jul 13 09:17:35.631912 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:35.631738 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn" Jul 13 09:17:35.631912 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:35.631780 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn" event={"ID":"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede","Type":"ContainerDied","Data":"ee37e456e262963c8a660f1834fa55f5a95a4e1d03d95cb4a8f4661340eb26cf"} Jul 13 09:17:35.631912 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:35.631835 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn" event={"ID":"f7c5b9b7-6dd3-4a22-97dd-9f1375beeede","Type":"ContainerDied","Data":"3615bf03cbe580c009a79b27482190efcd3a6b0886bcb2472e7ca863e78477bc"} Jul 13 09:17:35.631912 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:35.631865 2552 scope.go:117] "RemoveContainer" containerID="ee37e456e262963c8a660f1834fa55f5a95a4e1d03d95cb4a8f4661340eb26cf" Jul 13 09:17:35.633263 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:35.633196 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xxjmb" event={"ID":"0c3c450a-0391-4d1c-9db9-b9c1d3215a42","Type":"ContainerStarted","Data":"f837c8f06deb6a377ea35fae6fdccde9e706b0d317d19d26bc05b34d589fec88"} Jul 13 09:17:35.635108 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:35.635080 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25" event={"ID":"5ccd1a61-cd7d-41c1-8236-76d04bc28b65","Type":"ContainerStarted","Data":"7698efbceaa99506dd272d10115f58841c7c14bdd7d30d75cf2ff0e9e39891d1"} Jul 13 09:17:35.642470 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:35.642453 2552 scope.go:117] "RemoveContainer" containerID="ee37e456e262963c8a660f1834fa55f5a95a4e1d03d95cb4a8f4661340eb26cf" Jul 13 09:17:35.642738 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:17:35.642713 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee37e456e262963c8a660f1834fa55f5a95a4e1d03d95cb4a8f4661340eb26cf\": container with ID starting with ee37e456e262963c8a660f1834fa55f5a95a4e1d03d95cb4a8f4661340eb26cf not found: ID does not exist" containerID="ee37e456e262963c8a660f1834fa55f5a95a4e1d03d95cb4a8f4661340eb26cf" Jul 13 09:17:35.642789 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:35.642752 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee37e456e262963c8a660f1834fa55f5a95a4e1d03d95cb4a8f4661340eb26cf"} err="failed to get container status \"ee37e456e262963c8a660f1834fa55f5a95a4e1d03d95cb4a8f4661340eb26cf\": rpc error: code = NotFound desc = could not find container \"ee37e456e262963c8a660f1834fa55f5a95a4e1d03d95cb4a8f4661340eb26cf\": container with ID starting with ee37e456e262963c8a660f1834fa55f5a95a4e1d03d95cb4a8f4661340eb26cf not found: ID does not exist" Jul 13 09:17:35.666468 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:35.666435 2552 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-local-certs\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:17:35.666468 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:35.666467 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82c78\" (UniqueName: \"kubernetes.io/projected/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-kube-api-access-82c78\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:17:35.666723 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:35.666481 2552 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-cacerts\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:17:35.666723 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:35.666493 2552 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-istio-csr-dns-cert\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:17:35.666723 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:35.666506 2552 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-istio-csr-ca-configmap\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:17:35.666723 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:35.666517 2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-istio-token\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:17:35.666723 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:35.666529 2552 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede-istio-kubeconfig\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:17:35.678875 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:35.678843 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn"] Jul 13 09:17:35.686890 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:35.686859 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-v6xxn"] Jul 13 09:17:35.720842 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:35.720778 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25" podStartSLOduration=1.868143331 podStartE2EDuration="4.720753324s" totalImagesPullingTime="2.852609993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:17:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:17:35.719679293 +0000 UTC m=+539.599634560" watchObservedRunningTime="2026-07-13 09:17:35.720753324 +0000 UTC m=+539.600708590" Jul 13 09:17:36.299350 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:36.299318 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25" Jul 13 09:17:36.305085 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:36.305051 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25" Jul 13 09:17:36.640052 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:36.640025 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25" Jul 13 09:17:36.640829 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:36.640809 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wzv25" Jul 13 09:17:36.671730 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:36.671694 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7c5b9b7-6dd3-4a22-97dd-9f1375beeede" path="/var/lib/kubelet/pods/f7c5b9b7-6dd3-4a22-97dd-9f1375beeede/volumes" Jul 13 09:17:38.651277 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:38.651237 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xxjmb" event={"ID":"0c3c450a-0391-4d1c-9db9-b9c1d3215a42","Type":"ContainerStarted","Data":"6fd8b4064bbace64c9cd0a63b86a1692e4eb4a2dcf4b8a7fd6ad3eb4dd36dc3d"} Jul 13 09:17:38.651715 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:38.651330 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xxjmb" Jul 13 09:17:38.671718 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:38.671668 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xxjmb" podStartSLOduration=2.771429831 podStartE2EDuration="5.671650183s" totalImagesPullingTime="2.900220352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:17:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:17:38.670179132 +0000 UTC m=+542.550134399" watchObservedRunningTime="2026-07-13 09:17:38.671650183 +0000 UTC m=+542.551605450" Jul 13 09:17:41.602653 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:41.602622 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-gm9js" Jul 13 09:17:41.604432 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:17:41.604412 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-lx9jr" Jul 13 09:18:01.268030 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:01.267997 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:18:03.217799 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:03.217761 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:18:03.319372 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:03.319336 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:18:08.540865 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:08.540834 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:18:09.661764 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:09.661735 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xxjmb" Jul 13 09:18:24.074533 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:24.074499 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:18:24.143973 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:24.143941 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:18:24.214811 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:24.214784 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xxxtk"] Jul 13 09:18:24.215122 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:24.215110 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7c5b9b7-6dd3-4a22-97dd-9f1375beeede" containerName="discovery" Jul 13 09:18:24.215164 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:24.215123 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7c5b9b7-6dd3-4a22-97dd-9f1375beeede" containerName="discovery" Jul 13 09:18:24.215197 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:24.215180 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7c5b9b7-6dd3-4a22-97dd-9f1375beeede" containerName="discovery" Jul 13 09:18:24.229496 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:24.229458 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xxxtk"] Jul 13 09:18:24.229657 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:24.229588 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xxxtk" Jul 13 09:18:24.232668 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:24.232643 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 13 09:18:24.234117 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:24.234097 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-pwjst\"" Jul 13 09:18:24.234337 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:24.234314 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 13 09:18:24.276471 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:24.276445 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:18:24.361411 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:24.361385 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qqcg\" (UniqueName: \"kubernetes.io/projected/56df9b4f-706b-4d8e-ba47-1f33bb384002-kube-api-access-5qqcg\") pod \"kuadrant-operator-catalog-xxxtk\" (UID: \"56df9b4f-706b-4d8e-ba47-1f33bb384002\") " pod="kuadrant-system/kuadrant-operator-catalog-xxxtk" Jul 13 09:18:24.461916 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:24.461891 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qqcg\" (UniqueName: \"kubernetes.io/projected/56df9b4f-706b-4d8e-ba47-1f33bb384002-kube-api-access-5qqcg\") pod \"kuadrant-operator-catalog-xxxtk\" (UID: \"56df9b4f-706b-4d8e-ba47-1f33bb384002\") " pod="kuadrant-system/kuadrant-operator-catalog-xxxtk" Jul 13 09:18:24.470915 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:24.470891 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qqcg\" (UniqueName: \"kubernetes.io/projected/56df9b4f-706b-4d8e-ba47-1f33bb384002-kube-api-access-5qqcg\") pod \"kuadrant-operator-catalog-xxxtk\" (UID: \"56df9b4f-706b-4d8e-ba47-1f33bb384002\") " pod="kuadrant-system/kuadrant-operator-catalog-xxxtk" Jul 13 09:18:24.540610 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:24.540581 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xxxtk" Jul 13 09:18:24.577768 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:24.577743 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xxxtk"] Jul 13 09:18:24.672106 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:24.672082 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xxxtk"] Jul 13 09:18:24.674094 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:18:24.674067 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56df9b4f_706b_4d8e_ba47_1f33bb384002.slice/crio-95b831624a3474edc12ad2ad439f3a6b7a7a444d3671425cca66034d3d07c581 WatchSource:0}: Error finding container 95b831624a3474edc12ad2ad439f3a6b7a7a444d3671425cca66034d3d07c581: Status 404 returned error can't find the container with id 95b831624a3474edc12ad2ad439f3a6b7a7a444d3671425cca66034d3d07c581 Jul 13 09:18:24.792145 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:24.792125 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h6cpr"] Jul 13 09:18:24.796877 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:24.796857 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-h6cpr" Jul 13 09:18:24.805285 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:24.805263 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h6cpr"] Jul 13 09:18:24.865264 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:24.865237 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xxxtk" event={"ID":"56df9b4f-706b-4d8e-ba47-1f33bb384002","Type":"ContainerStarted","Data":"95b831624a3474edc12ad2ad439f3a6b7a7a444d3671425cca66034d3d07c581"} Jul 13 09:18:24.965745 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:24.965682 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lknrn\" (UniqueName: \"kubernetes.io/projected/ed5c28f2-3236-47ce-9665-1c74fc79b889-kube-api-access-lknrn\") pod \"kuadrant-operator-catalog-h6cpr\" (UID: \"ed5c28f2-3236-47ce-9665-1c74fc79b889\") " pod="kuadrant-system/kuadrant-operator-catalog-h6cpr" Jul 13 09:18:25.066272 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:25.066243 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lknrn\" (UniqueName: \"kubernetes.io/projected/ed5c28f2-3236-47ce-9665-1c74fc79b889-kube-api-access-lknrn\") pod \"kuadrant-operator-catalog-h6cpr\" (UID: \"ed5c28f2-3236-47ce-9665-1c74fc79b889\") " pod="kuadrant-system/kuadrant-operator-catalog-h6cpr" Jul 13 09:18:25.075566 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:25.075535 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lknrn\" (UniqueName: \"kubernetes.io/projected/ed5c28f2-3236-47ce-9665-1c74fc79b889-kube-api-access-lknrn\") pod \"kuadrant-operator-catalog-h6cpr\" (UID: \"ed5c28f2-3236-47ce-9665-1c74fc79b889\") " pod="kuadrant-system/kuadrant-operator-catalog-h6cpr" Jul 13 09:18:25.107443 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:25.107417 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-h6cpr" Jul 13 09:18:25.235061 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:25.235026 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h6cpr"] Jul 13 09:18:25.236598 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:18:25.236572 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded5c28f2_3236_47ce_9665_1c74fc79b889.slice/crio-b9e3dec2448816bc31f3b8139c2f211e4fbb8f84c62a7d555aea83dfc12f5844 WatchSource:0}: Error finding container b9e3dec2448816bc31f3b8139c2f211e4fbb8f84c62a7d555aea83dfc12f5844: Status 404 returned error can't find the container with id b9e3dec2448816bc31f3b8139c2f211e4fbb8f84c62a7d555aea83dfc12f5844 Jul 13 09:18:25.875802 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:25.875761 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-h6cpr" event={"ID":"ed5c28f2-3236-47ce-9665-1c74fc79b889","Type":"ContainerStarted","Data":"b9e3dec2448816bc31f3b8139c2f211e4fbb8f84c62a7d555aea83dfc12f5844"} Jul 13 09:18:26.883733 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:26.883503 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xxxtk" event={"ID":"56df9b4f-706b-4d8e-ba47-1f33bb384002","Type":"ContainerStarted","Data":"d3e02bbcf9c73081ba2acd881ef49bdcb1ac1a858f10684a2607fd562be385c8"} Jul 13 09:18:26.883733 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:26.883527 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-xxxtk" podUID="56df9b4f-706b-4d8e-ba47-1f33bb384002" containerName="registry-server" containerID="cri-o://d3e02bbcf9c73081ba2acd881ef49bdcb1ac1a858f10684a2607fd562be385c8" gracePeriod=2 Jul 13 09:18:26.886278 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:26.886241 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-h6cpr" event={"ID":"ed5c28f2-3236-47ce-9665-1c74fc79b889","Type":"ContainerStarted","Data":"974011ab4bb0695f1042d98fa9ea299a3e9b782e46aa731255845d5d08534469"} Jul 13 09:18:26.902639 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:26.902599 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-xxxtk" podStartSLOduration=0.791714639 podStartE2EDuration="2.90258497s" totalImagesPullingTime="2.110870331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:18:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:18:26.901506064 +0000 UTC m=+590.781461331" watchObservedRunningTime="2026-07-13 09:18:26.90258497 +0000 UTC m=+590.782540236" Jul 13 09:18:26.920433 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:26.920393 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-h6cpr" podStartSLOduration=1.369449881 podStartE2EDuration="2.92037408s" totalImagesPullingTime="1.550924199s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:18:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:18:26.919720648 +0000 UTC m=+590.799675915" watchObservedRunningTime="2026-07-13 09:18:26.92037408 +0000 UTC m=+590.800329347" Jul 13 09:18:27.125092 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:27.125072 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-xxxtk_56df9b4f-706b-4d8e-ba47-1f33bb384002/registry-server/0.log" Jul 13 09:18:27.127399 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:27.127383 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xxxtk" Jul 13 09:18:27.287342 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:27.287316 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qqcg\" (UniqueName: \"kubernetes.io/projected/56df9b4f-706b-4d8e-ba47-1f33bb384002-kube-api-access-5qqcg\") pod \"56df9b4f-706b-4d8e-ba47-1f33bb384002\" (UID: \"56df9b4f-706b-4d8e-ba47-1f33bb384002\") " Jul 13 09:18:27.289446 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:27.289424 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56df9b4f-706b-4d8e-ba47-1f33bb384002-kube-api-access-5qqcg" (OuterVolumeSpecName: "kube-api-access-5qqcg") pod "56df9b4f-706b-4d8e-ba47-1f33bb384002" (UID: "56df9b4f-706b-4d8e-ba47-1f33bb384002"). InnerVolumeSpecName "kube-api-access-5qqcg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:18:27.388666 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:27.388642 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qqcg\" (UniqueName: \"kubernetes.io/projected/56df9b4f-706b-4d8e-ba47-1f33bb384002-kube-api-access-5qqcg\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:18:27.892939 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:27.892916 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-xxxtk_56df9b4f-706b-4d8e-ba47-1f33bb384002/registry-server/0.log" Jul 13 09:18:27.893267 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:27.892952 2552 generic.go:356] "Generic (PLEG): container finished" podID="56df9b4f-706b-4d8e-ba47-1f33bb384002" containerID="d3e02bbcf9c73081ba2acd881ef49bdcb1ac1a858f10684a2607fd562be385c8" exitCode=2 Jul 13 09:18:27.893267 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:27.892982 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xxxtk" event={"ID":"56df9b4f-706b-4d8e-ba47-1f33bb384002","Type":"ContainerDied","Data":"d3e02bbcf9c73081ba2acd881ef49bdcb1ac1a858f10684a2607fd562be385c8"} Jul 13 09:18:27.893267 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:27.893006 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xxxtk" Jul 13 09:18:27.893267 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:27.893035 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xxxtk" event={"ID":"56df9b4f-706b-4d8e-ba47-1f33bb384002","Type":"ContainerDied","Data":"95b831624a3474edc12ad2ad439f3a6b7a7a444d3671425cca66034d3d07c581"} Jul 13 09:18:27.893889 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:27.893056 2552 scope.go:117] "RemoveContainer" containerID="d3e02bbcf9c73081ba2acd881ef49bdcb1ac1a858f10684a2607fd562be385c8" Jul 13 09:18:27.902628 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:27.902611 2552 scope.go:117] "RemoveContainer" containerID="d3e02bbcf9c73081ba2acd881ef49bdcb1ac1a858f10684a2607fd562be385c8" Jul 13 09:18:27.902871 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:18:27.902853 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3e02bbcf9c73081ba2acd881ef49bdcb1ac1a858f10684a2607fd562be385c8\": container with ID starting with d3e02bbcf9c73081ba2acd881ef49bdcb1ac1a858f10684a2607fd562be385c8 not found: ID does not exist" containerID="d3e02bbcf9c73081ba2acd881ef49bdcb1ac1a858f10684a2607fd562be385c8" Jul 13 09:18:27.902915 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:27.902879 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3e02bbcf9c73081ba2acd881ef49bdcb1ac1a858f10684a2607fd562be385c8"} err="failed to get container status \"d3e02bbcf9c73081ba2acd881ef49bdcb1ac1a858f10684a2607fd562be385c8\": rpc error: code = NotFound desc = could not find container \"d3e02bbcf9c73081ba2acd881ef49bdcb1ac1a858f10684a2607fd562be385c8\": container with ID starting with d3e02bbcf9c73081ba2acd881ef49bdcb1ac1a858f10684a2607fd562be385c8 not found: ID does not exist" Jul 13 09:18:27.916761 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:27.916739 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xxxtk"] Jul 13 09:18:27.923305 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:27.923286 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xxxtk"] Jul 13 09:18:28.670661 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:28.670629 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56df9b4f-706b-4d8e-ba47-1f33bb384002" path="/var/lib/kubelet/pods/56df9b4f-706b-4d8e-ba47-1f33bb384002/volumes" Jul 13 09:18:35.108149 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:35.108114 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-h6cpr" Jul 13 09:18:35.108658 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:35.108166 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-h6cpr" Jul 13 09:18:35.130243 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:35.130195 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-h6cpr" Jul 13 09:18:35.954954 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:35.954928 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-h6cpr" Jul 13 09:18:36.573537 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:36.573510 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gp6mv_47250d22-56b6-4503-af3f-36a9e79691b2/ovn-acl-logging/0.log" Jul 13 09:18:36.574427 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:36.574408 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gp6mv_47250d22-56b6-4503-af3f-36a9e79691b2/ovn-acl-logging/0.log" Jul 13 09:18:39.416473 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:39.416439 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bgqtp"] Jul 13 09:18:39.418938 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:39.417321 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56df9b4f-706b-4d8e-ba47-1f33bb384002" containerName="registry-server" Jul 13 09:18:39.418938 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:39.417341 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="56df9b4f-706b-4d8e-ba47-1f33bb384002" containerName="registry-server" Jul 13 09:18:39.418938 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:39.417453 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="56df9b4f-706b-4d8e-ba47-1f33bb384002" containerName="registry-server" Jul 13 09:18:39.421335 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:39.421317 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bgqtp" Jul 13 09:18:39.425731 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:39.425710 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-mr8xt\"" Jul 13 09:18:39.442732 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:39.442712 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bgqtp"] Jul 13 09:18:39.478228 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:39.478179 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bgqtp\" (UID: \"1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bgqtp" Jul 13 09:18:39.478330 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:39.478292 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bgqtp\" (UID: \"1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bgqtp" Jul 13 09:18:39.478371 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:39.478332 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbv2s\" (UniqueName: \"kubernetes.io/projected/1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f-kube-api-access-zbv2s\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bgqtp\" (UID: \"1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bgqtp" Jul 13 09:18:39.579467 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:39.579440 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bgqtp\" (UID: \"1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bgqtp" Jul 13 09:18:39.579573 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:39.579516 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bgqtp\" (UID: \"1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bgqtp" Jul 13 09:18:39.579573 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:39.579556 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbv2s\" (UniqueName: \"kubernetes.io/projected/1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f-kube-api-access-zbv2s\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bgqtp\" (UID: \"1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bgqtp" Jul 13 09:18:39.579892 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:39.579869 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bgqtp\" (UID: \"1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bgqtp" Jul 13 09:18:39.579962 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:39.579881 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bgqtp\" (UID: \"1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bgqtp" Jul 13 09:18:39.589815 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:39.589790 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbv2s\" (UniqueName: \"kubernetes.io/projected/1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f-kube-api-access-zbv2s\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bgqtp\" (UID: \"1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bgqtp" Jul 13 09:18:39.736581 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:39.736519 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bgqtp" Jul 13 09:18:39.870859 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:39.870837 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bgqtp"] Jul 13 09:18:39.872955 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:18:39.872918 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d6c1ccf_a6aa_483f_94aa_a5fa8a316b5f.slice/crio-e74c93b4b2490078bf9c4a6f4be06c8f5325a4b9b34d21723781deefc48a9621 WatchSource:0}: Error finding container e74c93b4b2490078bf9c4a6f4be06c8f5325a4b9b34d21723781deefc48a9621: Status 404 returned error can't find the container with id e74c93b4b2490078bf9c4a6f4be06c8f5325a4b9b34d21723781deefc48a9621 Jul 13 09:18:39.955117 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:39.955087 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bgqtp" event={"ID":"1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f","Type":"ContainerStarted","Data":"acb520bf35cd5d1349c6b959ef98b7327e23bc8fe5931cc0b56f74cc8f830917"} Jul 13 09:18:39.955282 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:39.955135 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bgqtp" event={"ID":"1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f","Type":"ContainerStarted","Data":"e74c93b4b2490078bf9c4a6f4be06c8f5325a4b9b34d21723781deefc48a9621"} Jul 13 09:18:40.005634 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:40.005581 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gzc8d"] Jul 13 09:18:40.009243 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:40.009202 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gzc8d" Jul 13 09:18:40.018128 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:40.018103 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gzc8d"] Jul 13 09:18:40.083946 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:40.083923 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf0b4d43-1b14-4990-867d-7302291b8628-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gzc8d\" (UID: \"cf0b4d43-1b14-4990-867d-7302291b8628\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gzc8d" Jul 13 09:18:40.084048 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:40.083951 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fk4j9\" (UniqueName: \"kubernetes.io/projected/cf0b4d43-1b14-4990-867d-7302291b8628-kube-api-access-fk4j9\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gzc8d\" (UID: \"cf0b4d43-1b14-4990-867d-7302291b8628\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gzc8d" Jul 13 09:18:40.084048 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:40.083996 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf0b4d43-1b14-4990-867d-7302291b8628-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gzc8d\" (UID: \"cf0b4d43-1b14-4990-867d-7302291b8628\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gzc8d" Jul 13 09:18:40.184415 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:40.184389 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf0b4d43-1b14-4990-867d-7302291b8628-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gzc8d\" (UID: \"cf0b4d43-1b14-4990-867d-7302291b8628\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gzc8d" Jul 13 09:18:40.184508 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:40.184420 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fk4j9\" (UniqueName: \"kubernetes.io/projected/cf0b4d43-1b14-4990-867d-7302291b8628-kube-api-access-fk4j9\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gzc8d\" (UID: \"cf0b4d43-1b14-4990-867d-7302291b8628\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gzc8d" Jul 13 09:18:40.184569 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:40.184549 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf0b4d43-1b14-4990-867d-7302291b8628-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gzc8d\" (UID: \"cf0b4d43-1b14-4990-867d-7302291b8628\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gzc8d" Jul 13 09:18:40.184737 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:40.184721 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf0b4d43-1b14-4990-867d-7302291b8628-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gzc8d\" (UID: \"cf0b4d43-1b14-4990-867d-7302291b8628\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gzc8d" Jul 13 09:18:40.184839 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:40.184823 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf0b4d43-1b14-4990-867d-7302291b8628-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gzc8d\" (UID: \"cf0b4d43-1b14-4990-867d-7302291b8628\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gzc8d" Jul 13 09:18:40.194290 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:40.194270 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fk4j9\" (UniqueName: \"kubernetes.io/projected/cf0b4d43-1b14-4990-867d-7302291b8628-kube-api-access-fk4j9\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gzc8d\" (UID: \"cf0b4d43-1b14-4990-867d-7302291b8628\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gzc8d" Jul 13 09:18:40.352819 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:40.352797 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gzc8d" Jul 13 09:18:40.490578 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:40.485687 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gzc8d"] Jul 13 09:18:40.590767 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:40.590741 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g5qcn"] Jul 13 09:18:40.594152 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:40.594134 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g5qcn" Jul 13 09:18:40.603753 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:40.603702 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g5qcn"] Jul 13 09:18:40.688310 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:40.688290 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2d9fb3a-d67b-4422-a065-ad6c163aa1fb-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g5qcn\" (UID: \"f2d9fb3a-d67b-4422-a065-ad6c163aa1fb\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g5qcn" Jul 13 09:18:40.688411 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:40.688321 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2d9fb3a-d67b-4422-a065-ad6c163aa1fb-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g5qcn\" (UID: \"f2d9fb3a-d67b-4422-a065-ad6c163aa1fb\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g5qcn" Jul 13 09:18:40.688411 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:40.688350 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sz49t\" (UniqueName: \"kubernetes.io/projected/f2d9fb3a-d67b-4422-a065-ad6c163aa1fb-kube-api-access-sz49t\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g5qcn\" (UID: \"f2d9fb3a-d67b-4422-a065-ad6c163aa1fb\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g5qcn" Jul 13 09:18:40.788804 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:40.788778 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2d9fb3a-d67b-4422-a065-ad6c163aa1fb-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g5qcn\" (UID: \"f2d9fb3a-d67b-4422-a065-ad6c163aa1fb\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g5qcn" Jul 13 09:18:40.788892 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:40.788812 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2d9fb3a-d67b-4422-a065-ad6c163aa1fb-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g5qcn\" (UID: \"f2d9fb3a-d67b-4422-a065-ad6c163aa1fb\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g5qcn" Jul 13 09:18:40.788892 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:40.788838 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sz49t\" (UniqueName: \"kubernetes.io/projected/f2d9fb3a-d67b-4422-a065-ad6c163aa1fb-kube-api-access-sz49t\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g5qcn\" (UID: \"f2d9fb3a-d67b-4422-a065-ad6c163aa1fb\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g5qcn" Jul 13 09:18:40.789169 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:40.789146 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2d9fb3a-d67b-4422-a065-ad6c163aa1fb-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g5qcn\" (UID: \"f2d9fb3a-d67b-4422-a065-ad6c163aa1fb\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g5qcn" Jul 13 09:18:40.789231 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:40.789166 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2d9fb3a-d67b-4422-a065-ad6c163aa1fb-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g5qcn\" (UID: \"f2d9fb3a-d67b-4422-a065-ad6c163aa1fb\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g5qcn" Jul 13 09:18:40.798023 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:40.797997 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sz49t\" (UniqueName: \"kubernetes.io/projected/f2d9fb3a-d67b-4422-a065-ad6c163aa1fb-kube-api-access-sz49t\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g5qcn\" (UID: \"f2d9fb3a-d67b-4422-a065-ad6c163aa1fb\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g5qcn" Jul 13 09:18:40.947914 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:40.947855 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g5qcn" Jul 13 09:18:40.962107 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:40.962082 2552 generic.go:356] "Generic (PLEG): container finished" podID="1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f" containerID="acb520bf35cd5d1349c6b959ef98b7327e23bc8fe5931cc0b56f74cc8f830917" exitCode=0 Jul 13 09:18:40.962270 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:40.962200 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bgqtp" event={"ID":"1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f","Type":"ContainerDied","Data":"acb520bf35cd5d1349c6b959ef98b7327e23bc8fe5931cc0b56f74cc8f830917"} Jul 13 09:18:40.964354 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:40.964331 2552 generic.go:356] "Generic (PLEG): container finished" podID="cf0b4d43-1b14-4990-867d-7302291b8628" containerID="a2b949a26809626930ddae2644bb40b3c8cde37c9a7b4abc178807cf3b9f960b" exitCode=0 Jul 13 09:18:40.964450 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:40.964409 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gzc8d" event={"ID":"cf0b4d43-1b14-4990-867d-7302291b8628","Type":"ContainerDied","Data":"a2b949a26809626930ddae2644bb40b3c8cde37c9a7b4abc178807cf3b9f960b"} Jul 13 09:18:40.964506 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:40.964448 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gzc8d" event={"ID":"cf0b4d43-1b14-4990-867d-7302291b8628","Type":"ContainerStarted","Data":"b60f51c19eb2c6c94e191ee03861627962bc59286fcdfbef4c372117a4996768"} Jul 13 09:18:40.995875 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:40.995851 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739v6hc"] Jul 13 09:18:41.000853 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:41.000737 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739v6hc" Jul 13 09:18:41.014362 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:41.014334 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739v6hc"] Jul 13 09:18:41.086086 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:41.086064 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g5qcn"] Jul 13 09:18:41.087766 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:18:41.087739 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2d9fb3a_d67b_4422_a065_ad6c163aa1fb.slice/crio-cf9075435c1223a6d0e3a0988ff2d3e309124f003f90bfd54b3ca2f52abea60b WatchSource:0}: Error finding container cf9075435c1223a6d0e3a0988ff2d3e309124f003f90bfd54b3ca2f52abea60b: Status 404 returned error can't find the container with id cf9075435c1223a6d0e3a0988ff2d3e309124f003f90bfd54b3ca2f52abea60b Jul 13 09:18:41.191059 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:41.191032 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e20a36e-85ef-4cf3-b242-318b75700abe-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739v6hc\" (UID: \"2e20a36e-85ef-4cf3-b242-318b75700abe\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739v6hc" Jul 13 09:18:41.191148 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:41.191074 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x82w8\" (UniqueName: \"kubernetes.io/projected/2e20a36e-85ef-4cf3-b242-318b75700abe-kube-api-access-x82w8\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739v6hc\" (UID: \"2e20a36e-85ef-4cf3-b242-318b75700abe\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739v6hc" Jul 13 09:18:41.191189 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:41.191149 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e20a36e-85ef-4cf3-b242-318b75700abe-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739v6hc\" (UID: \"2e20a36e-85ef-4cf3-b242-318b75700abe\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739v6hc" Jul 13 09:18:41.292053 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:41.292002 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e20a36e-85ef-4cf3-b242-318b75700abe-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739v6hc\" (UID: \"2e20a36e-85ef-4cf3-b242-318b75700abe\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739v6hc" Jul 13 09:18:41.292053 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:41.292045 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e20a36e-85ef-4cf3-b242-318b75700abe-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739v6hc\" (UID: \"2e20a36e-85ef-4cf3-b242-318b75700abe\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739v6hc" Jul 13 09:18:41.292193 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:41.292069 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x82w8\" (UniqueName: \"kubernetes.io/projected/2e20a36e-85ef-4cf3-b242-318b75700abe-kube-api-access-x82w8\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739v6hc\" (UID: \"2e20a36e-85ef-4cf3-b242-318b75700abe\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739v6hc" Jul 13 09:18:41.292392 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:41.292370 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e20a36e-85ef-4cf3-b242-318b75700abe-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739v6hc\" (UID: \"2e20a36e-85ef-4cf3-b242-318b75700abe\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739v6hc" Jul 13 09:18:41.292469 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:41.292424 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e20a36e-85ef-4cf3-b242-318b75700abe-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739v6hc\" (UID: \"2e20a36e-85ef-4cf3-b242-318b75700abe\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739v6hc" Jul 13 09:18:41.302370 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:41.302354 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x82w8\" (UniqueName: \"kubernetes.io/projected/2e20a36e-85ef-4cf3-b242-318b75700abe-kube-api-access-x82w8\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739v6hc\" (UID: \"2e20a36e-85ef-4cf3-b242-318b75700abe\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739v6hc" Jul 13 09:18:41.313202 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:41.313181 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739v6hc" Jul 13 09:18:41.447440 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:41.447415 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739v6hc"] Jul 13 09:18:41.448308 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:18:41.448282 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e20a36e_85ef_4cf3_b242_318b75700abe.slice/crio-3954dda92ca794303a709dc435e01d6733df7702ae481ab553ec0c81c64619ea WatchSource:0}: Error finding container 3954dda92ca794303a709dc435e01d6733df7702ae481ab553ec0c81c64619ea: Status 404 returned error can't find the container with id 3954dda92ca794303a709dc435e01d6733df7702ae481ab553ec0c81c64619ea Jul 13 09:18:41.972893 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:41.972858 2552 generic.go:356] "Generic (PLEG): container finished" podID="f2d9fb3a-d67b-4422-a065-ad6c163aa1fb" containerID="002477b3257140d08016ecc6ce70369bdfe5551adf5b99aff41d2e1edcf9138b" exitCode=0 Jul 13 09:18:41.973322 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:41.972942 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g5qcn" event={"ID":"f2d9fb3a-d67b-4422-a065-ad6c163aa1fb","Type":"ContainerDied","Data":"002477b3257140d08016ecc6ce70369bdfe5551adf5b99aff41d2e1edcf9138b"} Jul 13 09:18:41.973322 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:41.972991 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g5qcn" event={"ID":"f2d9fb3a-d67b-4422-a065-ad6c163aa1fb","Type":"ContainerStarted","Data":"cf9075435c1223a6d0e3a0988ff2d3e309124f003f90bfd54b3ca2f52abea60b"} Jul 13 09:18:41.975001 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:41.974979 2552 generic.go:356] "Generic (PLEG): container finished" podID="2e20a36e-85ef-4cf3-b242-318b75700abe" containerID="fc728a6924cfe2c8c9f3e9a1982ef7fa4e2fa1472e9bc74f9dad7541c64ec817" exitCode=0 Jul 13 09:18:41.975120 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:41.975031 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739v6hc" event={"ID":"2e20a36e-85ef-4cf3-b242-318b75700abe","Type":"ContainerDied","Data":"fc728a6924cfe2c8c9f3e9a1982ef7fa4e2fa1472e9bc74f9dad7541c64ec817"} Jul 13 09:18:41.975120 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:41.975076 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739v6hc" event={"ID":"2e20a36e-85ef-4cf3-b242-318b75700abe","Type":"ContainerStarted","Data":"3954dda92ca794303a709dc435e01d6733df7702ae481ab553ec0c81c64619ea"} Jul 13 09:18:42.982587 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:42.982188 2552 generic.go:356] "Generic (PLEG): container finished" podID="1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f" containerID="71df29104e4f2f9cb36bcd1ef7afad8e1b8becdbe5de6f8171cf8b4d73a3cb1a" exitCode=0 Jul 13 09:18:42.982587 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:42.982304 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bgqtp" event={"ID":"1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f","Type":"ContainerDied","Data":"71df29104e4f2f9cb36bcd1ef7afad8e1b8becdbe5de6f8171cf8b4d73a3cb1a"} Jul 13 09:18:42.984750 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:42.984729 2552 generic.go:356] "Generic (PLEG): container finished" podID="f2d9fb3a-d67b-4422-a065-ad6c163aa1fb" containerID="c7dcd7903fd5a35acb3d9f1eb43c112d0148e5862f31594740ba98c0441e88a6" exitCode=0 Jul 13 09:18:42.984845 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:42.984822 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g5qcn" event={"ID":"f2d9fb3a-d67b-4422-a065-ad6c163aa1fb","Type":"ContainerDied","Data":"c7dcd7903fd5a35acb3d9f1eb43c112d0148e5862f31594740ba98c0441e88a6"} Jul 13 09:18:42.992996 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:42.992883 2552 generic.go:356] "Generic (PLEG): container finished" podID="cf0b4d43-1b14-4990-867d-7302291b8628" containerID="22e770ddf3e0d06e558a57e77f670f27780964411f0614b79df2005457c73d9d" exitCode=0 Jul 13 09:18:42.992996 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:42.992942 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gzc8d" event={"ID":"cf0b4d43-1b14-4990-867d-7302291b8628","Type":"ContainerDied","Data":"22e770ddf3e0d06e558a57e77f670f27780964411f0614b79df2005457c73d9d"} Jul 13 09:18:44.000496 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:44.000465 2552 generic.go:356] "Generic (PLEG): container finished" podID="cf0b4d43-1b14-4990-867d-7302291b8628" containerID="11184e0ce397c89e6883e62d3a5a6911d36ee138624d286ba6b036b55390b76f" exitCode=0 Jul 13 09:18:44.000831 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:44.000556 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gzc8d" event={"ID":"cf0b4d43-1b14-4990-867d-7302291b8628","Type":"ContainerDied","Data":"11184e0ce397c89e6883e62d3a5a6911d36ee138624d286ba6b036b55390b76f"} Jul 13 09:18:44.002366 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:44.002343 2552 generic.go:356] "Generic (PLEG): container finished" podID="2e20a36e-85ef-4cf3-b242-318b75700abe" containerID="cf2fa53d9b62ffb048210147c6f50d80e62bb19346109ccdd4f5cd2111c6066e" exitCode=0 Jul 13 09:18:44.002481 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:44.002424 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739v6hc" event={"ID":"2e20a36e-85ef-4cf3-b242-318b75700abe","Type":"ContainerDied","Data":"cf2fa53d9b62ffb048210147c6f50d80e62bb19346109ccdd4f5cd2111c6066e"} Jul 13 09:18:44.004591 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:44.004568 2552 generic.go:356] "Generic (PLEG): container finished" podID="1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f" containerID="4961f75da7b8e4f144874e8c79c8e57662ca0c062207d75b95f7df25b1d0c389" exitCode=0 Jul 13 09:18:44.004739 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:44.004595 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bgqtp" event={"ID":"1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f","Type":"ContainerDied","Data":"4961f75da7b8e4f144874e8c79c8e57662ca0c062207d75b95f7df25b1d0c389"} Jul 13 09:18:44.006722 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:44.006702 2552 generic.go:356] "Generic (PLEG): container finished" podID="f2d9fb3a-d67b-4422-a065-ad6c163aa1fb" containerID="b881b770dbb1b58ca209be8ff520763f2cc8ffe5340acb53f7413f255deb3670" exitCode=0 Jul 13 09:18:44.006819 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:44.006738 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g5qcn" event={"ID":"f2d9fb3a-d67b-4422-a065-ad6c163aa1fb","Type":"ContainerDied","Data":"b881b770dbb1b58ca209be8ff520763f2cc8ffe5340acb53f7413f255deb3670"} Jul 13 09:18:45.013358 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:45.013328 2552 generic.go:356] "Generic (PLEG): container finished" podID="2e20a36e-85ef-4cf3-b242-318b75700abe" containerID="3acf39789c90141235bcfb0d2c94eeef1ae2c8ce619625433283d81802f3c25f" exitCode=0 Jul 13 09:18:45.013760 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:45.013409 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739v6hc" event={"ID":"2e20a36e-85ef-4cf3-b242-318b75700abe","Type":"ContainerDied","Data":"3acf39789c90141235bcfb0d2c94eeef1ae2c8ce619625433283d81802f3c25f"} Jul 13 09:18:45.166479 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:45.166454 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gzc8d" Jul 13 09:18:45.185052 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:45.185034 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g5qcn" Jul 13 09:18:45.208995 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:45.208965 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bgqtp" Jul 13 09:18:45.220748 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:45.220726 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbv2s\" (UniqueName: \"kubernetes.io/projected/1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f-kube-api-access-zbv2s\") pod \"1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f\" (UID: \"1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f\") " Jul 13 09:18:45.220850 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:45.220757 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sz49t\" (UniqueName: \"kubernetes.io/projected/f2d9fb3a-d67b-4422-a065-ad6c163aa1fb-kube-api-access-sz49t\") pod \"f2d9fb3a-d67b-4422-a065-ad6c163aa1fb\" (UID: \"f2d9fb3a-d67b-4422-a065-ad6c163aa1fb\") " Jul 13 09:18:45.220850 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:45.220804 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f-util\") pod \"1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f\" (UID: \"1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f\") " Jul 13 09:18:45.220850 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:45.220827 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2d9fb3a-d67b-4422-a065-ad6c163aa1fb-bundle\") pod \"f2d9fb3a-d67b-4422-a065-ad6c163aa1fb\" (UID: \"f2d9fb3a-d67b-4422-a065-ad6c163aa1fb\") " Jul 13 09:18:45.221007 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:45.220860 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf0b4d43-1b14-4990-867d-7302291b8628-bundle\") pod \"cf0b4d43-1b14-4990-867d-7302291b8628\" (UID: \"cf0b4d43-1b14-4990-867d-7302291b8628\") " Jul 13 09:18:45.221007 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:45.220911 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f-bundle\") pod \"1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f\" (UID: \"1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f\") " Jul 13 09:18:45.221007 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:45.220936 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf0b4d43-1b14-4990-867d-7302291b8628-util\") pod \"cf0b4d43-1b14-4990-867d-7302291b8628\" (UID: \"cf0b4d43-1b14-4990-867d-7302291b8628\") " Jul 13 09:18:45.221007 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:45.220973 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fk4j9\" (UniqueName: \"kubernetes.io/projected/cf0b4d43-1b14-4990-867d-7302291b8628-kube-api-access-fk4j9\") pod \"cf0b4d43-1b14-4990-867d-7302291b8628\" (UID: \"cf0b4d43-1b14-4990-867d-7302291b8628\") " Jul 13 09:18:45.221230 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:45.221057 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2d9fb3a-d67b-4422-a065-ad6c163aa1fb-util\") pod \"f2d9fb3a-d67b-4422-a065-ad6c163aa1fb\" (UID: \"f2d9fb3a-d67b-4422-a065-ad6c163aa1fb\") " Jul 13 09:18:45.221845 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:45.221692 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf0b4d43-1b14-4990-867d-7302291b8628-bundle" (OuterVolumeSpecName: "bundle") pod "cf0b4d43-1b14-4990-867d-7302291b8628" (UID: "cf0b4d43-1b14-4990-867d-7302291b8628"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:18:45.221845 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:45.221769 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2d9fb3a-d67b-4422-a065-ad6c163aa1fb-bundle" (OuterVolumeSpecName: "bundle") pod "f2d9fb3a-d67b-4422-a065-ad6c163aa1fb" (UID: "f2d9fb3a-d67b-4422-a065-ad6c163aa1fb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:18:45.222341 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:45.221848 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f-bundle" (OuterVolumeSpecName: "bundle") pod "1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f" (UID: "1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:18:45.223325 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:45.223291 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2d9fb3a-d67b-4422-a065-ad6c163aa1fb-kube-api-access-sz49t" (OuterVolumeSpecName: "kube-api-access-sz49t") pod "f2d9fb3a-d67b-4422-a065-ad6c163aa1fb" (UID: "f2d9fb3a-d67b-4422-a065-ad6c163aa1fb"). InnerVolumeSpecName "kube-api-access-sz49t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:18:45.224248 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:45.224194 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f-kube-api-access-zbv2s" (OuterVolumeSpecName: "kube-api-access-zbv2s") pod "1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f" (UID: "1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f"). InnerVolumeSpecName "kube-api-access-zbv2s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:18:45.224349 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:45.224255 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf0b4d43-1b14-4990-867d-7302291b8628-kube-api-access-fk4j9" (OuterVolumeSpecName: "kube-api-access-fk4j9") pod "cf0b4d43-1b14-4990-867d-7302291b8628" (UID: "cf0b4d43-1b14-4990-867d-7302291b8628"). InnerVolumeSpecName "kube-api-access-fk4j9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:18:45.228296 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:45.228270 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf0b4d43-1b14-4990-867d-7302291b8628-util" (OuterVolumeSpecName: "util") pod "cf0b4d43-1b14-4990-867d-7302291b8628" (UID: "cf0b4d43-1b14-4990-867d-7302291b8628"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:18:45.228622 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:45.228603 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2d9fb3a-d67b-4422-a065-ad6c163aa1fb-util" (OuterVolumeSpecName: "util") pod "f2d9fb3a-d67b-4422-a065-ad6c163aa1fb" (UID: "f2d9fb3a-d67b-4422-a065-ad6c163aa1fb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:18:45.321787 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:45.321725 2552 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2d9fb3a-d67b-4422-a065-ad6c163aa1fb-util\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:18:45.321787 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:45.321748 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbv2s\" (UniqueName: \"kubernetes.io/projected/1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f-kube-api-access-zbv2s\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:18:45.321787 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:45.321763 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sz49t\" (UniqueName: \"kubernetes.io/projected/f2d9fb3a-d67b-4422-a065-ad6c163aa1fb-kube-api-access-sz49t\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:18:45.321787 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:45.321777 2552 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2d9fb3a-d67b-4422-a065-ad6c163aa1fb-bundle\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:18:45.321996 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:45.321793 2552 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf0b4d43-1b14-4990-867d-7302291b8628-bundle\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:18:45.321996 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:45.321804 2552 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f-bundle\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:18:45.321996 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:45.321814 2552 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf0b4d43-1b14-4990-867d-7302291b8628-util\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:18:45.321996 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:45.321824 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fk4j9\" (UniqueName: \"kubernetes.io/projected/cf0b4d43-1b14-4990-867d-7302291b8628-kube-api-access-fk4j9\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:18:45.341564 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:45.341536 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f-util" (OuterVolumeSpecName: "util") pod "1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f" (UID: "1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:18:45.422114 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:45.422092 2552 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f-util\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:18:46.020189 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:46.020164 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g5qcn" Jul 13 09:18:46.020189 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:46.020183 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g5qcn" event={"ID":"f2d9fb3a-d67b-4422-a065-ad6c163aa1fb","Type":"ContainerDied","Data":"cf9075435c1223a6d0e3a0988ff2d3e309124f003f90bfd54b3ca2f52abea60b"} Jul 13 09:18:46.020641 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:46.020246 2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cf9075435c1223a6d0e3a0988ff2d3e309124f003f90bfd54b3ca2f52abea60b" Jul 13 09:18:46.022122 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:46.022104 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gzc8d" Jul 13 09:18:46.022256 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:46.022142 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gzc8d" event={"ID":"cf0b4d43-1b14-4990-867d-7302291b8628","Type":"ContainerDied","Data":"b60f51c19eb2c6c94e191ee03861627962bc59286fcdfbef4c372117a4996768"} Jul 13 09:18:46.022256 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:46.022183 2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b60f51c19eb2c6c94e191ee03861627962bc59286fcdfbef4c372117a4996768" Jul 13 09:18:46.024315 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:46.024290 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bgqtp" event={"ID":"1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f","Type":"ContainerDied","Data":"e74c93b4b2490078bf9c4a6f4be06c8f5325a4b9b34d21723781deefc48a9621"} Jul 13 09:18:46.024416 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:46.024335 2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e74c93b4b2490078bf9c4a6f4be06c8f5325a4b9b34d21723781deefc48a9621" Jul 13 09:18:46.024492 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:46.024477 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bgqtp" Jul 13 09:18:46.163026 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:46.163002 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739v6hc" Jul 13 09:18:46.227565 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:46.227530 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x82w8\" (UniqueName: \"kubernetes.io/projected/2e20a36e-85ef-4cf3-b242-318b75700abe-kube-api-access-x82w8\") pod \"2e20a36e-85ef-4cf3-b242-318b75700abe\" (UID: \"2e20a36e-85ef-4cf3-b242-318b75700abe\") " Jul 13 09:18:46.229505 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:46.229479 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e20a36e-85ef-4cf3-b242-318b75700abe-kube-api-access-x82w8" (OuterVolumeSpecName: "kube-api-access-x82w8") pod "2e20a36e-85ef-4cf3-b242-318b75700abe" (UID: "2e20a36e-85ef-4cf3-b242-318b75700abe"). InnerVolumeSpecName "kube-api-access-x82w8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:18:46.327920 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:46.327896 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e20a36e-85ef-4cf3-b242-318b75700abe-util\") pod \"2e20a36e-85ef-4cf3-b242-318b75700abe\" (UID: \"2e20a36e-85ef-4cf3-b242-318b75700abe\") " Jul 13 09:18:46.328024 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:46.327924 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e20a36e-85ef-4cf3-b242-318b75700abe-bundle\") pod \"2e20a36e-85ef-4cf3-b242-318b75700abe\" (UID: \"2e20a36e-85ef-4cf3-b242-318b75700abe\") " Jul 13 09:18:46.328102 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:46.328086 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x82w8\" (UniqueName: \"kubernetes.io/projected/2e20a36e-85ef-4cf3-b242-318b75700abe-kube-api-access-x82w8\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:18:46.328562 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:46.328539 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e20a36e-85ef-4cf3-b242-318b75700abe-bundle" (OuterVolumeSpecName: "bundle") pod "2e20a36e-85ef-4cf3-b242-318b75700abe" (UID: "2e20a36e-85ef-4cf3-b242-318b75700abe"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:18:46.333015 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:46.332993 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e20a36e-85ef-4cf3-b242-318b75700abe-util" (OuterVolumeSpecName: "util") pod "2e20a36e-85ef-4cf3-b242-318b75700abe" (UID: "2e20a36e-85ef-4cf3-b242-318b75700abe"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:18:46.428448 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:46.428426 2552 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e20a36e-85ef-4cf3-b242-318b75700abe-util\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:18:46.428448 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:46.428447 2552 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e20a36e-85ef-4cf3-b242-318b75700abe-bundle\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:18:47.031057 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:47.030974 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739v6hc" event={"ID":"2e20a36e-85ef-4cf3-b242-318b75700abe","Type":"ContainerDied","Data":"3954dda92ca794303a709dc435e01d6733df7702ae481ab553ec0c81c64619ea"} Jul 13 09:18:47.031057 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:47.031029 2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3954dda92ca794303a709dc435e01d6733df7702ae481ab553ec0c81c64619ea" Jul 13 09:18:47.031057 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:47.031032 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739v6hc" Jul 13 09:18:49.920152 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:49.920122 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f648df7b4-7n2bp"] Jul 13 09:18:49.920550 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:49.920472 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f" containerName="pull" Jul 13 09:18:49.920550 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:49.920486 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f" containerName="pull" Jul 13 09:18:49.920550 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:49.920499 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f" containerName="extract" Jul 13 09:18:49.920550 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:49.920507 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f" containerName="extract" Jul 13 09:18:49.920550 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:49.920518 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f" containerName="util" Jul 13 09:18:49.920550 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:49.920527 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f" containerName="util" Jul 13 09:18:49.920550 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:49.920538 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e20a36e-85ef-4cf3-b242-318b75700abe" containerName="util" Jul 13 09:18:49.920550 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:49.920547 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e20a36e-85ef-4cf3-b242-318b75700abe" containerName="util" Jul 13 09:18:49.920792 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:49.920560 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2d9fb3a-d67b-4422-a065-ad6c163aa1fb" containerName="extract" Jul 13 09:18:49.920792 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:49.920568 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2d9fb3a-d67b-4422-a065-ad6c163aa1fb" containerName="extract" Jul 13 09:18:49.920792 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:49.920579 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2d9fb3a-d67b-4422-a065-ad6c163aa1fb" containerName="util" Jul 13 09:18:49.920792 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:49.920586 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2d9fb3a-d67b-4422-a065-ad6c163aa1fb" containerName="util" Jul 13 09:18:49.920792 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:49.920599 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf0b4d43-1b14-4990-867d-7302291b8628" containerName="util" Jul 13 09:18:49.920792 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:49.920608 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf0b4d43-1b14-4990-867d-7302291b8628" containerName="util" Jul 13 09:18:49.920792 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:49.920620 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e20a36e-85ef-4cf3-b242-318b75700abe" containerName="pull" Jul 13 09:18:49.920792 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:49.920628 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e20a36e-85ef-4cf3-b242-318b75700abe" containerName="pull" Jul 13 09:18:49.920792 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:49.920642 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2d9fb3a-d67b-4422-a065-ad6c163aa1fb" containerName="pull" Jul 13 09:18:49.920792 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:49.920649 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2d9fb3a-d67b-4422-a065-ad6c163aa1fb" containerName="pull" Jul 13 09:18:49.920792 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:49.920661 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf0b4d43-1b14-4990-867d-7302291b8628" containerName="extract" Jul 13 09:18:49.920792 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:49.920669 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf0b4d43-1b14-4990-867d-7302291b8628" containerName="extract" Jul 13 09:18:49.920792 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:49.920684 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf0b4d43-1b14-4990-867d-7302291b8628" containerName="pull" Jul 13 09:18:49.920792 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:49.920692 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf0b4d43-1b14-4990-867d-7302291b8628" containerName="pull" Jul 13 09:18:49.920792 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:49.920701 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e20a36e-85ef-4cf3-b242-318b75700abe" containerName="extract" Jul 13 09:18:49.920792 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:49.920706 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e20a36e-85ef-4cf3-b242-318b75700abe" containerName="extract" Jul 13 09:18:49.920792 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:49.920778 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2d9fb3a-d67b-4422-a065-ad6c163aa1fb" containerName="extract" Jul 13 09:18:49.920792 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:49.920785 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf0b4d43-1b14-4990-867d-7302291b8628" containerName="extract" Jul 13 09:18:49.920792 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:49.920792 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f" containerName="extract" Jul 13 09:18:49.920792 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:49.920800 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e20a36e-85ef-4cf3-b242-318b75700abe" containerName="extract" Jul 13 09:18:49.924839 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:49.924822 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f648df7b4-7n2bp" Jul 13 09:18:49.936070 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:49.936047 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f648df7b4-7n2bp"] Jul 13 09:18:49.954953 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:49.954932 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d3775975-f09b-4db7-8711-18775737dfd5-oauth-serving-cert\") pod \"console-6f648df7b4-7n2bp\" (UID: \"d3775975-f09b-4db7-8711-18775737dfd5\") " pod="openshift-console/console-6f648df7b4-7n2bp" Jul 13 09:18:49.955039 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:49.954962 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3775975-f09b-4db7-8711-18775737dfd5-trusted-ca-bundle\") pod \"console-6f648df7b4-7n2bp\" (UID: \"d3775975-f09b-4db7-8711-18775737dfd5\") " pod="openshift-console/console-6f648df7b4-7n2bp" Jul 13 09:18:49.955039 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:49.954988 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d3775975-f09b-4db7-8711-18775737dfd5-console-oauth-config\") pod \"console-6f648df7b4-7n2bp\" (UID: \"d3775975-f09b-4db7-8711-18775737dfd5\") " pod="openshift-console/console-6f648df7b4-7n2bp" Jul 13 09:18:49.955112 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:49.955040 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d3775975-f09b-4db7-8711-18775737dfd5-console-serving-cert\") pod \"console-6f648df7b4-7n2bp\" (UID: \"d3775975-f09b-4db7-8711-18775737dfd5\") " pod="openshift-console/console-6f648df7b4-7n2bp" Jul 13 09:18:49.955112 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:49.955069 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d3775975-f09b-4db7-8711-18775737dfd5-console-config\") pod \"console-6f648df7b4-7n2bp\" (UID: \"d3775975-f09b-4db7-8711-18775737dfd5\") " pod="openshift-console/console-6f648df7b4-7n2bp" Jul 13 09:18:49.955112 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:49.955108 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjccw\" (UniqueName: \"kubernetes.io/projected/d3775975-f09b-4db7-8711-18775737dfd5-kube-api-access-jjccw\") pod \"console-6f648df7b4-7n2bp\" (UID: \"d3775975-f09b-4db7-8711-18775737dfd5\") " pod="openshift-console/console-6f648df7b4-7n2bp" Jul 13 09:18:49.955221 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:49.955153 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d3775975-f09b-4db7-8711-18775737dfd5-service-ca\") pod \"console-6f648df7b4-7n2bp\" (UID: \"d3775975-f09b-4db7-8711-18775737dfd5\") " pod="openshift-console/console-6f648df7b4-7n2bp" Jul 13 09:18:50.054740 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:50.054672 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:18:50.055710 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:50.055692 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjccw\" (UniqueName: \"kubernetes.io/projected/d3775975-f09b-4db7-8711-18775737dfd5-kube-api-access-jjccw\") pod \"console-6f648df7b4-7n2bp\" (UID: \"d3775975-f09b-4db7-8711-18775737dfd5\") " pod="openshift-console/console-6f648df7b4-7n2bp" Jul 13 09:18:50.055765 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:50.055725 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d3775975-f09b-4db7-8711-18775737dfd5-service-ca\") pod \"console-6f648df7b4-7n2bp\" (UID: \"d3775975-f09b-4db7-8711-18775737dfd5\") " pod="openshift-console/console-6f648df7b4-7n2bp" Jul 13 09:18:50.055929 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:50.055912 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d3775975-f09b-4db7-8711-18775737dfd5-oauth-serving-cert\") pod \"console-6f648df7b4-7n2bp\" (UID: \"d3775975-f09b-4db7-8711-18775737dfd5\") " pod="openshift-console/console-6f648df7b4-7n2bp" Jul 13 09:18:50.055998 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:50.055953 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3775975-f09b-4db7-8711-18775737dfd5-trusted-ca-bundle\") pod \"console-6f648df7b4-7n2bp\" (UID: \"d3775975-f09b-4db7-8711-18775737dfd5\") " pod="openshift-console/console-6f648df7b4-7n2bp" Jul 13 09:18:50.056051 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:50.056006 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d3775975-f09b-4db7-8711-18775737dfd5-console-oauth-config\") pod \"console-6f648df7b4-7n2bp\" (UID: \"d3775975-f09b-4db7-8711-18775737dfd5\") " pod="openshift-console/console-6f648df7b4-7n2bp" Jul 13 09:18:50.056051 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:50.056038 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d3775975-f09b-4db7-8711-18775737dfd5-console-serving-cert\") pod \"console-6f648df7b4-7n2bp\" (UID: \"d3775975-f09b-4db7-8711-18775737dfd5\") " pod="openshift-console/console-6f648df7b4-7n2bp" Jul 13 09:18:50.056143 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:50.056067 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d3775975-f09b-4db7-8711-18775737dfd5-console-config\") pod \"console-6f648df7b4-7n2bp\" (UID: \"d3775975-f09b-4db7-8711-18775737dfd5\") " pod="openshift-console/console-6f648df7b4-7n2bp" Jul 13 09:18:50.056614 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:50.056589 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d3775975-f09b-4db7-8711-18775737dfd5-service-ca\") pod \"console-6f648df7b4-7n2bp\" (UID: \"d3775975-f09b-4db7-8711-18775737dfd5\") " pod="openshift-console/console-6f648df7b4-7n2bp" Jul 13 09:18:50.056748 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:50.056589 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d3775975-f09b-4db7-8711-18775737dfd5-oauth-serving-cert\") pod \"console-6f648df7b4-7n2bp\" (UID: \"d3775975-f09b-4db7-8711-18775737dfd5\") " pod="openshift-console/console-6f648df7b4-7n2bp" Jul 13 09:18:50.056816 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:50.056772 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d3775975-f09b-4db7-8711-18775737dfd5-console-config\") pod \"console-6f648df7b4-7n2bp\" (UID: \"d3775975-f09b-4db7-8711-18775737dfd5\") " pod="openshift-console/console-6f648df7b4-7n2bp" Jul 13 09:18:50.056816 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:50.056794 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3775975-f09b-4db7-8711-18775737dfd5-trusted-ca-bundle\") pod \"console-6f648df7b4-7n2bp\" (UID: \"d3775975-f09b-4db7-8711-18775737dfd5\") " pod="openshift-console/console-6f648df7b4-7n2bp" Jul 13 09:18:50.058429 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:50.058403 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d3775975-f09b-4db7-8711-18775737dfd5-console-oauth-config\") pod \"console-6f648df7b4-7n2bp\" (UID: \"d3775975-f09b-4db7-8711-18775737dfd5\") " pod="openshift-console/console-6f648df7b4-7n2bp" Jul 13 09:18:50.064355 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:50.058729 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d3775975-f09b-4db7-8711-18775737dfd5-console-serving-cert\") pod \"console-6f648df7b4-7n2bp\" (UID: \"d3775975-f09b-4db7-8711-18775737dfd5\") " pod="openshift-console/console-6f648df7b4-7n2bp" Jul 13 09:18:50.067027 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:50.067004 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjccw\" (UniqueName: \"kubernetes.io/projected/d3775975-f09b-4db7-8711-18775737dfd5-kube-api-access-jjccw\") pod \"console-6f648df7b4-7n2bp\" (UID: \"d3775975-f09b-4db7-8711-18775737dfd5\") " pod="openshift-console/console-6f648df7b4-7n2bp" Jul 13 09:18:50.154845 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:50.154821 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:18:50.233454 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:50.233393 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f648df7b4-7n2bp" Jul 13 09:18:50.376429 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:50.376405 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f648df7b4-7n2bp"] Jul 13 09:18:50.377917 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:18:50.377891 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3775975_f09b_4db7_8711_18775737dfd5.slice/crio-0d058a626bff7bc276a5b7d6420b7c63c10dc7f58f5ff0e465c19958d9db1188 WatchSource:0}: Error finding container 0d058a626bff7bc276a5b7d6420b7c63c10dc7f58f5ff0e465c19958d9db1188: Status 404 returned error can't find the container with id 0d058a626bff7bc276a5b7d6420b7c63c10dc7f58f5ff0e465c19958d9db1188 Jul 13 09:18:50.721279 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:50.721246 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:18:51.053777 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:51.053684 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f648df7b4-7n2bp" event={"ID":"d3775975-f09b-4db7-8711-18775737dfd5","Type":"ContainerStarted","Data":"ca1f91b396e573324906e610b7568e5b8d6feddb981d6f3c36d026d7295f7cb4"} Jul 13 09:18:51.053777 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:51.053738 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f648df7b4-7n2bp" event={"ID":"d3775975-f09b-4db7-8711-18775737dfd5","Type":"ContainerStarted","Data":"0d058a626bff7bc276a5b7d6420b7c63c10dc7f58f5ff0e465c19958d9db1188"} Jul 13 09:18:51.075754 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:51.075706 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6f648df7b4-7n2bp" podStartSLOduration=2.075667556 podStartE2EDuration="2.075667556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:18:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:18:51.073503558 +0000 UTC m=+614.953458825" watchObservedRunningTime="2026-07-13 09:18:51.075667556 +0000 UTC m=+614.955622901" Jul 13 09:18:53.377527 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:53.377493 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:18:55.556793 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:55.556763 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-9dsj7"] Jul 13 09:18:55.561429 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:55.561414 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9dsj7" Jul 13 09:18:55.567394 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:55.567369 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-x65d4\"" Jul 13 09:18:55.569784 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:55.569766 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 13 09:18:55.590984 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:55.590954 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-9dsj7"] Jul 13 09:18:55.597648 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:55.597625 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqrm2\" (UniqueName: \"kubernetes.io/projected/99c56ba6-b6d4-42f1-a1cf-6212e111544b-kube-api-access-cqrm2\") pod \"dns-operator-controller-manager-864dd78c4c-9dsj7\" (UID: \"99c56ba6-b6d4-42f1-a1cf-6212e111544b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9dsj7" Jul 13 09:18:55.698309 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:55.698283 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqrm2\" (UniqueName: \"kubernetes.io/projected/99c56ba6-b6d4-42f1-a1cf-6212e111544b-kube-api-access-cqrm2\") pod \"dns-operator-controller-manager-864dd78c4c-9dsj7\" (UID: \"99c56ba6-b6d4-42f1-a1cf-6212e111544b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9dsj7" Jul 13 09:18:55.707382 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:55.707361 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqrm2\" (UniqueName: \"kubernetes.io/projected/99c56ba6-b6d4-42f1-a1cf-6212e111544b-kube-api-access-cqrm2\") pod \"dns-operator-controller-manager-864dd78c4c-9dsj7\" (UID: \"99c56ba6-b6d4-42f1-a1cf-6212e111544b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9dsj7" Jul 13 09:18:55.871966 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:55.871942 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9dsj7" Jul 13 09:18:56.020736 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:56.020707 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-9dsj7"] Jul 13 09:18:56.022897 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:18:56.022872 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99c56ba6_b6d4_42f1_a1cf_6212e111544b.slice/crio-efa62699781d9d473fccade9ca5c78b86ce8515caa99f3a1de6327b594d2c14f WatchSource:0}: Error finding container efa62699781d9d473fccade9ca5c78b86ce8515caa99f3a1de6327b594d2c14f: Status 404 returned error can't find the container with id efa62699781d9d473fccade9ca5c78b86ce8515caa99f3a1de6327b594d2c14f Jul 13 09:18:56.080071 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:56.080045 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9dsj7" event={"ID":"99c56ba6-b6d4-42f1-a1cf-6212e111544b","Type":"ContainerStarted","Data":"efa62699781d9d473fccade9ca5c78b86ce8515caa99f3a1de6327b594d2c14f"} Jul 13 09:18:57.092667 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:57.092543 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:18:59.097509 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:59.097430 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9dsj7" event={"ID":"99c56ba6-b6d4-42f1-a1cf-6212e111544b","Type":"ContainerStarted","Data":"7efe4faaa2ddf7319a1dc7c94b1f9f76f8d070f31d3e50241e573da23f197bd6"} Jul 13 09:18:59.097853 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:59.097566 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9dsj7" Jul 13 09:18:59.123308 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:59.123266 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9dsj7" podStartSLOduration=2.092864566 podStartE2EDuration="4.123245695s" totalImagesPullingTime="2.030381129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:18:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:18:59.120747074 +0000 UTC m=+623.000702341" watchObservedRunningTime="2026-07-13 09:18:59.123245695 +0000 UTC m=+623.003200964" Jul 13 09:18:59.179552 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:59.179531 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6mkm5"] Jul 13 09:18:59.183139 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:59.183122 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6mkm5" Jul 13 09:18:59.185898 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:59.185881 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-mzgbd\"" Jul 13 09:18:59.197724 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:59.197705 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6mkm5"] Jul 13 09:18:59.225178 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:59.225156 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjt5b\" (UniqueName: \"kubernetes.io/projected/77a43e65-2f4e-4455-ae58-710658839fcc-kube-api-access-zjt5b\") pod \"limitador-operator-controller-manager-859d7b78bd-6mkm5\" (UID: \"77a43e65-2f4e-4455-ae58-710658839fcc\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6mkm5" Jul 13 09:18:59.326121 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:59.326100 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjt5b\" (UniqueName: \"kubernetes.io/projected/77a43e65-2f4e-4455-ae58-710658839fcc-kube-api-access-zjt5b\") pod \"limitador-operator-controller-manager-859d7b78bd-6mkm5\" (UID: \"77a43e65-2f4e-4455-ae58-710658839fcc\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6mkm5" Jul 13 09:18:59.340418 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:59.340389 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjt5b\" (UniqueName: \"kubernetes.io/projected/77a43e65-2f4e-4455-ae58-710658839fcc-kube-api-access-zjt5b\") pod \"limitador-operator-controller-manager-859d7b78bd-6mkm5\" (UID: \"77a43e65-2f4e-4455-ae58-710658839fcc\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6mkm5" Jul 13 09:18:59.493523 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:59.493466 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6mkm5" Jul 13 09:18:59.665786 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:18:59.665756 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6mkm5"] Jul 13 09:18:59.667125 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:18:59.667081 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77a43e65_2f4e_4455_ae58_710658839fcc.slice/crio-dad8786bdc05edfef47504c705a432d928e82c7a0c083a88f4f14b0d1f0de404 WatchSource:0}: Error finding container dad8786bdc05edfef47504c705a432d928e82c7a0c083a88f4f14b0d1f0de404: Status 404 returned error can't find the container with id dad8786bdc05edfef47504c705a432d928e82c7a0c083a88f4f14b0d1f0de404 Jul 13 09:19:00.097252 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:00.097198 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:19:00.103632 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:00.103605 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6mkm5" event={"ID":"77a43e65-2f4e-4455-ae58-710658839fcc","Type":"ContainerStarted","Data":"dad8786bdc05edfef47504c705a432d928e82c7a0c083a88f4f14b0d1f0de404"} Jul 13 09:19:00.233864 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:00.233834 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f648df7b4-7n2bp" Jul 13 09:19:00.234029 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:00.233903 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6f648df7b4-7n2bp" Jul 13 09:19:00.239110 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:00.239090 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6f648df7b4-7n2bp" Jul 13 09:19:01.113147 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:01.113114 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6f648df7b4-7n2bp" Jul 13 09:19:01.174065 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:01.174018 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-678945cf5c-7lrjp"] Jul 13 09:19:01.306353 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:01.306326 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:19:01.415044 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:01.414972 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:19:01.510488 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:01.510462 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:19:02.115632 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:02.115599 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6mkm5" event={"ID":"77a43e65-2f4e-4455-ae58-710658839fcc","Type":"ContainerStarted","Data":"ca8c336fb5f44c92ca3d374de00a8ea69e4c73dc9955052ffb1c09e4a44b2ef8"} Jul 13 09:19:02.143048 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:02.143009 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6mkm5" podStartSLOduration=0.802871932 podStartE2EDuration="3.142994594s" totalImagesPullingTime="2.340122662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:18:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:19:02.141871007 +0000 UTC m=+626.021826274" watchObservedRunningTime="2026-07-13 09:19:02.142994594 +0000 UTC m=+626.022949862" Jul 13 09:19:03.120735 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:03.120706 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6mkm5" Jul 13 09:19:03.192137 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:03.192106 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:19:04.136388 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:04.136357 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pl2q"] Jul 13 09:19:04.139885 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:04.139866 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pl2q" Jul 13 09:19:04.143122 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:04.143104 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-qzhlj\"" Jul 13 09:19:04.155157 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:04.155133 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pl2q"] Jul 13 09:19:04.165249 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:04.165225 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvbs9\" (UniqueName: \"kubernetes.io/projected/481f8dfe-e7cd-4888-b13b-ad135227ae71-kube-api-access-fvbs9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6pl2q\" (UID: \"481f8dfe-e7cd-4888-b13b-ad135227ae71\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pl2q" Jul 13 09:19:04.165353 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:04.165339 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/481f8dfe-e7cd-4888-b13b-ad135227ae71-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6pl2q\" (UID: \"481f8dfe-e7cd-4888-b13b-ad135227ae71\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pl2q" Jul 13 09:19:04.266454 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:04.266429 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/481f8dfe-e7cd-4888-b13b-ad135227ae71-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6pl2q\" (UID: \"481f8dfe-e7cd-4888-b13b-ad135227ae71\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pl2q" Jul 13 09:19:04.266557 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:04.266501 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvbs9\" (UniqueName: \"kubernetes.io/projected/481f8dfe-e7cd-4888-b13b-ad135227ae71-kube-api-access-fvbs9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6pl2q\" (UID: \"481f8dfe-e7cd-4888-b13b-ad135227ae71\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pl2q" Jul 13 09:19:04.266791 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:04.266774 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/481f8dfe-e7cd-4888-b13b-ad135227ae71-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6pl2q\" (UID: \"481f8dfe-e7cd-4888-b13b-ad135227ae71\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pl2q" Jul 13 09:19:04.278021 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:04.277996 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvbs9\" (UniqueName: \"kubernetes.io/projected/481f8dfe-e7cd-4888-b13b-ad135227ae71-kube-api-access-fvbs9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6pl2q\" (UID: \"481f8dfe-e7cd-4888-b13b-ad135227ae71\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pl2q" Jul 13 09:19:04.449502 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:04.449443 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pl2q" Jul 13 09:19:04.535822 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:04.535233 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:19:04.601337 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:04.601308 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pl2q"] Jul 13 09:19:04.605112 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:19:04.605085 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod481f8dfe_e7cd_4888_b13b_ad135227ae71.slice/crio-900cc46ee1f062b8242259b466dda3567bff4508566a28d30095d714b82087f2 WatchSource:0}: Error finding container 900cc46ee1f062b8242259b466dda3567bff4508566a28d30095d714b82087f2: Status 404 returned error can't find the container with id 900cc46ee1f062b8242259b466dda3567bff4508566a28d30095d714b82087f2 Jul 13 09:19:04.682602 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:04.682570 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:19:05.133339 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:05.133304 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pl2q" event={"ID":"481f8dfe-e7cd-4888-b13b-ad135227ae71","Type":"ContainerStarted","Data":"900cc46ee1f062b8242259b466dda3567bff4508566a28d30095d714b82087f2"} Jul 13 09:19:05.319455 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:05.319421 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:19:06.333572 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:06.333542 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:19:08.080670 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:08.080642 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-h956b"] Jul 13 09:19:08.084823 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:08.084800 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h956b" Jul 13 09:19:08.087616 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:08.087598 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-bkbl2\"" Jul 13 09:19:08.095544 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:08.095521 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-h956b"] Jul 13 09:19:08.200063 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:08.200037 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8m8g\" (UniqueName: \"kubernetes.io/projected/4da2f93d-c905-4cc7-a2ef-509625c2a4f0-kube-api-access-t8m8g\") pod \"authorino-operator-75cc8d5cb7-h956b\" (UID: \"4da2f93d-c905-4cc7-a2ef-509625c2a4f0\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-h956b" Jul 13 09:19:08.300896 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:08.300874 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8m8g\" (UniqueName: \"kubernetes.io/projected/4da2f93d-c905-4cc7-a2ef-509625c2a4f0-kube-api-access-t8m8g\") pod \"authorino-operator-75cc8d5cb7-h956b\" (UID: \"4da2f93d-c905-4cc7-a2ef-509625c2a4f0\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-h956b" Jul 13 09:19:08.310913 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:08.310887 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8m8g\" (UniqueName: \"kubernetes.io/projected/4da2f93d-c905-4cc7-a2ef-509625c2a4f0-kube-api-access-t8m8g\") pod \"authorino-operator-75cc8d5cb7-h956b\" (UID: \"4da2f93d-c905-4cc7-a2ef-509625c2a4f0\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-h956b" Jul 13 09:19:08.398350 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:08.398288 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h956b" Jul 13 09:19:08.566335 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:08.566282 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-h956b"] Jul 13 09:19:08.572714 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:19:08.572684 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4da2f93d_c905_4cc7_a2ef_509625c2a4f0.slice/crio-777b7e67733cc28977891df09496399f8902554accf8e6c36b443af10719a2ae WatchSource:0}: Error finding container 777b7e67733cc28977891df09496399f8902554accf8e6c36b443af10719a2ae: Status 404 returned error can't find the container with id 777b7e67733cc28977891df09496399f8902554accf8e6c36b443af10719a2ae Jul 13 09:19:08.906087 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:08.906058 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:19:09.020252 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:09.020226 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:19:09.159031 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:09.158949 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h956b" event={"ID":"4da2f93d-c905-4cc7-a2ef-509625c2a4f0","Type":"ContainerStarted","Data":"777b7e67733cc28977891df09496399f8902554accf8e6c36b443af10719a2ae"} Jul 13 09:19:10.106807 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:10.106771 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9dsj7" Jul 13 09:19:10.286068 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:10.286024 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:19:10.387446 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:10.387361 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:19:12.177199 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:12.177150 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h956b" event={"ID":"4da2f93d-c905-4cc7-a2ef-509625c2a4f0","Type":"ContainerStarted","Data":"be19c59837143187a540d73fa35c99c27eb2d278172e4165fff21077f4652873"} Jul 13 09:19:12.177648 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:12.177249 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h956b" Jul 13 09:19:12.178909 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:12.178886 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pl2q" event={"ID":"481f8dfe-e7cd-4888-b13b-ad135227ae71","Type":"ContainerStarted","Data":"624cad21de76f780a202902388920e9167c0a28217804fcbf3d71638fe7a0259"} Jul 13 09:19:12.178980 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:12.178933 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pl2q" Jul 13 09:19:12.201025 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:12.200980 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h956b" podStartSLOduration=1.235650283 podStartE2EDuration="4.200965805s" totalImagesPullingTime="2.965315522s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:19:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:19:12.198690274 +0000 UTC m=+636.078645541" watchObservedRunningTime="2026-07-13 09:19:12.200965805 +0000 UTC m=+636.080921072" Jul 13 09:19:12.223425 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:12.223382 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pl2q" podStartSLOduration=1.325944135 podStartE2EDuration="8.223368088s" totalImagesPullingTime="6.897423953s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:19:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:19:12.22139707 +0000 UTC m=+636.101352342" watchObservedRunningTime="2026-07-13 09:19:12.223368088 +0000 UTC m=+636.103323354" Jul 13 09:19:14.128315 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:14.128277 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6mkm5" Jul 13 09:19:14.156375 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:14.156345 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:19:14.293193 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:14.293160 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:19:14.580502 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:14.580469 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-5jjbn"] Jul 13 09:19:14.584662 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:14.584638 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5jjbn" Jul 13 09:19:14.589329 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:14.589309 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 13 09:19:14.589452 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:14.589403 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-mr8xt\"" Jul 13 09:19:14.589611 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:14.589590 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 13 09:19:14.590401 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:14.590364 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:19:14.598530 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:14.598505 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-5jjbn"] Jul 13 09:19:14.654542 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:14.654508 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/12a4f316-ca65-40dd-a193-106f0fc79e01-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-5jjbn\" (UID: \"12a4f316-ca65-40dd-a193-106f0fc79e01\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5jjbn" Jul 13 09:19:14.654668 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:14.654604 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/12a4f316-ca65-40dd-a193-106f0fc79e01-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-5jjbn\" (UID: \"12a4f316-ca65-40dd-a193-106f0fc79e01\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5jjbn" Jul 13 09:19:14.654710 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:14.654665 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x485\" (UniqueName: \"kubernetes.io/projected/12a4f316-ca65-40dd-a193-106f0fc79e01-kube-api-access-4x485\") pod \"kuadrant-console-plugin-66b5774985-5jjbn\" (UID: \"12a4f316-ca65-40dd-a193-106f0fc79e01\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5jjbn" Jul 13 09:19:14.755977 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:14.755945 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/12a4f316-ca65-40dd-a193-106f0fc79e01-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-5jjbn\" (UID: \"12a4f316-ca65-40dd-a193-106f0fc79e01\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5jjbn" Jul 13 09:19:14.756128 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:14.755990 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x485\" (UniqueName: \"kubernetes.io/projected/12a4f316-ca65-40dd-a193-106f0fc79e01-kube-api-access-4x485\") pod \"kuadrant-console-plugin-66b5774985-5jjbn\" (UID: \"12a4f316-ca65-40dd-a193-106f0fc79e01\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5jjbn" Jul 13 09:19:14.756128 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:14.756030 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/12a4f316-ca65-40dd-a193-106f0fc79e01-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-5jjbn\" (UID: \"12a4f316-ca65-40dd-a193-106f0fc79e01\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5jjbn" Jul 13 09:19:14.756611 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:14.756590 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/12a4f316-ca65-40dd-a193-106f0fc79e01-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-5jjbn\" (UID: \"12a4f316-ca65-40dd-a193-106f0fc79e01\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5jjbn" Jul 13 09:19:14.758387 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:14.758368 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/12a4f316-ca65-40dd-a193-106f0fc79e01-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-5jjbn\" (UID: \"12a4f316-ca65-40dd-a193-106f0fc79e01\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5jjbn" Jul 13 09:19:14.767589 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:14.767568 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:19:14.768718 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:14.768698 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x485\" (UniqueName: \"kubernetes.io/projected/12a4f316-ca65-40dd-a193-106f0fc79e01-kube-api-access-4x485\") pod \"kuadrant-console-plugin-66b5774985-5jjbn\" (UID: \"12a4f316-ca65-40dd-a193-106f0fc79e01\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5jjbn" Jul 13 09:19:14.894099 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:14.894035 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5jjbn" Jul 13 09:19:15.039718 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:15.039691 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-5jjbn"] Jul 13 09:19:15.041302 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:19:15.041278 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod12a4f316_ca65_40dd_a193_106f0fc79e01.slice/crio-af373a814d0d05ffebd71f3cecc647270972365ee784171875b3b3c93439245c WatchSource:0}: Error finding container af373a814d0d05ffebd71f3cecc647270972365ee784171875b3b3c93439245c: Status 404 returned error can't find the container with id af373a814d0d05ffebd71f3cecc647270972365ee784171875b3b3c93439245c Jul 13 09:19:15.196522 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:15.196464 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5jjbn" event={"ID":"12a4f316-ca65-40dd-a193-106f0fc79e01","Type":"ContainerStarted","Data":"af373a814d0d05ffebd71f3cecc647270972365ee784171875b3b3c93439245c"} Jul 13 09:19:17.804943 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:17.804911 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:19:17.921335 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:17.921301 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:19:23.186683 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:23.186638 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h956b" Jul 13 09:19:23.187151 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:23.186721 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pl2q" Jul 13 09:19:25.246991 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:25.246957 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pl2q"] Jul 13 09:19:25.247484 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:25.247277 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pl2q" podUID="481f8dfe-e7cd-4888-b13b-ad135227ae71" containerName="manager" containerID="cri-o://624cad21de76f780a202902388920e9167c0a28217804fcbf3d71638fe7a0259" gracePeriod=2 Jul 13 09:19:25.259320 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:25.259290 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pl2q"] Jul 13 09:19:25.266338 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:19:25.266276 2552 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"481f8dfe-e7cd-4888-b13b-ad135227ae71\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-6pl2q\": pods \"kuadrant-operator-controller-manager-8684c9ff49-6pl2q\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pl2q" Jul 13 09:19:25.278855 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:25.278825 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6mkm5"] Jul 13 09:19:25.279346 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:25.279064 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6mkm5" podUID="77a43e65-2f4e-4455-ae58-710658839fcc" containerName="manager" containerID="cri-o://ca8c336fb5f44c92ca3d374de00a8ea69e4c73dc9955052ffb1c09e4a44b2ef8" gracePeriod=2 Jul 13 09:19:25.289819 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:25.289467 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6mkm5"] Jul 13 09:19:25.294992 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:19:25.294935 2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6pl2q\" is forbidden: User \"system:node:ip-10-0-129-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-99.ec2.internal' and this object" podUID="481f8dfe-e7cd-4888-b13b-ad135227ae71" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pl2q" Jul 13 09:19:25.318318 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:25.318293 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cth6m"] Jul 13 09:19:25.318782 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:25.318765 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77a43e65-2f4e-4455-ae58-710658839fcc" containerName="manager" Jul 13 09:19:25.318891 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:25.318784 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="77a43e65-2f4e-4455-ae58-710658839fcc" containerName="manager" Jul 13 09:19:25.318891 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:25.318814 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="481f8dfe-e7cd-4888-b13b-ad135227ae71" containerName="manager" Jul 13 09:19:25.318891 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:25.318823 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="481f8dfe-e7cd-4888-b13b-ad135227ae71" containerName="manager" Jul 13 09:19:25.319135 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:25.318932 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="77a43e65-2f4e-4455-ae58-710658839fcc" containerName="manager" Jul 13 09:19:25.319135 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:25.318944 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="481f8dfe-e7cd-4888-b13b-ad135227ae71" containerName="manager" Jul 13 09:19:25.323824 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:25.323800 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cth6m" Jul 13 09:19:25.327049 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:19:25.327020 2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6pl2q\" is forbidden: User \"system:node:ip-10-0-129-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-99.ec2.internal' and this object" podUID="481f8dfe-e7cd-4888-b13b-ad135227ae71" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pl2q" Jul 13 09:19:25.343849 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:25.343826 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cth6m"] Jul 13 09:19:25.363542 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:19:25.363508 2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6mkm5\" is forbidden: User \"system:node:ip-10-0-129-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-99.ec2.internal' and this object" podUID="77a43e65-2f4e-4455-ae58-710658839fcc" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6mkm5" Jul 13 09:19:25.456168 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:25.456133 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99tnn\" (UniqueName: \"kubernetes.io/projected/ba153729-3fe0-4ca7-a7f5-deec6c3586a0-kube-api-access-99tnn\") pod \"limitador-operator-controller-manager-859d7b78bd-cth6m\" (UID: \"ba153729-3fe0-4ca7-a7f5-deec6c3586a0\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cth6m" Jul 13 09:19:25.557323 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:25.557243 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99tnn\" (UniqueName: \"kubernetes.io/projected/ba153729-3fe0-4ca7-a7f5-deec6c3586a0-kube-api-access-99tnn\") pod \"limitador-operator-controller-manager-859d7b78bd-cth6m\" (UID: \"ba153729-3fe0-4ca7-a7f5-deec6c3586a0\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cth6m" Jul 13 09:19:25.573254 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:25.573193 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99tnn\" (UniqueName: \"kubernetes.io/projected/ba153729-3fe0-4ca7-a7f5-deec6c3586a0-kube-api-access-99tnn\") pod \"limitador-operator-controller-manager-859d7b78bd-cth6m\" (UID: \"ba153729-3fe0-4ca7-a7f5-deec6c3586a0\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cth6m" Jul 13 09:19:25.669312 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:25.669272 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cth6m" Jul 13 09:19:25.714321 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:25.714290 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:19:25.810923 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:25.810850 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:19:26.197685 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:26.197642 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-678945cf5c-7lrjp" podUID="af6e4f34-3bd7-46f2-95dc-f99cc52699f3" containerName="console" containerID="cri-o://2ac03e5941a052876ee137f1bae559e0150ac08b7e2ef089ab62ce3c614319f6" gracePeriod=15 Jul 13 09:19:26.666513 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:19:26.666481 2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6mkm5\" is forbidden: User \"system:node:ip-10-0-129-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-99.ec2.internal' and this object" podUID="77a43e65-2f4e-4455-ae58-710658839fcc" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6mkm5" Jul 13 09:19:26.668573 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:19:26.668545 2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6pl2q\" is forbidden: User \"system:node:ip-10-0-129-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-99.ec2.internal' and this object" podUID="481f8dfe-e7cd-4888-b13b-ad135227ae71" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pl2q" Jul 13 09:19:26.929045 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:26.928958 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jwvn"] Jul 13 09:19:26.955390 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:26.955354 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jwvn"] Jul 13 09:19:26.955563 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:26.955413 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jwvn" Jul 13 09:19:27.069994 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:27.069961 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmpqn\" (UniqueName: \"kubernetes.io/projected/0f0046b1-ae5b-4205-97a0-23607bc45064-kube-api-access-vmpqn\") pod \"kuadrant-operator-controller-manager-86878c769f-5jwvn\" (UID: \"0f0046b1-ae5b-4205-97a0-23607bc45064\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jwvn" Jul 13 09:19:27.070170 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:27.070036 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f0046b1-ae5b-4205-97a0-23607bc45064-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-5jwvn\" (UID: \"0f0046b1-ae5b-4205-97a0-23607bc45064\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jwvn" Jul 13 09:19:27.170934 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:27.170903 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f0046b1-ae5b-4205-97a0-23607bc45064-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-5jwvn\" (UID: \"0f0046b1-ae5b-4205-97a0-23607bc45064\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jwvn" Jul 13 09:19:27.171107 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:27.170983 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmpqn\" (UniqueName: \"kubernetes.io/projected/0f0046b1-ae5b-4205-97a0-23607bc45064-kube-api-access-vmpqn\") pod \"kuadrant-operator-controller-manager-86878c769f-5jwvn\" (UID: \"0f0046b1-ae5b-4205-97a0-23607bc45064\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jwvn" Jul 13 09:19:27.171360 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:27.171334 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f0046b1-ae5b-4205-97a0-23607bc45064-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-5jwvn\" (UID: \"0f0046b1-ae5b-4205-97a0-23607bc45064\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jwvn" Jul 13 09:19:27.184338 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:27.184264 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmpqn\" (UniqueName: \"kubernetes.io/projected/0f0046b1-ae5b-4205-97a0-23607bc45064-kube-api-access-vmpqn\") pod \"kuadrant-operator-controller-manager-86878c769f-5jwvn\" (UID: \"0f0046b1-ae5b-4205-97a0-23607bc45064\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jwvn" Jul 13 09:19:27.267905 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:27.267877 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jwvn" Jul 13 09:19:28.370494 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:28.370451 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:19:28.492535 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:28.492502 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:19:29.901052 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:29.901016 2552 patch_prober.go:28] interesting pod/console-678945cf5c-7lrjp container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.133.0.21:8443/health\": dial tcp 10.133.0.21:8443: connect: connection refused" start-of-body= Jul 13 09:19:29.901445 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:29.901068 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/console-678945cf5c-7lrjp" podUID="af6e4f34-3bd7-46f2-95dc-f99cc52699f3" containerName="console" probeResult="failure" output="Get \"https://10.133.0.21:8443/health\": dial tcp 10.133.0.21:8443: connect: connection refused" Jul 13 09:19:31.184670 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:31.184617 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:19:31.287463 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:31.287424 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:19:34.766118 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:34.766079 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:19:34.866305 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:34.866257 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:19:37.097584 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.097556 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-678945cf5c-7lrjp" Jul 13 09:19:37.101072 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.101051 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6mkm5" Jul 13 09:19:37.110225 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.110188 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pl2q" Jul 13 09:19:37.141664 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:19:37.141629 2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6mkm5\" is forbidden: User \"system:node:ip-10-0-129-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-99.ec2.internal' and this object" podUID="77a43e65-2f4e-4455-ae58-710658839fcc" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6mkm5" Jul 13 09:19:37.144365 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:19:37.144342 2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6pl2q\" is forbidden: User \"system:node:ip-10-0-129-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-99.ec2.internal' and this object" podUID="481f8dfe-e7cd-4888-b13b-ad135227ae71" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pl2q" Jul 13 09:19:37.170805 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.170742 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-service-ca\") pod \"af6e4f34-3bd7-46f2-95dc-f99cc52699f3\" (UID: \"af6e4f34-3bd7-46f2-95dc-f99cc52699f3\") " Jul 13 09:19:37.170805 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.170786 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/481f8dfe-e7cd-4888-b13b-ad135227ae71-extensions-socket-volume\") pod \"481f8dfe-e7cd-4888-b13b-ad135227ae71\" (UID: \"481f8dfe-e7cd-4888-b13b-ad135227ae71\") " Jul 13 09:19:37.170953 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.170808 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-console-config\") pod \"af6e4f34-3bd7-46f2-95dc-f99cc52699f3\" (UID: \"af6e4f34-3bd7-46f2-95dc-f99cc52699f3\") " Jul 13 09:19:37.170953 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.170858 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-console-oauth-config\") pod \"af6e4f34-3bd7-46f2-95dc-f99cc52699f3\" (UID: \"af6e4f34-3bd7-46f2-95dc-f99cc52699f3\") " Jul 13 09:19:37.170953 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.170875 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-oauth-serving-cert\") pod \"af6e4f34-3bd7-46f2-95dc-f99cc52699f3\" (UID: \"af6e4f34-3bd7-46f2-95dc-f99cc52699f3\") " Jul 13 09:19:37.170953 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.170901 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-trusted-ca-bundle\") pod \"af6e4f34-3bd7-46f2-95dc-f99cc52699f3\" (UID: \"af6e4f34-3bd7-46f2-95dc-f99cc52699f3\") " Jul 13 09:19:37.170953 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.170928 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-console-serving-cert\") pod \"af6e4f34-3bd7-46f2-95dc-f99cc52699f3\" (UID: \"af6e4f34-3bd7-46f2-95dc-f99cc52699f3\") " Jul 13 09:19:37.171186 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.170954 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zjt5b\" (UniqueName: \"kubernetes.io/projected/77a43e65-2f4e-4455-ae58-710658839fcc-kube-api-access-zjt5b\") pod \"77a43e65-2f4e-4455-ae58-710658839fcc\" (UID: \"77a43e65-2f4e-4455-ae58-710658839fcc\") " Jul 13 09:19:37.171186 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.171014 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-99lmc\" (UniqueName: \"kubernetes.io/projected/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-kube-api-access-99lmc\") pod \"af6e4f34-3bd7-46f2-95dc-f99cc52699f3\" (UID: \"af6e4f34-3bd7-46f2-95dc-f99cc52699f3\") " Jul 13 09:19:37.171186 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.171054 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fvbs9\" (UniqueName: \"kubernetes.io/projected/481f8dfe-e7cd-4888-b13b-ad135227ae71-kube-api-access-fvbs9\") pod \"481f8dfe-e7cd-4888-b13b-ad135227ae71\" (UID: \"481f8dfe-e7cd-4888-b13b-ad135227ae71\") " Jul 13 09:19:37.171336 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.171157 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-service-ca" (OuterVolumeSpecName: "service-ca") pod "af6e4f34-3bd7-46f2-95dc-f99cc52699f3" (UID: "af6e4f34-3bd7-46f2-95dc-f99cc52699f3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 09:19:37.171402 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.171367 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-console-config" (OuterVolumeSpecName: "console-config") pod "af6e4f34-3bd7-46f2-95dc-f99cc52699f3" (UID: "af6e4f34-3bd7-46f2-95dc-f99cc52699f3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 09:19:37.171402 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.171376 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/481f8dfe-e7cd-4888-b13b-ad135227ae71-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "481f8dfe-e7cd-4888-b13b-ad135227ae71" (UID: "481f8dfe-e7cd-4888-b13b-ad135227ae71"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:19:37.171402 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.171378 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "af6e4f34-3bd7-46f2-95dc-f99cc52699f3" (UID: "af6e4f34-3bd7-46f2-95dc-f99cc52699f3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 09:19:37.171747 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.171711 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "af6e4f34-3bd7-46f2-95dc-f99cc52699f3" (UID: "af6e4f34-3bd7-46f2-95dc-f99cc52699f3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 09:19:37.173286 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.173256 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77a43e65-2f4e-4455-ae58-710658839fcc-kube-api-access-zjt5b" (OuterVolumeSpecName: "kube-api-access-zjt5b") pod "77a43e65-2f4e-4455-ae58-710658839fcc" (UID: "77a43e65-2f4e-4455-ae58-710658839fcc"). InnerVolumeSpecName "kube-api-access-zjt5b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:19:37.173603 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.173586 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "af6e4f34-3bd7-46f2-95dc-f99cc52699f3" (UID: "af6e4f34-3bd7-46f2-95dc-f99cc52699f3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 09:19:37.173670 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.173625 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/481f8dfe-e7cd-4888-b13b-ad135227ae71-kube-api-access-fvbs9" (OuterVolumeSpecName: "kube-api-access-fvbs9") pod "481f8dfe-e7cd-4888-b13b-ad135227ae71" (UID: "481f8dfe-e7cd-4888-b13b-ad135227ae71"). InnerVolumeSpecName "kube-api-access-fvbs9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:19:37.173716 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.173696 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "af6e4f34-3bd7-46f2-95dc-f99cc52699f3" (UID: "af6e4f34-3bd7-46f2-95dc-f99cc52699f3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 09:19:37.173753 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.173724 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-kube-api-access-99lmc" (OuterVolumeSpecName: "kube-api-access-99lmc") pod "af6e4f34-3bd7-46f2-95dc-f99cc52699f3" (UID: "af6e4f34-3bd7-46f2-95dc-f99cc52699f3"). InnerVolumeSpecName "kube-api-access-99lmc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:19:37.272293 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.272270 2552 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-trusted-ca-bundle\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:19:37.272293 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.272291 2552 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-console-serving-cert\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:19:37.272427 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.272301 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjt5b\" (UniqueName: \"kubernetes.io/projected/77a43e65-2f4e-4455-ae58-710658839fcc-kube-api-access-zjt5b\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:19:37.272427 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.272310 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99lmc\" (UniqueName: \"kubernetes.io/projected/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-kube-api-access-99lmc\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:19:37.272427 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.272321 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fvbs9\" (UniqueName: \"kubernetes.io/projected/481f8dfe-e7cd-4888-b13b-ad135227ae71-kube-api-access-fvbs9\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:19:37.272427 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.272329 2552 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-service-ca\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:19:37.272427 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.272337 2552 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/481f8dfe-e7cd-4888-b13b-ad135227ae71-extensions-socket-volume\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:19:37.272427 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.272345 2552 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-console-config\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:19:37.272427 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.272353 2552 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-console-oauth-config\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:19:37.272427 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.272361 2552 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af6e4f34-3bd7-46f2-95dc-f99cc52699f3-oauth-serving-cert\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:19:37.333838 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.333814 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cth6m"] Jul 13 09:19:37.335541 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:19:37.335519 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba153729_3fe0_4ca7_a7f5_deec6c3586a0.slice/crio-0d2d87cfa8834daf8e0c2ec541d705cc158199cdf4c3359521ece7dbceb23ea2 WatchSource:0}: Error finding container 0d2d87cfa8834daf8e0c2ec541d705cc158199cdf4c3359521ece7dbceb23ea2: Status 404 returned error can't find the container with id 0d2d87cfa8834daf8e0c2ec541d705cc158199cdf4c3359521ece7dbceb23ea2 Jul 13 09:19:37.345205 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.345172 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cth6m" event={"ID":"ba153729-3fe0-4ca7-a7f5-deec6c3586a0","Type":"ContainerStarted","Data":"0d2d87cfa8834daf8e0c2ec541d705cc158199cdf4c3359521ece7dbceb23ea2"} Jul 13 09:19:37.346428 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.346406 2552 generic.go:356] "Generic (PLEG): container finished" podID="af6e4f34-3bd7-46f2-95dc-f99cc52699f3" containerID="2ac03e5941a052876ee137f1bae559e0150ac08b7e2ef089ab62ce3c614319f6" exitCode=0 Jul 13 09:19:37.346523 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.346457 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-678945cf5c-7lrjp" Jul 13 09:19:37.346523 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.346492 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-678945cf5c-7lrjp" event={"ID":"af6e4f34-3bd7-46f2-95dc-f99cc52699f3","Type":"ContainerDied","Data":"2ac03e5941a052876ee137f1bae559e0150ac08b7e2ef089ab62ce3c614319f6"} Jul 13 09:19:37.346644 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.346548 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-678945cf5c-7lrjp" event={"ID":"af6e4f34-3bd7-46f2-95dc-f99cc52699f3","Type":"ContainerDied","Data":"42683b82c92e70f0eeeaeab33eb7622e897de793195f3a79bc3f8e574be03359"} Jul 13 09:19:37.346644 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.346580 2552 scope.go:117] "RemoveContainer" containerID="2ac03e5941a052876ee137f1bae559e0150ac08b7e2ef089ab62ce3c614319f6" Jul 13 09:19:37.347729 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.347708 2552 generic.go:356] "Generic (PLEG): container finished" podID="77a43e65-2f4e-4455-ae58-710658839fcc" containerID="ca8c336fb5f44c92ca3d374de00a8ea69e4c73dc9955052ffb1c09e4a44b2ef8" exitCode=0 Jul 13 09:19:37.347796 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.347750 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6mkm5" Jul 13 09:19:37.349484 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.349344 2552 generic.go:356] "Generic (PLEG): container finished" podID="481f8dfe-e7cd-4888-b13b-ad135227ae71" containerID="624cad21de76f780a202902388920e9167c0a28217804fcbf3d71638fe7a0259" exitCode=0 Jul 13 09:19:37.349484 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.349404 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pl2q" Jul 13 09:19:37.349484 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:19:37.349456 2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6mkm5\" is forbidden: User \"system:node:ip-10-0-129-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-99.ec2.internal' and this object" podUID="77a43e65-2f4e-4455-ae58-710658839fcc" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6mkm5" Jul 13 09:19:37.355470 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:19:37.355392 2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6pl2q\" is forbidden: User \"system:node:ip-10-0-129-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-99.ec2.internal' and this object" podUID="481f8dfe-e7cd-4888-b13b-ad135227ae71" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pl2q" Jul 13 09:19:37.355993 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.355951 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5jjbn" event={"ID":"12a4f316-ca65-40dd-a193-106f0fc79e01","Type":"ContainerStarted","Data":"8e4eafb235702beab6587a5e49593e2ead8664417c1ba30bb0a69698a6aca9a1"} Jul 13 09:19:37.361753 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.361732 2552 scope.go:117] "RemoveContainer" containerID="2ac03e5941a052876ee137f1bae559e0150ac08b7e2ef089ab62ce3c614319f6" Jul 13 09:19:37.362006 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:19:37.361989 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ac03e5941a052876ee137f1bae559e0150ac08b7e2ef089ab62ce3c614319f6\": container with ID starting with 2ac03e5941a052876ee137f1bae559e0150ac08b7e2ef089ab62ce3c614319f6 not found: ID does not exist" containerID="2ac03e5941a052876ee137f1bae559e0150ac08b7e2ef089ab62ce3c614319f6" Jul 13 09:19:37.362081 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.362011 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ac03e5941a052876ee137f1bae559e0150ac08b7e2ef089ab62ce3c614319f6"} err="failed to get container status \"2ac03e5941a052876ee137f1bae559e0150ac08b7e2ef089ab62ce3c614319f6\": rpc error: code = NotFound desc = could not find container \"2ac03e5941a052876ee137f1bae559e0150ac08b7e2ef089ab62ce3c614319f6\": container with ID starting with 2ac03e5941a052876ee137f1bae559e0150ac08b7e2ef089ab62ce3c614319f6 not found: ID does not exist" Jul 13 09:19:37.362081 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.362028 2552 scope.go:117] "RemoveContainer" containerID="ca8c336fb5f44c92ca3d374de00a8ea69e4c73dc9955052ffb1c09e4a44b2ef8" Jul 13 09:19:37.368868 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:19:37.368845 2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6mkm5\" is forbidden: User \"system:node:ip-10-0-129-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-99.ec2.internal' and this object" podUID="77a43e65-2f4e-4455-ae58-710658839fcc" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6mkm5" Jul 13 09:19:37.376611 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:19:37.376274 2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6pl2q\" is forbidden: User \"system:node:ip-10-0-129-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-99.ec2.internal' and this object" podUID="481f8dfe-e7cd-4888-b13b-ad135227ae71" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pl2q" Jul 13 09:19:37.379239 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.378899 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jwvn"] Jul 13 09:19:37.381778 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.381753 2552 scope.go:117] "RemoveContainer" containerID="ca8c336fb5f44c92ca3d374de00a8ea69e4c73dc9955052ffb1c09e4a44b2ef8" Jul 13 09:19:37.382055 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:19:37.382034 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca8c336fb5f44c92ca3d374de00a8ea69e4c73dc9955052ffb1c09e4a44b2ef8\": container with ID starting with ca8c336fb5f44c92ca3d374de00a8ea69e4c73dc9955052ffb1c09e4a44b2ef8 not found: ID does not exist" containerID="ca8c336fb5f44c92ca3d374de00a8ea69e4c73dc9955052ffb1c09e4a44b2ef8" Jul 13 09:19:37.382134 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.382063 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca8c336fb5f44c92ca3d374de00a8ea69e4c73dc9955052ffb1c09e4a44b2ef8"} err="failed to get container status \"ca8c336fb5f44c92ca3d374de00a8ea69e4c73dc9955052ffb1c09e4a44b2ef8\": rpc error: code = NotFound desc = could not find container \"ca8c336fb5f44c92ca3d374de00a8ea69e4c73dc9955052ffb1c09e4a44b2ef8\": container with ID starting with ca8c336fb5f44c92ca3d374de00a8ea69e4c73dc9955052ffb1c09e4a44b2ef8 not found: ID does not exist" Jul 13 09:19:37.382134 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.382086 2552 scope.go:117] "RemoveContainer" containerID="624cad21de76f780a202902388920e9167c0a28217804fcbf3d71638fe7a0259" Jul 13 09:19:37.393863 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:19:37.393833 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f0046b1_ae5b_4205_97a0_23607bc45064.slice/crio-4ea464b2c528af6bdc7a182b80d2e429289ffb224f5252b71795853c8d4d5e37 WatchSource:0}: Error finding container 4ea464b2c528af6bdc7a182b80d2e429289ffb224f5252b71795853c8d4d5e37: Status 404 returned error can't find the container with id 4ea464b2c528af6bdc7a182b80d2e429289ffb224f5252b71795853c8d4d5e37 Jul 13 09:19:37.397792 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.397772 2552 scope.go:117] "RemoveContainer" containerID="624cad21de76f780a202902388920e9167c0a28217804fcbf3d71638fe7a0259" Jul 13 09:19:37.398121 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:19:37.398095 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"624cad21de76f780a202902388920e9167c0a28217804fcbf3d71638fe7a0259\": container with ID starting with 624cad21de76f780a202902388920e9167c0a28217804fcbf3d71638fe7a0259 not found: ID does not exist" containerID="624cad21de76f780a202902388920e9167c0a28217804fcbf3d71638fe7a0259" Jul 13 09:19:37.398337 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.398305 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"624cad21de76f780a202902388920e9167c0a28217804fcbf3d71638fe7a0259"} err="failed to get container status \"624cad21de76f780a202902388920e9167c0a28217804fcbf3d71638fe7a0259\": rpc error: code = NotFound desc = could not find container \"624cad21de76f780a202902388920e9167c0a28217804fcbf3d71638fe7a0259\": container with ID starting with 624cad21de76f780a202902388920e9167c0a28217804fcbf3d71638fe7a0259 not found: ID does not exist" Jul 13 09:19:37.445103 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.445061 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5jjbn" podStartSLOduration=1.491955397 podStartE2EDuration="23.445045107s" totalImagesPullingTime="21.95308971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:19:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:19:37.44277963 +0000 UTC m=+661.322734898" watchObservedRunningTime="2026-07-13 09:19:37.445045107 +0000 UTC m=+661.325000405" Jul 13 09:19:37.445250 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:19:37.445079 2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6mkm5\" is forbidden: User \"system:node:ip-10-0-129-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-99.ec2.internal' and this object" podUID="77a43e65-2f4e-4455-ae58-710658839fcc" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6mkm5" Jul 13 09:19:37.447439 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:19:37.447413 2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6pl2q\" is forbidden: User \"system:node:ip-10-0-129-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-99.ec2.internal' and this object" podUID="481f8dfe-e7cd-4888-b13b-ad135227ae71" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pl2q" Jul 13 09:19:37.474933 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.474902 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-678945cf5c-7lrjp"] Jul 13 09:19:37.481428 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:37.481400 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-678945cf5c-7lrjp"] Jul 13 09:19:38.365431 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:38.365402 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cth6m" event={"ID":"ba153729-3fe0-4ca7-a7f5-deec6c3586a0","Type":"ContainerStarted","Data":"18b3913b86f1b0a7d2acf7e1b6bcdeb04b75045d28a4bdd28b33433ba0edd341"} Jul 13 09:19:38.365866 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:38.365527 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cth6m" Jul 13 09:19:38.369061 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:38.369032 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jwvn" event={"ID":"0f0046b1-ae5b-4205-97a0-23607bc45064","Type":"ContainerStarted","Data":"820988123c0554b54f01e53adeb4710505f0d3866d1cefb5a5664abf759013ca"} Jul 13 09:19:38.369189 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:38.369087 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jwvn" event={"ID":"0f0046b1-ae5b-4205-97a0-23607bc45064","Type":"ContainerStarted","Data":"4ea464b2c528af6bdc7a182b80d2e429289ffb224f5252b71795853c8d4d5e37"} Jul 13 09:19:38.386780 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:38.386727 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cth6m" podStartSLOduration=13.386712289 podStartE2EDuration="13.386712289s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:19:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:19:38.385967805 +0000 UTC m=+662.265923073" watchObservedRunningTime="2026-07-13 09:19:38.386712289 +0000 UTC m=+662.266667559" Jul 13 09:19:38.388286 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:19:38.388250 2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6mkm5\" is forbidden: User \"system:node:ip-10-0-129-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-99.ec2.internal' and this object" podUID="77a43e65-2f4e-4455-ae58-710658839fcc" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6mkm5" Jul 13 09:19:38.390406 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:19:38.390384 2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6pl2q\" is forbidden: User \"system:node:ip-10-0-129-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-99.ec2.internal' and this object" podUID="481f8dfe-e7cd-4888-b13b-ad135227ae71" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pl2q" Jul 13 09:19:38.392491 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:19:38.392470 2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6mkm5\" is forbidden: User \"system:node:ip-10-0-129-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-99.ec2.internal' and this object" podUID="77a43e65-2f4e-4455-ae58-710658839fcc" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6mkm5" Jul 13 09:19:38.394404 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:19:38.394380 2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6pl2q\" is forbidden: User \"system:node:ip-10-0-129-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-99.ec2.internal' and this object" podUID="481f8dfe-e7cd-4888-b13b-ad135227ae71" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pl2q" Jul 13 09:19:38.428481 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:38.428441 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jwvn" podStartSLOduration=12.428428984 podStartE2EDuration="12.428428984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:19:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:19:38.427902636 +0000 UTC m=+662.307857904" watchObservedRunningTime="2026-07-13 09:19:38.428428984 +0000 UTC m=+662.308384253" Jul 13 09:19:38.667377 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:38.667311 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="481f8dfe-e7cd-4888-b13b-ad135227ae71" path="/var/lib/kubelet/pods/481f8dfe-e7cd-4888-b13b-ad135227ae71/volumes" Jul 13 09:19:38.667628 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:38.667616 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77a43e65-2f4e-4455-ae58-710658839fcc" path="/var/lib/kubelet/pods/77a43e65-2f4e-4455-ae58-710658839fcc/volumes" Jul 13 09:19:38.667951 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:38.667940 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af6e4f34-3bd7-46f2-95dc-f99cc52699f3" path="/var/lib/kubelet/pods/af6e4f34-3bd7-46f2-95dc-f99cc52699f3/volumes" Jul 13 09:19:39.374759 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:39.374725 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jwvn" Jul 13 09:19:43.658062 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:43.658026 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:19:43.730453 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:43.730423 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:19:46.227661 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:46.227626 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:19:46.343632 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:46.343597 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:19:49.377361 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:49.377328 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cth6m" Jul 13 09:19:49.418716 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:49.418679 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:19:49.518363 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:49.518331 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:19:50.383673 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:50.383637 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jwvn" Jul 13 09:19:50.801749 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:50.801717 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:19:50.903900 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:50.903834 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:19:53.005855 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:53.005818 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:19:53.106509 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:19:53.106484 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:20:01.527680 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:01.527643 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:20:01.611087 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:01.611055 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:20:02.021453 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:02.021419 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:20:02.121886 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:02.121859 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:20:06.780551 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:06.780520 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z"] Jul 13 09:20:06.780968 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:06.780952 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af6e4f34-3bd7-46f2-95dc-f99cc52699f3" containerName="console" Jul 13 09:20:06.781015 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:06.780971 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="af6e4f34-3bd7-46f2-95dc-f99cc52699f3" containerName="console" Jul 13 09:20:06.781050 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:06.781037 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="af6e4f34-3bd7-46f2-95dc-f99cc52699f3" containerName="console" Jul 13 09:20:06.809801 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:06.809768 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:20:06.915787 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:06.915758 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z"] Jul 13 09:20:06.915952 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:06.915855 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" Jul 13 09:20:06.919175 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:06.919145 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-56f8p\"" Jul 13 09:20:06.996508 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:06.996483 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8863895d-7df7-4937-a6a5-06472e72737a-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-cmw8z\" (UID: \"8863895d-7df7-4937-a6a5-06472e72737a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" Jul 13 09:20:06.996636 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:06.996515 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8863895d-7df7-4937-a6a5-06472e72737a-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-cmw8z\" (UID: \"8863895d-7df7-4937-a6a5-06472e72737a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" Jul 13 09:20:06.996636 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:06.996534 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8863895d-7df7-4937-a6a5-06472e72737a-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-cmw8z\" (UID: \"8863895d-7df7-4937-a6a5-06472e72737a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" Jul 13 09:20:06.996636 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:06.996587 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8863895d-7df7-4937-a6a5-06472e72737a-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-cmw8z\" (UID: \"8863895d-7df7-4937-a6a5-06472e72737a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" Jul 13 09:20:06.996636 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:06.996629 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8863895d-7df7-4937-a6a5-06472e72737a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-cmw8z\" (UID: \"8863895d-7df7-4937-a6a5-06472e72737a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" Jul 13 09:20:06.996849 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:06.996704 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vd5tr\" (UniqueName: \"kubernetes.io/projected/8863895d-7df7-4937-a6a5-06472e72737a-kube-api-access-vd5tr\") pod \"maas-default-gateway-openshift-default-56654dcf7-cmw8z\" (UID: \"8863895d-7df7-4937-a6a5-06472e72737a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" Jul 13 09:20:06.996849 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:06.996750 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8863895d-7df7-4937-a6a5-06472e72737a-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-cmw8z\" (UID: \"8863895d-7df7-4937-a6a5-06472e72737a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" Jul 13 09:20:06.996849 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:06.996772 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8863895d-7df7-4937-a6a5-06472e72737a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-cmw8z\" (UID: \"8863895d-7df7-4937-a6a5-06472e72737a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" Jul 13 09:20:06.996849 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:06.996827 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8863895d-7df7-4937-a6a5-06472e72737a-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-cmw8z\" (UID: \"8863895d-7df7-4937-a6a5-06472e72737a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" Jul 13 09:20:07.097587 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:07.097558 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8863895d-7df7-4937-a6a5-06472e72737a-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-cmw8z\" (UID: \"8863895d-7df7-4937-a6a5-06472e72737a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" Jul 13 09:20:07.097726 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:07.097591 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8863895d-7df7-4937-a6a5-06472e72737a-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-cmw8z\" (UID: \"8863895d-7df7-4937-a6a5-06472e72737a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" Jul 13 09:20:07.097726 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:07.097611 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8863895d-7df7-4937-a6a5-06472e72737a-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-cmw8z\" (UID: \"8863895d-7df7-4937-a6a5-06472e72737a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" Jul 13 09:20:07.097726 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:07.097630 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8863895d-7df7-4937-a6a5-06472e72737a-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-cmw8z\" (UID: \"8863895d-7df7-4937-a6a5-06472e72737a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" Jul 13 09:20:07.097726 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:07.097664 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8863895d-7df7-4937-a6a5-06472e72737a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-cmw8z\" (UID: \"8863895d-7df7-4937-a6a5-06472e72737a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" Jul 13 09:20:07.097936 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:07.097758 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vd5tr\" (UniqueName: \"kubernetes.io/projected/8863895d-7df7-4937-a6a5-06472e72737a-kube-api-access-vd5tr\") pod \"maas-default-gateway-openshift-default-56654dcf7-cmw8z\" (UID: \"8863895d-7df7-4937-a6a5-06472e72737a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" Jul 13 09:20:07.097936 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:07.097811 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8863895d-7df7-4937-a6a5-06472e72737a-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-cmw8z\" (UID: \"8863895d-7df7-4937-a6a5-06472e72737a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" Jul 13 09:20:07.097936 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:07.097836 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8863895d-7df7-4937-a6a5-06472e72737a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-cmw8z\" (UID: \"8863895d-7df7-4937-a6a5-06472e72737a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" Jul 13 09:20:07.097936 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:07.097897 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8863895d-7df7-4937-a6a5-06472e72737a-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-cmw8z\" (UID: \"8863895d-7df7-4937-a6a5-06472e72737a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" Jul 13 09:20:07.098190 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:07.098006 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8863895d-7df7-4937-a6a5-06472e72737a-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-cmw8z\" (UID: \"8863895d-7df7-4937-a6a5-06472e72737a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" Jul 13 09:20:07.098275 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:07.098240 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8863895d-7df7-4937-a6a5-06472e72737a-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-cmw8z\" (UID: \"8863895d-7df7-4937-a6a5-06472e72737a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" Jul 13 09:20:07.098368 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:07.098346 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8863895d-7df7-4937-a6a5-06472e72737a-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-cmw8z\" (UID: \"8863895d-7df7-4937-a6a5-06472e72737a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" Jul 13 09:20:07.098425 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:07.098410 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8863895d-7df7-4937-a6a5-06472e72737a-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-cmw8z\" (UID: \"8863895d-7df7-4937-a6a5-06472e72737a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" Jul 13 09:20:07.098721 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:07.098698 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8863895d-7df7-4937-a6a5-06472e72737a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-cmw8z\" (UID: \"8863895d-7df7-4937-a6a5-06472e72737a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" Jul 13 09:20:07.100171 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:07.100151 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8863895d-7df7-4937-a6a5-06472e72737a-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-cmw8z\" (UID: \"8863895d-7df7-4937-a6a5-06472e72737a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" Jul 13 09:20:07.100359 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:07.100342 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8863895d-7df7-4937-a6a5-06472e72737a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-cmw8z\" (UID: \"8863895d-7df7-4937-a6a5-06472e72737a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" Jul 13 09:20:07.107004 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:07.106975 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8863895d-7df7-4937-a6a5-06472e72737a-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-cmw8z\" (UID: \"8863895d-7df7-4937-a6a5-06472e72737a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" Jul 13 09:20:07.107397 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:07.107378 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vd5tr\" (UniqueName: \"kubernetes.io/projected/8863895d-7df7-4937-a6a5-06472e72737a-kube-api-access-vd5tr\") pod \"maas-default-gateway-openshift-default-56654dcf7-cmw8z\" (UID: \"8863895d-7df7-4937-a6a5-06472e72737a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" Jul 13 09:20:07.226584 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:07.226558 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" Jul 13 09:20:07.236106 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:07.236078 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:20:07.456943 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:07.456917 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z"] Jul 13 09:20:07.457574 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:20:07.457549 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8863895d_7df7_4937_a6a5_06472e72737a.slice/crio-3fe7775a7293a2a26ea1eccd568110658f43594bbec448d8a73968b1e7c77fb3 WatchSource:0}: Error finding container 3fe7775a7293a2a26ea1eccd568110658f43594bbec448d8a73968b1e7c77fb3: Status 404 returned error can't find the container with id 3fe7775a7293a2a26ea1eccd568110658f43594bbec448d8a73968b1e7c77fb3 Jul 13 09:20:07.459744 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:07.459715 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:20:07.459817 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:07.459774 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:20:07.459817 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:07.459803 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:20:07.527677 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:07.527648 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" event={"ID":"8863895d-7df7-4937-a6a5-06472e72737a","Type":"ContainerStarted","Data":"010a872e309f3ecfee1b1e84db5dd480004399c3e189089ed7171924af8293b3"} Jul 13 09:20:07.527793 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:07.527695 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" event={"ID":"8863895d-7df7-4937-a6a5-06472e72737a","Type":"ContainerStarted","Data":"3fe7775a7293a2a26ea1eccd568110658f43594bbec448d8a73968b1e7c77fb3"} Jul 13 09:20:07.550466 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:07.550418 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" podStartSLOduration=1.550403127 podStartE2EDuration="1.550403127s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:20:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:20:07.548747879 +0000 UTC m=+691.428703138" watchObservedRunningTime="2026-07-13 09:20:07.550403127 +0000 UTC m=+691.430358394" Jul 13 09:20:08.227121 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:08.227088 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" Jul 13 09:20:08.231427 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:08.231403 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" Jul 13 09:20:08.532960 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:08.532885 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" Jul 13 09:20:08.533755 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:08.533735 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" Jul 13 09:20:12.230256 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:12.230194 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:20:12.346713 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:12.346685 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:20:12.418390 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:12.418359 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:20:13.107761 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:13.107730 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:20:13.209235 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:13.209177 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:20:13.233563 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:13.233534 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-kfjb8"] Jul 13 09:20:13.237779 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:13.237761 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-kfjb8" Jul 13 09:20:13.240453 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:13.240438 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-575dh\"" Jul 13 09:20:13.244680 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:13.244657 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-kfjb8"] Jul 13 09:20:13.348846 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:13.348813 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4bnt\" (UniqueName: \"kubernetes.io/projected/6b62b2ad-7d1d-4a60-af36-c8582c6864f4-kube-api-access-s4bnt\") pod \"authorino-75d556bd77-kfjb8\" (UID: \"6b62b2ad-7d1d-4a60-af36-c8582c6864f4\") " pod="kuadrant-system/authorino-75d556bd77-kfjb8" Jul 13 09:20:13.449204 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:13.449138 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4bnt\" (UniqueName: \"kubernetes.io/projected/6b62b2ad-7d1d-4a60-af36-c8582c6864f4-kube-api-access-s4bnt\") pod \"authorino-75d556bd77-kfjb8\" (UID: \"6b62b2ad-7d1d-4a60-af36-c8582c6864f4\") " pod="kuadrant-system/authorino-75d556bd77-kfjb8" Jul 13 09:20:13.458294 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:13.458274 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4bnt\" (UniqueName: \"kubernetes.io/projected/6b62b2ad-7d1d-4a60-af36-c8582c6864f4-kube-api-access-s4bnt\") pod \"authorino-75d556bd77-kfjb8\" (UID: \"6b62b2ad-7d1d-4a60-af36-c8582c6864f4\") " pod="kuadrant-system/authorino-75d556bd77-kfjb8" Jul 13 09:20:13.548316 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:13.548293 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-kfjb8" Jul 13 09:20:13.680894 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:13.680871 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-kfjb8"] Jul 13 09:20:13.682138 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:20:13.682109 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6b62b2ad_7d1d_4a60_af36_c8582c6864f4.slice/crio-6107bca1cc0c25cb72f97729cfae544a1bd5e87a5fca772fcecb5fb774bdd407 WatchSource:0}: Error finding container 6107bca1cc0c25cb72f97729cfae544a1bd5e87a5fca772fcecb5fb774bdd407: Status 404 returned error can't find the container with id 6107bca1cc0c25cb72f97729cfae544a1bd5e87a5fca772fcecb5fb774bdd407 Jul 13 09:20:13.683539 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:13.683521 2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 09:20:14.588102 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:14.584299 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-kfjb8" event={"ID":"6b62b2ad-7d1d-4a60-af36-c8582c6864f4","Type":"ContainerStarted","Data":"6107bca1cc0c25cb72f97729cfae544a1bd5e87a5fca772fcecb5fb774bdd407"} Jul 13 09:20:17.606116 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:17.606079 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-kfjb8" event={"ID":"6b62b2ad-7d1d-4a60-af36-c8582c6864f4","Type":"ContainerStarted","Data":"fda9d56761ffede4ca2171ac5ab13a211c11d223c1294608255c2f57493efafe"} Jul 13 09:20:17.625036 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:17.624986 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-kfjb8" podStartSLOduration=1.248603209 podStartE2EDuration="4.624971679s" totalImagesPullingTime="3.37636847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:20:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:20:17.622334741 +0000 UTC m=+701.502290010" watchObservedRunningTime="2026-07-13 09:20:17.624971679 +0000 UTC m=+701.504926946" Jul 13 09:20:21.443736 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:21.443691 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:20:29.793983 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:29.793898 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:20:29.890571 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:29.890540 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:20:30.254548 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:30.254505 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:20:42.892106 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:42.892055 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:20:43.299337 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:43.299195 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:20:43.618114 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:43.618084 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-54d78f4d4-dwr5r"] Jul 13 09:20:43.623307 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:43.623285 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54d78f4d4-dwr5r" Jul 13 09:20:43.627019 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:43.626994 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 13 09:20:43.629725 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:43.629698 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-54d78f4d4-dwr5r"] Jul 13 09:20:43.693888 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:43.693861 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/941d7b1a-250c-4d0e-afb3-1fc2925c5b1f-tls-cert\") pod \"authorino-54d78f4d4-dwr5r\" (UID: \"941d7b1a-250c-4d0e-afb3-1fc2925c5b1f\") " pod="kuadrant-system/authorino-54d78f4d4-dwr5r" Jul 13 09:20:43.694002 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:43.693891 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2lzv\" (UniqueName: \"kubernetes.io/projected/941d7b1a-250c-4d0e-afb3-1fc2925c5b1f-kube-api-access-t2lzv\") pod \"authorino-54d78f4d4-dwr5r\" (UID: \"941d7b1a-250c-4d0e-afb3-1fc2925c5b1f\") " pod="kuadrant-system/authorino-54d78f4d4-dwr5r" Jul 13 09:20:43.794343 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:43.794316 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/941d7b1a-250c-4d0e-afb3-1fc2925c5b1f-tls-cert\") pod \"authorino-54d78f4d4-dwr5r\" (UID: \"941d7b1a-250c-4d0e-afb3-1fc2925c5b1f\") " pod="kuadrant-system/authorino-54d78f4d4-dwr5r" Jul 13 09:20:43.794448 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:43.794346 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2lzv\" (UniqueName: \"kubernetes.io/projected/941d7b1a-250c-4d0e-afb3-1fc2925c5b1f-kube-api-access-t2lzv\") pod \"authorino-54d78f4d4-dwr5r\" (UID: \"941d7b1a-250c-4d0e-afb3-1fc2925c5b1f\") " pod="kuadrant-system/authorino-54d78f4d4-dwr5r" Jul 13 09:20:43.796632 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:43.796613 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/941d7b1a-250c-4d0e-afb3-1fc2925c5b1f-tls-cert\") pod \"authorino-54d78f4d4-dwr5r\" (UID: \"941d7b1a-250c-4d0e-afb3-1fc2925c5b1f\") " pod="kuadrant-system/authorino-54d78f4d4-dwr5r" Jul 13 09:20:43.811391 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:43.811372 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2lzv\" (UniqueName: \"kubernetes.io/projected/941d7b1a-250c-4d0e-afb3-1fc2925c5b1f-kube-api-access-t2lzv\") pod \"authorino-54d78f4d4-dwr5r\" (UID: \"941d7b1a-250c-4d0e-afb3-1fc2925c5b1f\") " pod="kuadrant-system/authorino-54d78f4d4-dwr5r" Jul 13 09:20:43.934395 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:43.934340 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54d78f4d4-dwr5r" Jul 13 09:20:44.066913 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:44.066888 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-54d78f4d4-dwr5r"] Jul 13 09:20:44.068361 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:20:44.068333 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod941d7b1a_250c_4d0e_afb3_1fc2925c5b1f.slice/crio-7c636ccc017ca5d405e960bd1c44aeaf78458c8a47c53c3f5c42eb9b56f3d1b0 WatchSource:0}: Error finding container 7c636ccc017ca5d405e960bd1c44aeaf78458c8a47c53c3f5c42eb9b56f3d1b0: Status 404 returned error can't find the container with id 7c636ccc017ca5d405e960bd1c44aeaf78458c8a47c53c3f5c42eb9b56f3d1b0 Jul 13 09:20:44.750425 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:44.750330 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54d78f4d4-dwr5r" event={"ID":"941d7b1a-250c-4d0e-afb3-1fc2925c5b1f","Type":"ContainerStarted","Data":"c7e785e91236227477817d0241b1c72a442910447113f1154a2f39ea1efe0681"} Jul 13 09:20:44.750425 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:44.750373 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54d78f4d4-dwr5r" event={"ID":"941d7b1a-250c-4d0e-afb3-1fc2925c5b1f","Type":"ContainerStarted","Data":"7c636ccc017ca5d405e960bd1c44aeaf78458c8a47c53c3f5c42eb9b56f3d1b0"} Jul 13 09:20:44.768910 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:44.768866 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-54d78f4d4-dwr5r" podStartSLOduration=1.3782855999999999 podStartE2EDuration="1.768853627s" totalImagesPullingTime="390.568027ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:20:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:20:44.768651086 +0000 UTC m=+728.648606352" watchObservedRunningTime="2026-07-13 09:20:44.768853627 +0000 UTC m=+728.648808894" Jul 13 09:20:44.806108 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:44.802192 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-kfjb8"] Jul 13 09:20:44.806108 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:44.802445 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-kfjb8" podUID="6b62b2ad-7d1d-4a60-af36-c8582c6864f4" containerName="authorino" containerID="cri-o://fda9d56761ffede4ca2171ac5ab13a211c11d223c1294608255c2f57493efafe" gracePeriod=30 Jul 13 09:20:45.059600 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:45.059574 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-kfjb8" Jul 13 09:20:45.109104 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:45.109067 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s4bnt\" (UniqueName: \"kubernetes.io/projected/6b62b2ad-7d1d-4a60-af36-c8582c6864f4-kube-api-access-s4bnt\") pod \"6b62b2ad-7d1d-4a60-af36-c8582c6864f4\" (UID: \"6b62b2ad-7d1d-4a60-af36-c8582c6864f4\") " Jul 13 09:20:45.111012 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:45.110983 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b62b2ad-7d1d-4a60-af36-c8582c6864f4-kube-api-access-s4bnt" (OuterVolumeSpecName: "kube-api-access-s4bnt") pod "6b62b2ad-7d1d-4a60-af36-c8582c6864f4" (UID: "6b62b2ad-7d1d-4a60-af36-c8582c6864f4"). InnerVolumeSpecName "kube-api-access-s4bnt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:20:45.210541 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:45.210507 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s4bnt\" (UniqueName: \"kubernetes.io/projected/6b62b2ad-7d1d-4a60-af36-c8582c6864f4-kube-api-access-s4bnt\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:20:45.756775 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:45.756740 2552 generic.go:356] "Generic (PLEG): container finished" podID="6b62b2ad-7d1d-4a60-af36-c8582c6864f4" containerID="fda9d56761ffede4ca2171ac5ab13a211c11d223c1294608255c2f57493efafe" exitCode=0 Jul 13 09:20:45.757000 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:45.756779 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-kfjb8" Jul 13 09:20:45.757000 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:45.756822 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-kfjb8" event={"ID":"6b62b2ad-7d1d-4a60-af36-c8582c6864f4","Type":"ContainerDied","Data":"fda9d56761ffede4ca2171ac5ab13a211c11d223c1294608255c2f57493efafe"} Jul 13 09:20:45.757000 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:45.756863 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-kfjb8" event={"ID":"6b62b2ad-7d1d-4a60-af36-c8582c6864f4","Type":"ContainerDied","Data":"6107bca1cc0c25cb72f97729cfae544a1bd5e87a5fca772fcecb5fb774bdd407"} Jul 13 09:20:45.757000 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:45.756880 2552 scope.go:117] "RemoveContainer" containerID="fda9d56761ffede4ca2171ac5ab13a211c11d223c1294608255c2f57493efafe" Jul 13 09:20:45.765779 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:45.765764 2552 scope.go:117] "RemoveContainer" containerID="fda9d56761ffede4ca2171ac5ab13a211c11d223c1294608255c2f57493efafe" Jul 13 09:20:45.766042 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:20:45.766026 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fda9d56761ffede4ca2171ac5ab13a211c11d223c1294608255c2f57493efafe\": container with ID starting with fda9d56761ffede4ca2171ac5ab13a211c11d223c1294608255c2f57493efafe not found: ID does not exist" containerID="fda9d56761ffede4ca2171ac5ab13a211c11d223c1294608255c2f57493efafe" Jul 13 09:20:45.766098 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:45.766049 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fda9d56761ffede4ca2171ac5ab13a211c11d223c1294608255c2f57493efafe"} err="failed to get container status \"fda9d56761ffede4ca2171ac5ab13a211c11d223c1294608255c2f57493efafe\": rpc error: code = NotFound desc = could not find container \"fda9d56761ffede4ca2171ac5ab13a211c11d223c1294608255c2f57493efafe\": container with ID starting with fda9d56761ffede4ca2171ac5ab13a211c11d223c1294608255c2f57493efafe not found: ID does not exist" Jul 13 09:20:45.782699 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:45.782675 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-kfjb8"] Jul 13 09:20:45.784941 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:45.784918 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-kfjb8"] Jul 13 09:20:46.666108 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:46.666076 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b62b2ad-7d1d-4a60-af36-c8582c6864f4" path="/var/lib/kubelet/pods/6b62b2ad-7d1d-4a60-af36-c8582c6864f4/volumes" Jul 13 09:20:49.285382 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:49.285353 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:20:49.499974 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:49.499946 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:20:49.584533 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:49.584503 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:20:49.743773 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:49.743745 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:20:53.122295 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:53.122259 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:20:53.225527 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:53.225483 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:20:58.310576 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:20:58.310541 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:21:25.495110 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:21:25.495074 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:21:26.003799 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:21:26.003763 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:21:26.416752 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:21:26.416722 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:21:26.552329 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:21:26.552303 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:21:26.650754 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:21:26.650725 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:21:35.027843 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:21:35.027806 2552 generic.go:356] "Generic (PLEG): container finished" podID="8863895d-7df7-4937-a6a5-06472e72737a" containerID="010a872e309f3ecfee1b1e84db5dd480004399c3e189089ed7171924af8293b3" exitCode=0 Jul 13 09:21:35.028295 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:21:35.027874 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" event={"ID":"8863895d-7df7-4937-a6a5-06472e72737a","Type":"ContainerDied","Data":"010a872e309f3ecfee1b1e84db5dd480004399c3e189089ed7171924af8293b3"} Jul 13 09:21:35.028379 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:21:35.028320 2552 scope.go:117] "RemoveContainer" containerID="010a872e309f3ecfee1b1e84db5dd480004399c3e189089ed7171924af8293b3" Jul 13 09:21:35.029296 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:21:35.029270 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:21:35.029362 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:21:35.029336 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:21:35.029399 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:21:35.029370 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:21:36.035262 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:21:36.035233 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" event={"ID":"8863895d-7df7-4937-a6a5-06472e72737a","Type":"ContainerStarted","Data":"7af5156b4f31c654e035071839eb414df4c7640f2a9cdd60bf8b4e69b90e657f"} Jul 13 09:21:36.227540 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:21:36.227498 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" Jul 13 09:21:36.228762 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:21:36.228733 2552 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-cmw8z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 13 09:21:36.228888 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:21:36.228792 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" podUID="8863895d-7df7-4937-a6a5-06472e72737a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 13 09:21:36.815697 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:21:36.815661 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:21:36.914373 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:21:36.914293 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:21:37.227131 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:21:37.227042 2552 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-cmw8z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 13 09:21:37.227131 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:21:37.227102 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" podUID="8863895d-7df7-4937-a6a5-06472e72737a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 13 09:21:38.226957 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:21:38.226921 2552 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-cmw8z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 13 09:21:38.227122 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:21:38.226980 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" podUID="8863895d-7df7-4937-a6a5-06472e72737a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 13 09:21:38.533672 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:21:38.533582 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" Jul 13 09:21:39.227746 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:21:39.227702 2552 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-cmw8z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 13 09:21:39.227952 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:21:39.227776 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" podUID="8863895d-7df7-4937-a6a5-06472e72737a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 13 09:21:40.231418 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:21:40.231390 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" Jul 13 09:21:40.232507 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:21:40.232481 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cmw8z" Jul 13 09:21:44.783956 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:21:44.783923 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:22:09.248382 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:09.248304 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6kx2s"] Jul 13 09:22:09.248816 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:09.248666 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b62b2ad-7d1d-4a60-af36-c8582c6864f4" containerName="authorino" Jul 13 09:22:09.248816 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:09.248677 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b62b2ad-7d1d-4a60-af36-c8582c6864f4" containerName="authorino" Jul 13 09:22:09.248816 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:09.248743 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b62b2ad-7d1d-4a60-af36-c8582c6864f4" containerName="authorino" Jul 13 09:22:09.251669 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:09.251652 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6kx2s" Jul 13 09:22:09.255854 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:09.255826 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 13 09:22:09.255993 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:09.255828 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-96d22\"" Jul 13 09:22:09.255993 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:09.255867 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 13 09:22:09.256144 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:09.256130 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 13 09:22:09.268445 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:09.268424 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6kx2s"] Jul 13 09:22:09.273833 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:09.273807 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:22:09.278998 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:09.278962 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkkp5\" (UniqueName: \"kubernetes.io/projected/902feeb3-2172-433a-8d33-5a86d16d1513-kube-api-access-jkkp5\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6kx2s\" (UID: \"902feeb3-2172-433a-8d33-5a86d16d1513\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6kx2s" Jul 13 09:22:09.279109 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:09.279027 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/902feeb3-2172-433a-8d33-5a86d16d1513-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6kx2s\" (UID: \"902feeb3-2172-433a-8d33-5a86d16d1513\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6kx2s" Jul 13 09:22:09.279109 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:09.279089 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/902feeb3-2172-433a-8d33-5a86d16d1513-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6kx2s\" (UID: \"902feeb3-2172-433a-8d33-5a86d16d1513\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6kx2s" Jul 13 09:22:09.279245 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:09.279113 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/902feeb3-2172-433a-8d33-5a86d16d1513-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6kx2s\" (UID: \"902feeb3-2172-433a-8d33-5a86d16d1513\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6kx2s" Jul 13 09:22:09.279245 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:09.279146 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/902feeb3-2172-433a-8d33-5a86d16d1513-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6kx2s\" (UID: \"902feeb3-2172-433a-8d33-5a86d16d1513\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6kx2s" Jul 13 09:22:09.279344 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:09.279246 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/902feeb3-2172-433a-8d33-5a86d16d1513-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6kx2s\" (UID: \"902feeb3-2172-433a-8d33-5a86d16d1513\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6kx2s" Jul 13 09:22:09.380079 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:09.380052 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkkp5\" (UniqueName: \"kubernetes.io/projected/902feeb3-2172-433a-8d33-5a86d16d1513-kube-api-access-jkkp5\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6kx2s\" (UID: \"902feeb3-2172-433a-8d33-5a86d16d1513\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6kx2s" Jul 13 09:22:09.380177 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:09.380090 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/902feeb3-2172-433a-8d33-5a86d16d1513-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6kx2s\" (UID: \"902feeb3-2172-433a-8d33-5a86d16d1513\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6kx2s" Jul 13 09:22:09.380177 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:09.380122 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/902feeb3-2172-433a-8d33-5a86d16d1513-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6kx2s\" (UID: \"902feeb3-2172-433a-8d33-5a86d16d1513\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6kx2s" Jul 13 09:22:09.380177 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:09.380138 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/902feeb3-2172-433a-8d33-5a86d16d1513-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6kx2s\" (UID: \"902feeb3-2172-433a-8d33-5a86d16d1513\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6kx2s" Jul 13 09:22:09.380177 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:09.380157 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/902feeb3-2172-433a-8d33-5a86d16d1513-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6kx2s\" (UID: \"902feeb3-2172-433a-8d33-5a86d16d1513\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6kx2s" Jul 13 09:22:09.380424 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:09.380191 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/902feeb3-2172-433a-8d33-5a86d16d1513-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6kx2s\" (UID: \"902feeb3-2172-433a-8d33-5a86d16d1513\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6kx2s" Jul 13 09:22:09.380619 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:09.380599 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/902feeb3-2172-433a-8d33-5a86d16d1513-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6kx2s\" (UID: \"902feeb3-2172-433a-8d33-5a86d16d1513\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6kx2s" Jul 13 09:22:09.380680 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:09.380664 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/902feeb3-2172-433a-8d33-5a86d16d1513-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6kx2s\" (UID: \"902feeb3-2172-433a-8d33-5a86d16d1513\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6kx2s" Jul 13 09:22:09.380717 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:09.380688 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/902feeb3-2172-433a-8d33-5a86d16d1513-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6kx2s\" (UID: \"902feeb3-2172-433a-8d33-5a86d16d1513\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6kx2s" Jul 13 09:22:09.382418 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:09.382394 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/902feeb3-2172-433a-8d33-5a86d16d1513-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6kx2s\" (UID: \"902feeb3-2172-433a-8d33-5a86d16d1513\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6kx2s" Jul 13 09:22:09.382545 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:09.382523 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/902feeb3-2172-433a-8d33-5a86d16d1513-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6kx2s\" (UID: \"902feeb3-2172-433a-8d33-5a86d16d1513\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6kx2s" Jul 13 09:22:09.391499 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:09.391477 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkkp5\" (UniqueName: \"kubernetes.io/projected/902feeb3-2172-433a-8d33-5a86d16d1513-kube-api-access-jkkp5\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6kx2s\" (UID: \"902feeb3-2172-433a-8d33-5a86d16d1513\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6kx2s" Jul 13 09:22:09.563149 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:09.563066 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6kx2s" Jul 13 09:22:09.704379 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:09.704353 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6kx2s"] Jul 13 09:22:09.705864 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:22:09.705834 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod902feeb3_2172_433a_8d33_5a86d16d1513.slice/crio-fc8eefaa53eba37cd60865d5860e3cfe050149dc79a8ec1c89915be19e309432 WatchSource:0}: Error finding container fc8eefaa53eba37cd60865d5860e3cfe050149dc79a8ec1c89915be19e309432: Status 404 returned error can't find the container with id fc8eefaa53eba37cd60865d5860e3cfe050149dc79a8ec1c89915be19e309432 Jul 13 09:22:10.233020 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:10.232902 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6kx2s" event={"ID":"902feeb3-2172-433a-8d33-5a86d16d1513","Type":"ContainerStarted","Data":"fc8eefaa53eba37cd60865d5860e3cfe050149dc79a8ec1c89915be19e309432"} Jul 13 09:22:11.240056 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:11.239972 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6kx2s" event={"ID":"902feeb3-2172-433a-8d33-5a86d16d1513","Type":"ContainerStarted","Data":"5551986c88785e73d69d04546d2b1af73c313ebfe7071dae8dccf3e8cdbf6ce8"} Jul 13 09:22:11.259269 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:11.259191 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6kx2s" podStartSLOduration=1.081735355 podStartE2EDuration="2.259175845s" totalImagesPullingTime="1.17744049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:22:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:22:11.25899295 +0000 UTC m=+815.138948219" watchObservedRunningTime="2026-07-13 09:22:11.259175845 +0000 UTC m=+815.139131111" Jul 13 09:22:14.984240 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:14.984193 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:22:19.563369 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:19.563333 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6kx2s" Jul 13 09:22:19.563737 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:19.563403 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6kx2s" Jul 13 09:22:19.575830 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:19.575805 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6kx2s" Jul 13 09:22:20.301267 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:20.301229 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6kx2s" Jul 13 09:22:28.005826 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:28.005791 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:22:33.982135 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:33.982102 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:22:40.062185 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:40.062151 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr"] Jul 13 09:22:40.066065 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:40.066039 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr" Jul 13 09:22:40.068835 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:40.068816 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 13 09:22:40.078758 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:40.078735 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr"] Jul 13 09:22:40.099563 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:40.099541 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:22:40.129754 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:40.129731 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7724876e-e287-4b92-b0bc-7d75618ea5ec-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr\" (UID: \"7724876e-e287-4b92-b0bc-7d75618ea5ec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr" Jul 13 09:22:40.129847 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:40.129767 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7724876e-e287-4b92-b0bc-7d75618ea5ec-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr\" (UID: \"7724876e-e287-4b92-b0bc-7d75618ea5ec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr" Jul 13 09:22:40.129847 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:40.129786 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bxch\" (UniqueName: \"kubernetes.io/projected/7724876e-e287-4b92-b0bc-7d75618ea5ec-kube-api-access-4bxch\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr\" (UID: \"7724876e-e287-4b92-b0bc-7d75618ea5ec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr" Jul 13 09:22:40.129926 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:40.129898 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7724876e-e287-4b92-b0bc-7d75618ea5ec-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr\" (UID: \"7724876e-e287-4b92-b0bc-7d75618ea5ec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr" Jul 13 09:22:40.129963 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:40.129922 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7724876e-e287-4b92-b0bc-7d75618ea5ec-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr\" (UID: \"7724876e-e287-4b92-b0bc-7d75618ea5ec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr" Jul 13 09:22:40.130016 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:40.129997 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7724876e-e287-4b92-b0bc-7d75618ea5ec-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr\" (UID: \"7724876e-e287-4b92-b0bc-7d75618ea5ec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr" Jul 13 09:22:40.230819 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:40.230788 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7724876e-e287-4b92-b0bc-7d75618ea5ec-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr\" (UID: \"7724876e-e287-4b92-b0bc-7d75618ea5ec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr" Jul 13 09:22:40.230944 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:40.230825 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7724876e-e287-4b92-b0bc-7d75618ea5ec-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr\" (UID: \"7724876e-e287-4b92-b0bc-7d75618ea5ec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr" Jul 13 09:22:40.230944 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:40.230843 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bxch\" (UniqueName: \"kubernetes.io/projected/7724876e-e287-4b92-b0bc-7d75618ea5ec-kube-api-access-4bxch\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr\" (UID: \"7724876e-e287-4b92-b0bc-7d75618ea5ec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr" Jul 13 09:22:40.230944 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:40.230881 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7724876e-e287-4b92-b0bc-7d75618ea5ec-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr\" (UID: \"7724876e-e287-4b92-b0bc-7d75618ea5ec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr" Jul 13 09:22:40.230944 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:40.230897 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7724876e-e287-4b92-b0bc-7d75618ea5ec-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr\" (UID: \"7724876e-e287-4b92-b0bc-7d75618ea5ec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr" Jul 13 09:22:40.230944 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:40.230922 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7724876e-e287-4b92-b0bc-7d75618ea5ec-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr\" (UID: \"7724876e-e287-4b92-b0bc-7d75618ea5ec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr" Jul 13 09:22:40.231286 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:40.231259 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7724876e-e287-4b92-b0bc-7d75618ea5ec-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr\" (UID: \"7724876e-e287-4b92-b0bc-7d75618ea5ec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr" Jul 13 09:22:40.231374 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:40.231304 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7724876e-e287-4b92-b0bc-7d75618ea5ec-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr\" (UID: \"7724876e-e287-4b92-b0bc-7d75618ea5ec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr" Jul 13 09:22:40.231374 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:40.231333 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7724876e-e287-4b92-b0bc-7d75618ea5ec-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr\" (UID: \"7724876e-e287-4b92-b0bc-7d75618ea5ec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr" Jul 13 09:22:40.233183 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:40.233150 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7724876e-e287-4b92-b0bc-7d75618ea5ec-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr\" (UID: \"7724876e-e287-4b92-b0bc-7d75618ea5ec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr" Jul 13 09:22:40.233484 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:40.233467 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7724876e-e287-4b92-b0bc-7d75618ea5ec-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr\" (UID: \"7724876e-e287-4b92-b0bc-7d75618ea5ec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr" Jul 13 09:22:40.242022 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:40.241995 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bxch\" (UniqueName: \"kubernetes.io/projected/7724876e-e287-4b92-b0bc-7d75618ea5ec-kube-api-access-4bxch\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr\" (UID: \"7724876e-e287-4b92-b0bc-7d75618ea5ec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr" Jul 13 09:22:40.375637 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:40.375607 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr" Jul 13 09:22:40.512142 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:40.512116 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr"] Jul 13 09:22:40.514865 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:22:40.514768 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7724876e_e287_4b92_b0bc_7d75618ea5ec.slice/crio-ce1737837355f3457ae7da5dbac522b7ad83fda72efa630735612f5f0e957e2b WatchSource:0}: Error finding container ce1737837355f3457ae7da5dbac522b7ad83fda72efa630735612f5f0e957e2b: Status 404 returned error can't find the container with id ce1737837355f3457ae7da5dbac522b7ad83fda72efa630735612f5f0e957e2b Jul 13 09:22:41.409008 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:41.408969 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr" event={"ID":"7724876e-e287-4b92-b0bc-7d75618ea5ec","Type":"ContainerStarted","Data":"07664cd7306c716a263594c6c446da8aca92a8d09c4b2faa6eeb296ba6357bd7"} Jul 13 09:22:41.409388 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:41.409019 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr" event={"ID":"7724876e-e287-4b92-b0bc-7d75618ea5ec","Type":"ContainerStarted","Data":"ce1737837355f3457ae7da5dbac522b7ad83fda72efa630735612f5f0e957e2b"} Jul 13 09:22:41.427348 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:41.427302 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr" podStartSLOduration=1.209309955 podStartE2EDuration="1.427287687s" totalImagesPullingTime="217.977732ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:22:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:22:41.426462401 +0000 UTC m=+845.306417668" watchObservedRunningTime="2026-07-13 09:22:41.427287687 +0000 UTC m=+845.307242953" Jul 13 09:22:50.376285 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:50.376249 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr" Jul 13 09:22:50.376651 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:50.376339 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr" Jul 13 09:22:50.388430 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:50.388409 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr" Jul 13 09:22:50.474825 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:22:50.474799 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr" Jul 13 09:23:32.144430 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:23:32.144352 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-67757b8bf-58t5v"] Jul 13 09:23:32.148592 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:23:32.148571 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67757b8bf-58t5v" Jul 13 09:23:32.169479 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:23:32.169453 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-67757b8bf-58t5v"] Jul 13 09:23:32.233883 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:23:32.233855 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psdx2\" (UniqueName: \"kubernetes.io/projected/7c2323b5-0d28-4d00-a77a-b880a3920ffd-kube-api-access-psdx2\") pod \"authorino-67757b8bf-58t5v\" (UID: \"7c2323b5-0d28-4d00-a77a-b880a3920ffd\") " pod="kuadrant-system/authorino-67757b8bf-58t5v" Jul 13 09:23:32.233984 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:23:32.233933 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7c2323b5-0d28-4d00-a77a-b880a3920ffd-tls-cert\") pod \"authorino-67757b8bf-58t5v\" (UID: \"7c2323b5-0d28-4d00-a77a-b880a3920ffd\") " pod="kuadrant-system/authorino-67757b8bf-58t5v" Jul 13 09:23:32.334444 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:23:32.334415 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7c2323b5-0d28-4d00-a77a-b880a3920ffd-tls-cert\") pod \"authorino-67757b8bf-58t5v\" (UID: \"7c2323b5-0d28-4d00-a77a-b880a3920ffd\") " pod="kuadrant-system/authorino-67757b8bf-58t5v" Jul 13 09:23:32.334561 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:23:32.334476 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psdx2\" (UniqueName: \"kubernetes.io/projected/7c2323b5-0d28-4d00-a77a-b880a3920ffd-kube-api-access-psdx2\") pod \"authorino-67757b8bf-58t5v\" (UID: \"7c2323b5-0d28-4d00-a77a-b880a3920ffd\") " pod="kuadrant-system/authorino-67757b8bf-58t5v" Jul 13 09:23:32.336719 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:23:32.336697 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7c2323b5-0d28-4d00-a77a-b880a3920ffd-tls-cert\") pod \"authorino-67757b8bf-58t5v\" (UID: \"7c2323b5-0d28-4d00-a77a-b880a3920ffd\") " pod="kuadrant-system/authorino-67757b8bf-58t5v" Jul 13 09:23:32.343025 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:23:32.343004 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-psdx2\" (UniqueName: \"kubernetes.io/projected/7c2323b5-0d28-4d00-a77a-b880a3920ffd-kube-api-access-psdx2\") pod \"authorino-67757b8bf-58t5v\" (UID: \"7c2323b5-0d28-4d00-a77a-b880a3920ffd\") " pod="kuadrant-system/authorino-67757b8bf-58t5v" Jul 13 09:23:32.474016 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:23:32.473943 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67757b8bf-58t5v" Jul 13 09:23:32.487709 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:23:32.487680 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:23:32.615786 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:23:32.615762 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-67757b8bf-58t5v"] Jul 13 09:23:32.618147 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:23:32.618118 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7c2323b5_0d28_4d00_a77a_b880a3920ffd.slice/crio-8f12819658f52671398006e1b357474675d5d29969a2587d85b07a61cd9f3edb WatchSource:0}: Error finding container 8f12819658f52671398006e1b357474675d5d29969a2587d85b07a61cd9f3edb: Status 404 returned error can't find the container with id 8f12819658f52671398006e1b357474675d5d29969a2587d85b07a61cd9f3edb Jul 13 09:23:32.698950 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:23:32.698920 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67757b8bf-58t5v" event={"ID":"7c2323b5-0d28-4d00-a77a-b880a3920ffd","Type":"ContainerStarted","Data":"8f12819658f52671398006e1b357474675d5d29969a2587d85b07a61cd9f3edb"} Jul 13 09:23:33.695372 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:23:33.695329 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:23:33.706313 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:23:33.706284 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67757b8bf-58t5v" event={"ID":"7c2323b5-0d28-4d00-a77a-b880a3920ffd","Type":"ContainerStarted","Data":"0c2f2a65e4363d913867699a224b5745bb77c749fbd7d61949ba531229a8e26b"} Jul 13 09:23:33.732374 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:23:33.732316 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-67757b8bf-58t5v" podStartSLOduration=1.142349152 podStartE2EDuration="1.732294637s" totalImagesPullingTime="589.945485ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:23:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:23:33.729523711 +0000 UTC m=+897.609478979" watchObservedRunningTime="2026-07-13 09:23:33.732294637 +0000 UTC m=+897.612249905" Jul 13 09:23:33.773900 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:23:33.773866 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-54d78f4d4-dwr5r"] Jul 13 09:23:33.774409 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:23:33.774354 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-54d78f4d4-dwr5r" podUID="941d7b1a-250c-4d0e-afb3-1fc2925c5b1f" containerName="authorino" containerID="cri-o://c7e785e91236227477817d0241b1c72a442910447113f1154a2f39ea1efe0681" gracePeriod=30 Jul 13 09:23:34.033718 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:23:34.033688 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54d78f4d4-dwr5r" Jul 13 09:23:34.152670 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:23:34.152632 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/941d7b1a-250c-4d0e-afb3-1fc2925c5b1f-tls-cert\") pod \"941d7b1a-250c-4d0e-afb3-1fc2925c5b1f\" (UID: \"941d7b1a-250c-4d0e-afb3-1fc2925c5b1f\") " Jul 13 09:23:34.152853 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:23:34.152676 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t2lzv\" (UniqueName: \"kubernetes.io/projected/941d7b1a-250c-4d0e-afb3-1fc2925c5b1f-kube-api-access-t2lzv\") pod \"941d7b1a-250c-4d0e-afb3-1fc2925c5b1f\" (UID: \"941d7b1a-250c-4d0e-afb3-1fc2925c5b1f\") " Jul 13 09:23:34.155278 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:23:34.155229 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/941d7b1a-250c-4d0e-afb3-1fc2925c5b1f-kube-api-access-t2lzv" (OuterVolumeSpecName: "kube-api-access-t2lzv") pod "941d7b1a-250c-4d0e-afb3-1fc2925c5b1f" (UID: "941d7b1a-250c-4d0e-afb3-1fc2925c5b1f"). InnerVolumeSpecName "kube-api-access-t2lzv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:23:34.167398 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:23:34.167364 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/941d7b1a-250c-4d0e-afb3-1fc2925c5b1f-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "941d7b1a-250c-4d0e-afb3-1fc2925c5b1f" (UID: "941d7b1a-250c-4d0e-afb3-1fc2925c5b1f"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 09:23:34.254457 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:23:34.254365 2552 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/941d7b1a-250c-4d0e-afb3-1fc2925c5b1f-tls-cert\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:23:34.254457 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:23:34.254413 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t2lzv\" (UniqueName: \"kubernetes.io/projected/941d7b1a-250c-4d0e-afb3-1fc2925c5b1f-kube-api-access-t2lzv\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:23:34.714178 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:23:34.714138 2552 generic.go:356] "Generic (PLEG): container finished" podID="941d7b1a-250c-4d0e-afb3-1fc2925c5b1f" containerID="c7e785e91236227477817d0241b1c72a442910447113f1154a2f39ea1efe0681" exitCode=0 Jul 13 09:23:34.714636 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:23:34.714183 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54d78f4d4-dwr5r" Jul 13 09:23:34.714636 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:23:34.714238 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54d78f4d4-dwr5r" event={"ID":"941d7b1a-250c-4d0e-afb3-1fc2925c5b1f","Type":"ContainerDied","Data":"c7e785e91236227477817d0241b1c72a442910447113f1154a2f39ea1efe0681"} Jul 13 09:23:34.714636 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:23:34.714283 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54d78f4d4-dwr5r" event={"ID":"941d7b1a-250c-4d0e-afb3-1fc2925c5b1f","Type":"ContainerDied","Data":"7c636ccc017ca5d405e960bd1c44aeaf78458c8a47c53c3f5c42eb9b56f3d1b0"} Jul 13 09:23:34.714636 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:23:34.714303 2552 scope.go:117] "RemoveContainer" containerID="c7e785e91236227477817d0241b1c72a442910447113f1154a2f39ea1efe0681" Jul 13 09:23:34.726637 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:23:34.726620 2552 scope.go:117] "RemoveContainer" containerID="c7e785e91236227477817d0241b1c72a442910447113f1154a2f39ea1efe0681" Jul 13 09:23:34.727093 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:23:34.727027 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7e785e91236227477817d0241b1c72a442910447113f1154a2f39ea1efe0681\": container with ID starting with c7e785e91236227477817d0241b1c72a442910447113f1154a2f39ea1efe0681 not found: ID does not exist" containerID="c7e785e91236227477817d0241b1c72a442910447113f1154a2f39ea1efe0681" Jul 13 09:23:34.727093 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:23:34.727061 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7e785e91236227477817d0241b1c72a442910447113f1154a2f39ea1efe0681"} err="failed to get container status \"c7e785e91236227477817d0241b1c72a442910447113f1154a2f39ea1efe0681\": rpc error: code = NotFound desc = could not find container \"c7e785e91236227477817d0241b1c72a442910447113f1154a2f39ea1efe0681\": container with ID starting with c7e785e91236227477817d0241b1c72a442910447113f1154a2f39ea1efe0681 not found: ID does not exist" Jul 13 09:23:34.735374 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:23:34.735352 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-54d78f4d4-dwr5r"] Jul 13 09:23:34.743455 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:23:34.743432 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-54d78f4d4-dwr5r"] Jul 13 09:23:36.609794 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:23:36.609767 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gp6mv_47250d22-56b6-4503-af3f-36a9e79691b2/ovn-acl-logging/0.log" Jul 13 09:23:36.611016 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:23:36.610992 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gp6mv_47250d22-56b6-4503-af3f-36a9e79691b2/ovn-acl-logging/0.log" Jul 13 09:23:36.665977 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:23:36.665949 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="941d7b1a-250c-4d0e-afb3-1fc2925c5b1f" path="/var/lib/kubelet/pods/941d7b1a-250c-4d0e-afb3-1fc2925c5b1f/volumes" Jul 13 09:24:19.456701 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:24:19.456663 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:24:42.173374 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:24:42.173345 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:24:42.273341 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:24:42.273312 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:24:43.791158 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:24:43.791131 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:24:45.545021 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:24:45.544989 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:24:59.952688 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:24:59.952617 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:25:00.053963 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:25:00.053934 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:25:10.633030 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:25:10.632994 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:25:10.732031 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:25:10.731989 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:25:25.380798 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:25:25.380764 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:25:25.463183 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:25:25.463149 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:25:25.591974 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:25:25.591946 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:25:31.407713 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:25:31.407660 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:25:31.519576 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:25:31.519539 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:25:31.608238 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:25:31.608154 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:25:31.786699 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:25:31.786586 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:25:37.902593 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:25:37.902535 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:25:38.111258 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:25:38.111198 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:26:59.553535 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:26:59.553502 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:26:59.654081 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:26:59.654027 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:28:36.645307 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:28:36.645274 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gp6mv_47250d22-56b6-4503-af3f-36a9e79691b2/ovn-acl-logging/0.log" Jul 13 09:28:36.648359 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:28:36.648339 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gp6mv_47250d22-56b6-4503-af3f-36a9e79691b2/ovn-acl-logging/0.log" Jul 13 09:29:35.184313 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:29:35.184198 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jwvn"] Jul 13 09:29:35.184845 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:29:35.184485 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jwvn" podUID="0f0046b1-ae5b-4205-97a0-23607bc45064" containerName="manager" containerID="cri-o://820988123c0554b54f01e53adeb4710505f0d3866d1cefb5a5664abf759013ca" gracePeriod=10 Jul 13 09:29:38.340078 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:29:38.340054 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jwvn" Jul 13 09:29:38.465243 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:29:38.465125 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f0046b1-ae5b-4205-97a0-23607bc45064-extensions-socket-volume\") pod \"0f0046b1-ae5b-4205-97a0-23607bc45064\" (UID: \"0f0046b1-ae5b-4205-97a0-23607bc45064\") " Jul 13 09:29:38.465398 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:29:38.465248 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vmpqn\" (UniqueName: \"kubernetes.io/projected/0f0046b1-ae5b-4205-97a0-23607bc45064-kube-api-access-vmpqn\") pod \"0f0046b1-ae5b-4205-97a0-23607bc45064\" (UID: \"0f0046b1-ae5b-4205-97a0-23607bc45064\") " Jul 13 09:29:38.465493 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:29:38.465468 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f0046b1-ae5b-4205-97a0-23607bc45064-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0f0046b1-ae5b-4205-97a0-23607bc45064" (UID: "0f0046b1-ae5b-4205-97a0-23607bc45064"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:29:38.465589 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:29:38.465572 2552 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f0046b1-ae5b-4205-97a0-23607bc45064-extensions-socket-volume\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:29:38.467396 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:29:38.467367 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f0046b1-ae5b-4205-97a0-23607bc45064-kube-api-access-vmpqn" (OuterVolumeSpecName: "kube-api-access-vmpqn") pod "0f0046b1-ae5b-4205-97a0-23607bc45064" (UID: "0f0046b1-ae5b-4205-97a0-23607bc45064"). InnerVolumeSpecName "kube-api-access-vmpqn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:29:38.566587 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:29:38.566564 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmpqn\" (UniqueName: \"kubernetes.io/projected/0f0046b1-ae5b-4205-97a0-23607bc45064-kube-api-access-vmpqn\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:29:38.768621 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:29:38.768553 2552 generic.go:356] "Generic (PLEG): container finished" podID="0f0046b1-ae5b-4205-97a0-23607bc45064" containerID="820988123c0554b54f01e53adeb4710505f0d3866d1cefb5a5664abf759013ca" exitCode=0 Jul 13 09:29:38.768621 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:29:38.768593 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jwvn" Jul 13 09:29:38.768758 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:29:38.768638 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jwvn" event={"ID":"0f0046b1-ae5b-4205-97a0-23607bc45064","Type":"ContainerDied","Data":"820988123c0554b54f01e53adeb4710505f0d3866d1cefb5a5664abf759013ca"} Jul 13 09:29:38.768758 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:29:38.768691 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jwvn" event={"ID":"0f0046b1-ae5b-4205-97a0-23607bc45064","Type":"ContainerDied","Data":"4ea464b2c528af6bdc7a182b80d2e429289ffb224f5252b71795853c8d4d5e37"} Jul 13 09:29:38.768758 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:29:38.768721 2552 scope.go:117] "RemoveContainer" containerID="820988123c0554b54f01e53adeb4710505f0d3866d1cefb5a5664abf759013ca" Jul 13 09:29:38.778700 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:29:38.778682 2552 scope.go:117] "RemoveContainer" containerID="820988123c0554b54f01e53adeb4710505f0d3866d1cefb5a5664abf759013ca" Jul 13 09:29:38.778953 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:29:38.778935 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"820988123c0554b54f01e53adeb4710505f0d3866d1cefb5a5664abf759013ca\": container with ID starting with 820988123c0554b54f01e53adeb4710505f0d3866d1cefb5a5664abf759013ca not found: ID does not exist" containerID="820988123c0554b54f01e53adeb4710505f0d3866d1cefb5a5664abf759013ca" Jul 13 09:29:38.779022 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:29:38.778966 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"820988123c0554b54f01e53adeb4710505f0d3866d1cefb5a5664abf759013ca"} err="failed to get container status \"820988123c0554b54f01e53adeb4710505f0d3866d1cefb5a5664abf759013ca\": rpc error: code = NotFound desc = could not find container \"820988123c0554b54f01e53adeb4710505f0d3866d1cefb5a5664abf759013ca\": container with ID starting with 820988123c0554b54f01e53adeb4710505f0d3866d1cefb5a5664abf759013ca not found: ID does not exist" Jul 13 09:29:38.788775 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:29:38.788751 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jwvn"] Jul 13 09:29:38.793800 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:29:38.793780 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jwvn"] Jul 13 09:29:40.666314 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:29:40.666277 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f0046b1-ae5b-4205-97a0-23607bc45064" path="/var/lib/kubelet/pods/0f0046b1-ae5b-4205-97a0-23607bc45064/volumes" Jul 13 09:30:00.153355 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:00.153316 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732250-sqdks"] Jul 13 09:30:00.153716 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:00.153705 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f0046b1-ae5b-4205-97a0-23607bc45064" containerName="manager" Jul 13 09:30:00.153758 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:00.153717 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f0046b1-ae5b-4205-97a0-23607bc45064" containerName="manager" Jul 13 09:30:00.153758 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:00.153745 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="941d7b1a-250c-4d0e-afb3-1fc2925c5b1f" containerName="authorino" Jul 13 09:30:00.153758 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:00.153751 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="941d7b1a-250c-4d0e-afb3-1fc2925c5b1f" containerName="authorino" Jul 13 09:30:00.153854 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:00.153812 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="941d7b1a-250c-4d0e-afb3-1fc2925c5b1f" containerName="authorino" Jul 13 09:30:00.153854 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:00.153825 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f0046b1-ae5b-4205-97a0-23607bc45064" containerName="manager" Jul 13 09:30:00.158446 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:00.158429 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732250-sqdks" Jul 13 09:30:00.161336 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:00.161312 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 13 09:30:00.162524 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:00.162311 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-f8w49\"" Jul 13 09:30:00.162524 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:00.162334 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 09:30:00.166114 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:00.165861 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732250-sqdks"] Jul 13 09:30:00.236257 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:00.236228 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxzml\" (UniqueName: \"kubernetes.io/projected/5471e70a-8fab-4056-91c9-c44c6926d3c8-kube-api-access-sxzml\") pod \"maas-api-key-cleanup-29732250-sqdks\" (UID: \"5471e70a-8fab-4056-91c9-c44c6926d3c8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732250-sqdks" Jul 13 09:30:00.337516 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:00.337492 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxzml\" (UniqueName: \"kubernetes.io/projected/5471e70a-8fab-4056-91c9-c44c6926d3c8-kube-api-access-sxzml\") pod \"maas-api-key-cleanup-29732250-sqdks\" (UID: \"5471e70a-8fab-4056-91c9-c44c6926d3c8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732250-sqdks" Jul 13 09:30:00.347689 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:00.347664 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxzml\" (UniqueName: \"kubernetes.io/projected/5471e70a-8fab-4056-91c9-c44c6926d3c8-kube-api-access-sxzml\") pod \"maas-api-key-cleanup-29732250-sqdks\" (UID: \"5471e70a-8fab-4056-91c9-c44c6926d3c8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732250-sqdks" Jul 13 09:30:00.469783 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:00.469714 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732250-sqdks" Jul 13 09:30:00.601628 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:00.601604 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732250-sqdks"] Jul 13 09:30:00.603336 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:30:00.603308 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5471e70a_8fab_4056_91c9_c44c6926d3c8.slice/crio-e1b62acf8a809a7e2e1ddad7c358169dd05d6e48b0dac8f801630c002619cdfc WatchSource:0}: Error finding container e1b62acf8a809a7e2e1ddad7c358169dd05d6e48b0dac8f801630c002619cdfc: Status 404 returned error can't find the container with id e1b62acf8a809a7e2e1ddad7c358169dd05d6e48b0dac8f801630c002619cdfc Jul 13 09:30:00.605336 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:00.605320 2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 09:30:00.901579 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:00.901552 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732250-sqdks" event={"ID":"5471e70a-8fab-4056-91c9-c44c6926d3c8","Type":"ContainerStarted","Data":"e1b62acf8a809a7e2e1ddad7c358169dd05d6e48b0dac8f801630c002619cdfc"} Jul 13 09:30:03.920803 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:03.920775 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732250-sqdks" event={"ID":"5471e70a-8fab-4056-91c9-c44c6926d3c8","Type":"ContainerStarted","Data":"393e77f0d29f5feb557e55130877489957a0092d604b7c4a9699004b3e41b571"} Jul 13 09:30:03.937915 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:03.937871 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732250-sqdks" podStartSLOduration=1.090798091 podStartE2EDuration="3.937851717s" totalImagesPullingTime="2.847053626s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:30:03.937513522 +0000 UTC m=+1287.817468800" watchObservedRunningTime="2026-07-13 09:30:03.937851717 +0000 UTC m=+1287.817806984" Jul 13 09:30:27.712362 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:27.712331 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:30:36.321306 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:36.321274 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ms4nv"] Jul 13 09:30:36.324731 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:36.324711 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ms4nv" Jul 13 09:30:36.327600 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:36.327578 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-qzhlj\"" Jul 13 09:30:36.337284 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:36.337263 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ms4nv"] Jul 13 09:30:36.428536 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:36.422722 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aaaf1412-55cc-4083-b4ea-d1b574a2a3b9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ms4nv\" (UID: \"aaaf1412-55cc-4083-b4ea-d1b574a2a3b9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ms4nv" Jul 13 09:30:36.428536 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:36.422892 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7fv6\" (UniqueName: \"kubernetes.io/projected/aaaf1412-55cc-4083-b4ea-d1b574a2a3b9-kube-api-access-v7fv6\") pod \"kuadrant-operator-controller-manager-86878c769f-ms4nv\" (UID: \"aaaf1412-55cc-4083-b4ea-d1b574a2a3b9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ms4nv" Jul 13 09:30:36.523780 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:36.523747 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7fv6\" (UniqueName: \"kubernetes.io/projected/aaaf1412-55cc-4083-b4ea-d1b574a2a3b9-kube-api-access-v7fv6\") pod \"kuadrant-operator-controller-manager-86878c769f-ms4nv\" (UID: \"aaaf1412-55cc-4083-b4ea-d1b574a2a3b9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ms4nv" Jul 13 09:30:36.523938 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:36.523793 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aaaf1412-55cc-4083-b4ea-d1b574a2a3b9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ms4nv\" (UID: \"aaaf1412-55cc-4083-b4ea-d1b574a2a3b9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ms4nv" Jul 13 09:30:36.524199 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:36.524181 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aaaf1412-55cc-4083-b4ea-d1b574a2a3b9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ms4nv\" (UID: \"aaaf1412-55cc-4083-b4ea-d1b574a2a3b9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ms4nv" Jul 13 09:30:36.532920 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:36.532885 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7fv6\" (UniqueName: \"kubernetes.io/projected/aaaf1412-55cc-4083-b4ea-d1b574a2a3b9-kube-api-access-v7fv6\") pod \"kuadrant-operator-controller-manager-86878c769f-ms4nv\" (UID: \"aaaf1412-55cc-4083-b4ea-d1b574a2a3b9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ms4nv" Jul 13 09:30:36.637806 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:36.637781 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-qzhlj\"" Jul 13 09:30:36.646129 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:36.646109 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ms4nv" Jul 13 09:30:37.005297 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:37.005273 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ms4nv"] Jul 13 09:30:37.007110 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:30:37.007070 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaaf1412_55cc_4083_b4ea_d1b574a2a3b9.slice/crio-9079c3437f9cc1b2efc62b250a3031e7a0a8f32fd9e36ec52875a8334c774a37 WatchSource:0}: Error finding container 9079c3437f9cc1b2efc62b250a3031e7a0a8f32fd9e36ec52875a8334c774a37: Status 404 returned error can't find the container with id 9079c3437f9cc1b2efc62b250a3031e7a0a8f32fd9e36ec52875a8334c774a37 Jul 13 09:30:37.111983 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:37.111953 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ms4nv" event={"ID":"aaaf1412-55cc-4083-b4ea-d1b574a2a3b9","Type":"ContainerStarted","Data":"e8bf2560dc8b80b30a995f5c4a1466bad49dd3a8c4bce1865a818704ff60d0f2"} Jul 13 09:30:37.112129 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:37.112010 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ms4nv" event={"ID":"aaaf1412-55cc-4083-b4ea-d1b574a2a3b9","Type":"ContainerStarted","Data":"9079c3437f9cc1b2efc62b250a3031e7a0a8f32fd9e36ec52875a8334c774a37"} Jul 13 09:30:37.112129 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:37.112066 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ms4nv" Jul 13 09:30:37.145430 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:37.145385 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ms4nv" podStartSLOduration=1.14537069 podStartE2EDuration="1.14537069s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:30:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:30:37.143250625 +0000 UTC m=+1321.023205894" watchObservedRunningTime="2026-07-13 09:30:37.14537069 +0000 UTC m=+1321.025325956" Jul 13 09:30:48.119800 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:48.119767 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ms4nv" Jul 13 09:30:48.291188 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:30:48.291157 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:31:00.012717 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:31:00.012641 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732250-sqdks"] Jul 13 09:31:00.013233 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:31:00.012836 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732250-sqdks" podUID="5471e70a-8fab-4056-91c9-c44c6926d3c8" containerName="cleanup" containerID="cri-o://393e77f0d29f5feb557e55130877489957a0092d604b7c4a9699004b3e41b571" gracePeriod=30 Jul 13 09:31:30.209019 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:31:30.208989 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732250-sqdks" Jul 13 09:31:30.268601 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:31:30.268522 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sxzml\" (UniqueName: \"kubernetes.io/projected/5471e70a-8fab-4056-91c9-c44c6926d3c8-kube-api-access-sxzml\") pod \"5471e70a-8fab-4056-91c9-c44c6926d3c8\" (UID: \"5471e70a-8fab-4056-91c9-c44c6926d3c8\") " Jul 13 09:31:30.270657 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:31:30.270630 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5471e70a-8fab-4056-91c9-c44c6926d3c8-kube-api-access-sxzml" (OuterVolumeSpecName: "kube-api-access-sxzml") pod "5471e70a-8fab-4056-91c9-c44c6926d3c8" (UID: "5471e70a-8fab-4056-91c9-c44c6926d3c8"). InnerVolumeSpecName "kube-api-access-sxzml". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:31:30.370033 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:31:30.370001 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxzml\" (UniqueName: \"kubernetes.io/projected/5471e70a-8fab-4056-91c9-c44c6926d3c8-kube-api-access-sxzml\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:31:30.416166 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:31:30.416133 2552 generic.go:356] "Generic (PLEG): container finished" podID="5471e70a-8fab-4056-91c9-c44c6926d3c8" containerID="393e77f0d29f5feb557e55130877489957a0092d604b7c4a9699004b3e41b571" exitCode=137 Jul 13 09:31:30.416299 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:31:30.416181 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732250-sqdks" Jul 13 09:31:30.416299 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:31:30.416195 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732250-sqdks" event={"ID":"5471e70a-8fab-4056-91c9-c44c6926d3c8","Type":"ContainerDied","Data":"393e77f0d29f5feb557e55130877489957a0092d604b7c4a9699004b3e41b571"} Jul 13 09:31:30.416299 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:31:30.416296 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732250-sqdks" event={"ID":"5471e70a-8fab-4056-91c9-c44c6926d3c8","Type":"ContainerDied","Data":"e1b62acf8a809a7e2e1ddad7c358169dd05d6e48b0dac8f801630c002619cdfc"} Jul 13 09:31:30.416421 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:31:30.416320 2552 scope.go:117] "RemoveContainer" containerID="393e77f0d29f5feb557e55130877489957a0092d604b7c4a9699004b3e41b571" Jul 13 09:31:30.427533 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:31:30.427483 2552 scope.go:117] "RemoveContainer" containerID="393e77f0d29f5feb557e55130877489957a0092d604b7c4a9699004b3e41b571" Jul 13 09:31:30.427945 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:31:30.427924 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"393e77f0d29f5feb557e55130877489957a0092d604b7c4a9699004b3e41b571\": container with ID starting with 393e77f0d29f5feb557e55130877489957a0092d604b7c4a9699004b3e41b571 not found: ID does not exist" containerID="393e77f0d29f5feb557e55130877489957a0092d604b7c4a9699004b3e41b571" Jul 13 09:31:30.428047 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:31:30.427949 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"393e77f0d29f5feb557e55130877489957a0092d604b7c4a9699004b3e41b571"} err="failed to get container status \"393e77f0d29f5feb557e55130877489957a0092d604b7c4a9699004b3e41b571\": rpc error: code = NotFound desc = could not find container \"393e77f0d29f5feb557e55130877489957a0092d604b7c4a9699004b3e41b571\": container with ID starting with 393e77f0d29f5feb557e55130877489957a0092d604b7c4a9699004b3e41b571 not found: ID does not exist" Jul 13 09:31:30.441102 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:31:30.441073 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732250-sqdks"] Jul 13 09:31:30.445040 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:31:30.445020 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732250-sqdks"] Jul 13 09:31:30.667366 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:31:30.667337 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5471e70a-8fab-4056-91c9-c44c6926d3c8" path="/var/lib/kubelet/pods/5471e70a-8fab-4056-91c9-c44c6926d3c8/volumes" Jul 13 09:31:49.448031 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:31:49.447995 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:33:36.682176 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:33:36.682146 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gp6mv_47250d22-56b6-4503-af3f-36a9e79691b2/ovn-acl-logging/0.log" Jul 13 09:33:36.687335 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:33:36.687315 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gp6mv_47250d22-56b6-4503-af3f-36a9e79691b2/ovn-acl-logging/0.log" Jul 13 09:36:31.497303 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:31.497256 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:36:32.886350 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:32.886318 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:36:36.900970 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:36.900898 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:36:36.970163 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:36.970129 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:36:37.090305 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.090282 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp"] Jul 13 09:36:37.090664 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.090652 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5471e70a-8fab-4056-91c9-c44c6926d3c8" containerName="cleanup" Jul 13 09:36:37.090714 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.090665 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="5471e70a-8fab-4056-91c9-c44c6926d3c8" containerName="cleanup" Jul 13 09:36:37.090756 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.090749 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="5471e70a-8fab-4056-91c9-c44c6926d3c8" containerName="cleanup" Jul 13 09:36:37.093856 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.093838 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" Jul 13 09:36:37.096654 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.096624 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-f09dcdea-openshift-default-dockercfg-qq7r4\"" Jul 13 09:36:37.116006 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.115981 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp"] Jul 13 09:36:37.137947 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.137923 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:36:37.181724 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.181658 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-workload-socket\") pod \"e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp\" (UID: \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\") " pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" Jul 13 09:36:37.181724 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.181702 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-istio-envoy\") pod \"e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp\" (UID: \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\") " pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" Jul 13 09:36:37.181874 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.181723 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-istiod-ca-cert\") pod \"e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp\" (UID: \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\") " pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" Jul 13 09:36:37.181874 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.181773 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-istio-podinfo\") pod \"e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp\" (UID: \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\") " pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" Jul 13 09:36:37.181874 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.181823 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-credential-socket\") pod \"e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp\" (UID: \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\") " pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" Jul 13 09:36:37.181874 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.181853 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-istio-data\") pod \"e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp\" (UID: \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\") " pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" Jul 13 09:36:37.182266 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.181919 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-workload-certs\") pod \"e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp\" (UID: \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\") " pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" Jul 13 09:36:37.182266 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.181949 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-istio-token\") pod \"e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp\" (UID: \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\") " pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" Jul 13 09:36:37.182266 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.181972 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24xtv\" (UniqueName: \"kubernetes.io/projected/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-kube-api-access-24xtv\") pod \"e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp\" (UID: \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\") " pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" Jul 13 09:36:37.208341 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.208311 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:36:37.283219 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.283189 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-credential-socket\") pod \"e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp\" (UID: \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\") " pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" Jul 13 09:36:37.283356 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.283253 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-istio-data\") pod \"e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp\" (UID: \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\") " pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" Jul 13 09:36:37.283356 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.283282 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-workload-certs\") pod \"e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp\" (UID: \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\") " pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" Jul 13 09:36:37.283356 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.283296 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-istio-token\") pod \"e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp\" (UID: \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\") " pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" Jul 13 09:36:37.283356 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.283311 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24xtv\" (UniqueName: \"kubernetes.io/projected/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-kube-api-access-24xtv\") pod \"e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp\" (UID: \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\") " pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" Jul 13 09:36:37.283588 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.283382 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-workload-socket\") pod \"e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp\" (UID: \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\") " pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" Jul 13 09:36:37.283588 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.283420 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-istio-envoy\") pod \"e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp\" (UID: \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\") " pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" Jul 13 09:36:37.283588 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.283441 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-istiod-ca-cert\") pod \"e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp\" (UID: \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\") " pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" Jul 13 09:36:37.283588 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.283466 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-istio-podinfo\") pod \"e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp\" (UID: \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\") " pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" Jul 13 09:36:37.283794 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.283647 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-credential-socket\") pod \"e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp\" (UID: \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\") " pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" Jul 13 09:36:37.283794 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.283745 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-workload-certs\") pod \"e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp\" (UID: \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\") " pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" Jul 13 09:36:37.283889 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.283818 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-istio-data\") pod \"e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp\" (UID: \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\") " pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" Jul 13 09:36:37.283889 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.283852 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-workload-socket\") pod \"e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp\" (UID: \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\") " pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" Jul 13 09:36:37.284180 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.284158 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-istiod-ca-cert\") pod \"e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp\" (UID: \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\") " pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" Jul 13 09:36:37.285790 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.285771 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-istio-envoy\") pod \"e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp\" (UID: \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\") " pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" Jul 13 09:36:37.285850 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.285832 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-istio-podinfo\") pod \"e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp\" (UID: \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\") " pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" Jul 13 09:36:37.292229 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.292193 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-istio-token\") pod \"e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp\" (UID: \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\") " pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" Jul 13 09:36:37.292511 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.292495 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24xtv\" (UniqueName: \"kubernetes.io/projected/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-kube-api-access-24xtv\") pod \"e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp\" (UID: \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\") " pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" Jul 13 09:36:37.404735 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.404712 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" Jul 13 09:36:37.545401 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.545375 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp"] Jul 13 09:36:37.546632 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:36:37.546593 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58c1e08d_dbb3_4a76_8efe_c5291a22dc3a.slice/crio-2f4a77acf673525982d0e5f0f1aa6539b962bc6a4f236a58ceccd15f51acf639 WatchSource:0}: Error finding container 2f4a77acf673525982d0e5f0f1aa6539b962bc6a4f236a58ceccd15f51acf639: Status 404 returned error can't find the container with id 2f4a77acf673525982d0e5f0f1aa6539b962bc6a4f236a58ceccd15f51acf639 Jul 13 09:36:37.551890 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.551860 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:36:37.551966 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.551926 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:36:37.551966 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:37.551952 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:36:38.160631 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:38.160597 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" event={"ID":"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a","Type":"ContainerStarted","Data":"d5da915a18a27d7f0ff419f6bd26e1657bbdecee59ddd3591f54d08ca8703c06"} Jul 13 09:36:38.161096 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:38.160647 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" event={"ID":"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a","Type":"ContainerStarted","Data":"2f4a77acf673525982d0e5f0f1aa6539b962bc6a4f236a58ceccd15f51acf639"} Jul 13 09:36:38.186458 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:38.186407 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" podStartSLOduration=1.186392551 podStartE2EDuration="1.186392551s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:36:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:36:38.183431996 +0000 UTC m=+1682.063387264" watchObservedRunningTime="2026-07-13 09:36:38.186392551 +0000 UTC m=+1682.066347817" Jul 13 09:36:38.294594 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:38.294567 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:36:38.397563 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:38.397537 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:36:38.405972 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:38.405947 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" Jul 13 09:36:38.415829 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:38.415749 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" Jul 13 09:36:38.608850 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:38.608820 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:36:38.844042 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:38.844016 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:36:39.175788 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:39.175691 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" Jul 13 09:36:39.176559 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:39.176542 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" Jul 13 09:36:39.179162 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:39.179140 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:36:39.283309 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:39.283272 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:36:41.062265 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:41.062238 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:36:43.340271 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:43.340236 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:36:43.506255 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:43.506220 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp"] Jul 13 09:36:43.506431 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:43.506391 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" podUID="58c1e08d-dbb3-4a76-8efe-c5291a22dc3a" containerName="istio-proxy" containerID="cri-o://d5da915a18a27d7f0ff419f6bd26e1657bbdecee59ddd3591f54d08ca8703c06" gracePeriod=30 Jul 13 09:36:43.539552 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:43.539526 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:36:43.653570 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:43.653483 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:36:43.717335 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:43.717307 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:36:45.016970 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:45.016930 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:36:45.535548 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:45.535517 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:36:48.569581 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:48.569524 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" Jul 13 09:36:48.685974 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:48.685946 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-workload-socket\") pod \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\" (UID: \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\") " Jul 13 09:36:48.686116 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:48.685990 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-istiod-ca-cert\") pod \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\" (UID: \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\") " Jul 13 09:36:48.686116 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:48.686047 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-istio-podinfo\") pod \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\" (UID: \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\") " Jul 13 09:36:48.686116 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:48.686072 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-istio-data\") pod \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\" (UID: \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\") " Jul 13 09:36:48.686116 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:48.686088 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-24xtv\" (UniqueName: \"kubernetes.io/projected/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-kube-api-access-24xtv\") pod \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\" (UID: \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\") " Jul 13 09:36:48.686369 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:48.686158 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-istio-envoy\") pod \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\" (UID: \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\") " Jul 13 09:36:48.686369 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:48.686204 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-credential-socket\") pod \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\" (UID: \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\") " Jul 13 09:36:48.686369 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:48.686260 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-istio-token\") pod \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\" (UID: \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\") " Jul 13 09:36:48.686369 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:48.686294 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-workload-certs\") pod \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\" (UID: \"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a\") " Jul 13 09:36:48.686369 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:48.686226 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "58c1e08d-dbb3-4a76-8efe-c5291a22dc3a" (UID: "58c1e08d-dbb3-4a76-8efe-c5291a22dc3a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:36:48.686642 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:48.686469 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "58c1e08d-dbb3-4a76-8efe-c5291a22dc3a" (UID: "58c1e08d-dbb3-4a76-8efe-c5291a22dc3a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:36:48.686707 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:48.686655 2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-credential-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:36:48.686707 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:48.686667 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "58c1e08d-dbb3-4a76-8efe-c5291a22dc3a" (UID: "58c1e08d-dbb3-4a76-8efe-c5291a22dc3a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:36:48.686707 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:48.686675 2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-workload-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:36:48.686862 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:48.686793 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "58c1e08d-dbb3-4a76-8efe-c5291a22dc3a" (UID: "58c1e08d-dbb3-4a76-8efe-c5291a22dc3a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 09:36:48.686999 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:48.686973 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-istio-data" (OuterVolumeSpecName: "istio-data") pod "58c1e08d-dbb3-4a76-8efe-c5291a22dc3a" (UID: "58c1e08d-dbb3-4a76-8efe-c5291a22dc3a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:36:48.688419 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:48.688388 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "58c1e08d-dbb3-4a76-8efe-c5291a22dc3a" (UID: "58c1e08d-dbb3-4a76-8efe-c5291a22dc3a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:36:48.688419 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:48.688397 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-istio-token" (OuterVolumeSpecName: "istio-token") pod "58c1e08d-dbb3-4a76-8efe-c5291a22dc3a" (UID: "58c1e08d-dbb3-4a76-8efe-c5291a22dc3a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:36:48.688718 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:48.688694 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "58c1e08d-dbb3-4a76-8efe-c5291a22dc3a" (UID: "58c1e08d-dbb3-4a76-8efe-c5291a22dc3a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 09:36:48.688718 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:48.688710 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-kube-api-access-24xtv" (OuterVolumeSpecName: "kube-api-access-24xtv") pod "58c1e08d-dbb3-4a76-8efe-c5291a22dc3a" (UID: "58c1e08d-dbb3-4a76-8efe-c5291a22dc3a"). InnerVolumeSpecName "kube-api-access-24xtv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:36:48.788082 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:48.788060 2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-istio-podinfo\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:36:48.788082 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:48.788082 2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-istio-data\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:36:48.788251 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:48.788091 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24xtv\" (UniqueName: \"kubernetes.io/projected/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-kube-api-access-24xtv\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:36:48.788251 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:48.788099 2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-istio-envoy\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:36:48.788251 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:48.788108 2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-istio-token\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:36:48.788251 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:48.788116 2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-workload-certs\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:36:48.788251 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:48.788124 2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a-istiod-ca-cert\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:36:49.235180 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:49.235141 2552 generic.go:356] "Generic (PLEG): container finished" podID="58c1e08d-dbb3-4a76-8efe-c5291a22dc3a" containerID="d5da915a18a27d7f0ff419f6bd26e1657bbdecee59ddd3591f54d08ca8703c06" exitCode=0 Jul 13 09:36:49.235387 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:49.235187 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" Jul 13 09:36:49.235387 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:49.235237 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" event={"ID":"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a","Type":"ContainerDied","Data":"d5da915a18a27d7f0ff419f6bd26e1657bbdecee59ddd3591f54d08ca8703c06"} Jul 13 09:36:49.235387 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:49.235293 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp" event={"ID":"58c1e08d-dbb3-4a76-8efe-c5291a22dc3a","Type":"ContainerDied","Data":"2f4a77acf673525982d0e5f0f1aa6539b962bc6a4f236a58ceccd15f51acf639"} Jul 13 09:36:49.235387 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:49.235316 2552 scope.go:117] "RemoveContainer" containerID="d5da915a18a27d7f0ff419f6bd26e1657bbdecee59ddd3591f54d08ca8703c06" Jul 13 09:36:49.245583 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:49.245567 2552 scope.go:117] "RemoveContainer" containerID="d5da915a18a27d7f0ff419f6bd26e1657bbdecee59ddd3591f54d08ca8703c06" Jul 13 09:36:49.245863 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:36:49.245845 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5da915a18a27d7f0ff419f6bd26e1657bbdecee59ddd3591f54d08ca8703c06\": container with ID starting with d5da915a18a27d7f0ff419f6bd26e1657bbdecee59ddd3591f54d08ca8703c06 not found: ID does not exist" containerID="d5da915a18a27d7f0ff419f6bd26e1657bbdecee59ddd3591f54d08ca8703c06" Jul 13 09:36:49.245919 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:49.245871 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5da915a18a27d7f0ff419f6bd26e1657bbdecee59ddd3591f54d08ca8703c06"} err="failed to get container status \"d5da915a18a27d7f0ff419f6bd26e1657bbdecee59ddd3591f54d08ca8703c06\": rpc error: code = NotFound desc = could not find container \"d5da915a18a27d7f0ff419f6bd26e1657bbdecee59ddd3591f54d08ca8703c06\": container with ID starting with d5da915a18a27d7f0ff419f6bd26e1657bbdecee59ddd3591f54d08ca8703c06 not found: ID does not exist" Jul 13 09:36:49.259755 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:49.259730 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp"] Jul 13 09:36:49.265082 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:49.265060 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-f09dcdea-openshift-default-7dc879d6b7-m9zrp"] Jul 13 09:36:49.670250 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:49.670204 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:36:49.743524 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:49.743496 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:36:49.863034 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:49.863004 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx"] Jul 13 09:36:49.863679 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:49.863654 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58c1e08d-dbb3-4a76-8efe-c5291a22dc3a" containerName="istio-proxy" Jul 13 09:36:49.863679 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:49.863678 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="58c1e08d-dbb3-4a76-8efe-c5291a22dc3a" containerName="istio-proxy" Jul 13 09:36:49.863868 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:49.863780 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="58c1e08d-dbb3-4a76-8efe-c5291a22dc3a" containerName="istio-proxy" Jul 13 09:36:49.868113 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:49.868093 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" Jul 13 09:36:49.871941 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:49.871922 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-f7615a16-openshift-default-dockercfg-xk62n\"" Jul 13 09:36:49.884830 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:49.884808 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx"] Jul 13 09:36:49.893912 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:49.893892 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:36:49.994272 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:49.994188 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:36:49.997375 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:49.997355 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20b674ae-6b2a-4084-b746-b5b1c3638e73-istiod-ca-cert\") pod \"e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx\" (UID: \"20b674ae-6b2a-4084-b746-b5b1c3638e73\") " pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" Jul 13 09:36:49.997441 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:49.997410 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20b674ae-6b2a-4084-b746-b5b1c3638e73-istio-token\") pod \"e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx\" (UID: \"20b674ae-6b2a-4084-b746-b5b1c3638e73\") " pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" Jul 13 09:36:49.997484 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:49.997447 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20b674ae-6b2a-4084-b746-b5b1c3638e73-workload-socket\") pod \"e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx\" (UID: \"20b674ae-6b2a-4084-b746-b5b1c3638e73\") " pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" Jul 13 09:36:49.997484 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:49.997474 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20b674ae-6b2a-4084-b746-b5b1c3638e73-credential-socket\") pod \"e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx\" (UID: \"20b674ae-6b2a-4084-b746-b5b1c3638e73\") " pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" Jul 13 09:36:49.997547 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:49.997517 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20b674ae-6b2a-4084-b746-b5b1c3638e73-workload-certs\") pod \"e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx\" (UID: \"20b674ae-6b2a-4084-b746-b5b1c3638e73\") " pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" Jul 13 09:36:49.997547 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:49.997534 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20b674ae-6b2a-4084-b746-b5b1c3638e73-istio-envoy\") pod \"e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx\" (UID: \"20b674ae-6b2a-4084-b746-b5b1c3638e73\") " pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" Jul 13 09:36:49.997611 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:49.997555 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9m5lm\" (UniqueName: \"kubernetes.io/projected/20b674ae-6b2a-4084-b746-b5b1c3638e73-kube-api-access-9m5lm\") pod \"e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx\" (UID: \"20b674ae-6b2a-4084-b746-b5b1c3638e73\") " pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" Jul 13 09:36:49.997648 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:49.997613 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20b674ae-6b2a-4084-b746-b5b1c3638e73-istio-data\") pod \"e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx\" (UID: \"20b674ae-6b2a-4084-b746-b5b1c3638e73\") " pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" Jul 13 09:36:49.997648 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:49.997633 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20b674ae-6b2a-4084-b746-b5b1c3638e73-istio-podinfo\") pod \"e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx\" (UID: \"20b674ae-6b2a-4084-b746-b5b1c3638e73\") " pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" Jul 13 09:36:50.098718 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:50.098691 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20b674ae-6b2a-4084-b746-b5b1c3638e73-istiod-ca-cert\") pod \"e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx\" (UID: \"20b674ae-6b2a-4084-b746-b5b1c3638e73\") " pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" Jul 13 09:36:50.098848 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:50.098735 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20b674ae-6b2a-4084-b746-b5b1c3638e73-istio-token\") pod \"e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx\" (UID: \"20b674ae-6b2a-4084-b746-b5b1c3638e73\") " pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" Jul 13 09:36:50.098848 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:50.098777 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20b674ae-6b2a-4084-b746-b5b1c3638e73-workload-socket\") pod \"e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx\" (UID: \"20b674ae-6b2a-4084-b746-b5b1c3638e73\") " pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" Jul 13 09:36:50.098848 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:50.098808 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20b674ae-6b2a-4084-b746-b5b1c3638e73-credential-socket\") pod \"e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx\" (UID: \"20b674ae-6b2a-4084-b746-b5b1c3638e73\") " pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" Jul 13 09:36:50.098848 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:50.098842 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20b674ae-6b2a-4084-b746-b5b1c3638e73-workload-certs\") pod \"e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx\" (UID: \"20b674ae-6b2a-4084-b746-b5b1c3638e73\") " pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" Jul 13 09:36:50.098995 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:50.098864 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20b674ae-6b2a-4084-b746-b5b1c3638e73-istio-envoy\") pod \"e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx\" (UID: \"20b674ae-6b2a-4084-b746-b5b1c3638e73\") " pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" Jul 13 09:36:50.098995 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:50.098892 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9m5lm\" (UniqueName: \"kubernetes.io/projected/20b674ae-6b2a-4084-b746-b5b1c3638e73-kube-api-access-9m5lm\") pod \"e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx\" (UID: \"20b674ae-6b2a-4084-b746-b5b1c3638e73\") " pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" Jul 13 09:36:50.098995 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:50.098958 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20b674ae-6b2a-4084-b746-b5b1c3638e73-istio-data\") pod \"e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx\" (UID: \"20b674ae-6b2a-4084-b746-b5b1c3638e73\") " pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" Jul 13 09:36:50.098995 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:50.098987 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20b674ae-6b2a-4084-b746-b5b1c3638e73-istio-podinfo\") pod \"e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx\" (UID: \"20b674ae-6b2a-4084-b746-b5b1c3638e73\") " pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" Jul 13 09:36:50.099326 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:50.099297 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20b674ae-6b2a-4084-b746-b5b1c3638e73-workload-socket\") pod \"e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx\" (UID: \"20b674ae-6b2a-4084-b746-b5b1c3638e73\") " pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" Jul 13 09:36:50.099442 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:50.099384 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20b674ae-6b2a-4084-b746-b5b1c3638e73-credential-socket\") pod \"e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx\" (UID: \"20b674ae-6b2a-4084-b746-b5b1c3638e73\") " pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" Jul 13 09:36:50.099442 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:50.099414 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20b674ae-6b2a-4084-b746-b5b1c3638e73-workload-certs\") pod \"e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx\" (UID: \"20b674ae-6b2a-4084-b746-b5b1c3638e73\") " pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" Jul 13 09:36:50.099442 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:50.099431 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20b674ae-6b2a-4084-b746-b5b1c3638e73-istio-data\") pod \"e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx\" (UID: \"20b674ae-6b2a-4084-b746-b5b1c3638e73\") " pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" Jul 13 09:36:50.099590 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:50.099572 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20b674ae-6b2a-4084-b746-b5b1c3638e73-istiod-ca-cert\") pod \"e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx\" (UID: \"20b674ae-6b2a-4084-b746-b5b1c3638e73\") " pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" Jul 13 09:36:50.101470 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:50.101440 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20b674ae-6b2a-4084-b746-b5b1c3638e73-istio-envoy\") pod \"e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx\" (UID: \"20b674ae-6b2a-4084-b746-b5b1c3638e73\") " pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" Jul 13 09:36:50.101632 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:50.101617 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20b674ae-6b2a-4084-b746-b5b1c3638e73-istio-podinfo\") pod \"e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx\" (UID: \"20b674ae-6b2a-4084-b746-b5b1c3638e73\") " pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" Jul 13 09:36:50.114853 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:50.114828 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20b674ae-6b2a-4084-b746-b5b1c3638e73-istio-token\") pod \"e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx\" (UID: \"20b674ae-6b2a-4084-b746-b5b1c3638e73\") " pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" Jul 13 09:36:50.115045 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:50.115025 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9m5lm\" (UniqueName: \"kubernetes.io/projected/20b674ae-6b2a-4084-b746-b5b1c3638e73-kube-api-access-9m5lm\") pod \"e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx\" (UID: \"20b674ae-6b2a-4084-b746-b5b1c3638e73\") " pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" Jul 13 09:36:50.179202 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:50.179177 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" Jul 13 09:36:50.326704 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:50.326674 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx"] Jul 13 09:36:50.327864 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:36:50.327838 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20b674ae_6b2a_4084_b746_b5b1c3638e73.slice/crio-f33b002fa06874756b02e6a56aeeaf615937af41e02c5ee0d22394be5464801e WatchSource:0}: Error finding container f33b002fa06874756b02e6a56aeeaf615937af41e02c5ee0d22394be5464801e: Status 404 returned error can't find the container with id f33b002fa06874756b02e6a56aeeaf615937af41e02c5ee0d22394be5464801e Jul 13 09:36:50.329990 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:50.329960 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:36:50.330120 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:50.330020 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:36:50.330120 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:50.330047 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:36:50.667093 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:50.667058 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58c1e08d-dbb3-4a76-8efe-c5291a22dc3a" path="/var/lib/kubelet/pods/58c1e08d-dbb3-4a76-8efe-c5291a22dc3a/volumes" Jul 13 09:36:50.991104 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:50.991008 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:36:51.093205 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:51.093172 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:36:51.263284 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:51.263183 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" event={"ID":"20b674ae-6b2a-4084-b746-b5b1c3638e73","Type":"ContainerStarted","Data":"eeb0f2b702d81daab5883c4c04dd43d416a4f5729cbf6ec4c03473ad1e8f6123"} Jul 13 09:36:51.263284 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:51.263261 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" event={"ID":"20b674ae-6b2a-4084-b746-b5b1c3638e73","Type":"ContainerStarted","Data":"f33b002fa06874756b02e6a56aeeaf615937af41e02c5ee0d22394be5464801e"} Jul 13 09:36:51.292162 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:51.292108 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" podStartSLOduration=2.292086369 podStartE2EDuration="2.292086369s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:36:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:36:51.290229879 +0000 UTC m=+1695.170185147" watchObservedRunningTime="2026-07-13 09:36:51.292086369 +0000 UTC m=+1695.172041640" Jul 13 09:36:51.303910 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:51.303878 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:36:51.751944 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:51.751900 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:36:51.850144 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:51.850116 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:36:51.951555 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:51.951513 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:36:52.179376 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:52.179343 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" Jul 13 09:36:52.184239 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:52.184192 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" Jul 13 09:36:52.269892 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:52.269857 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" Jul 13 09:36:52.270785 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:52.270763 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" Jul 13 09:36:53.083351 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:53.083323 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:36:56.096623 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:56.096585 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:36:56.516011 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:36:56.515922 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:37:02.166514 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.166447 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx"] Jul 13 09:37:02.166930 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.166661 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" podUID="20b674ae-6b2a-4084-b746-b5b1c3638e73" containerName="istio-proxy" containerID="cri-o://eeb0f2b702d81daab5883c4c04dd43d416a4f5729cbf6ec4c03473ad1e8f6123" gracePeriod=30 Jul 13 09:37:02.195765 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.195737 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:37:02.299423 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.299371 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:37:02.321328 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.321306 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" Jul 13 09:37:02.328973 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.328947 2552 generic.go:356] "Generic (PLEG): container finished" podID="20b674ae-6b2a-4084-b746-b5b1c3638e73" containerID="eeb0f2b702d81daab5883c4c04dd43d416a4f5729cbf6ec4c03473ad1e8f6123" exitCode=0 Jul 13 09:37:02.329075 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.328982 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" Jul 13 09:37:02.329075 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.328991 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" event={"ID":"20b674ae-6b2a-4084-b746-b5b1c3638e73","Type":"ContainerDied","Data":"eeb0f2b702d81daab5883c4c04dd43d416a4f5729cbf6ec4c03473ad1e8f6123"} Jul 13 09:37:02.329075 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.329043 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx" event={"ID":"20b674ae-6b2a-4084-b746-b5b1c3638e73","Type":"ContainerDied","Data":"f33b002fa06874756b02e6a56aeeaf615937af41e02c5ee0d22394be5464801e"} Jul 13 09:37:02.329075 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.329067 2552 scope.go:117] "RemoveContainer" containerID="eeb0f2b702d81daab5883c4c04dd43d416a4f5729cbf6ec4c03473ad1e8f6123" Jul 13 09:37:02.337608 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.337593 2552 scope.go:117] "RemoveContainer" containerID="eeb0f2b702d81daab5883c4c04dd43d416a4f5729cbf6ec4c03473ad1e8f6123" Jul 13 09:37:02.337843 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:37:02.337823 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eeb0f2b702d81daab5883c4c04dd43d416a4f5729cbf6ec4c03473ad1e8f6123\": container with ID starting with eeb0f2b702d81daab5883c4c04dd43d416a4f5729cbf6ec4c03473ad1e8f6123 not found: ID does not exist" containerID="eeb0f2b702d81daab5883c4c04dd43d416a4f5729cbf6ec4c03473ad1e8f6123" Jul 13 09:37:02.337917 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.337856 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eeb0f2b702d81daab5883c4c04dd43d416a4f5729cbf6ec4c03473ad1e8f6123"} err="failed to get container status \"eeb0f2b702d81daab5883c4c04dd43d416a4f5729cbf6ec4c03473ad1e8f6123\": rpc error: code = NotFound desc = could not find container \"eeb0f2b702d81daab5883c4c04dd43d416a4f5729cbf6ec4c03473ad1e8f6123\": container with ID starting with eeb0f2b702d81daab5883c4c04dd43d416a4f5729cbf6ec4c03473ad1e8f6123 not found: ID does not exist" Jul 13 09:37:02.409816 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.409794 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20b674ae-6b2a-4084-b746-b5b1c3638e73-istiod-ca-cert\") pod \"20b674ae-6b2a-4084-b746-b5b1c3638e73\" (UID: \"20b674ae-6b2a-4084-b746-b5b1c3638e73\") " Jul 13 09:37:02.409938 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.409857 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20b674ae-6b2a-4084-b746-b5b1c3638e73-workload-certs\") pod \"20b674ae-6b2a-4084-b746-b5b1c3638e73\" (UID: \"20b674ae-6b2a-4084-b746-b5b1c3638e73\") " Jul 13 09:37:02.409938 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.409878 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20b674ae-6b2a-4084-b746-b5b1c3638e73-istio-envoy\") pod \"20b674ae-6b2a-4084-b746-b5b1c3638e73\" (UID: \"20b674ae-6b2a-4084-b746-b5b1c3638e73\") " Jul 13 09:37:02.409938 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.409893 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20b674ae-6b2a-4084-b746-b5b1c3638e73-istio-data\") pod \"20b674ae-6b2a-4084-b746-b5b1c3638e73\" (UID: \"20b674ae-6b2a-4084-b746-b5b1c3638e73\") " Jul 13 09:37:02.409938 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.409912 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20b674ae-6b2a-4084-b746-b5b1c3638e73-istio-token\") pod \"20b674ae-6b2a-4084-b746-b5b1c3638e73\" (UID: \"20b674ae-6b2a-4084-b746-b5b1c3638e73\") " Jul 13 09:37:02.410151 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.409965 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9m5lm\" (UniqueName: \"kubernetes.io/projected/20b674ae-6b2a-4084-b746-b5b1c3638e73-kube-api-access-9m5lm\") pod \"20b674ae-6b2a-4084-b746-b5b1c3638e73\" (UID: \"20b674ae-6b2a-4084-b746-b5b1c3638e73\") " Jul 13 09:37:02.410151 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.410008 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20b674ae-6b2a-4084-b746-b5b1c3638e73-istio-podinfo\") pod \"20b674ae-6b2a-4084-b746-b5b1c3638e73\" (UID: \"20b674ae-6b2a-4084-b746-b5b1c3638e73\") " Jul 13 09:37:02.410151 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.410052 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20b674ae-6b2a-4084-b746-b5b1c3638e73-workload-socket\") pod \"20b674ae-6b2a-4084-b746-b5b1c3638e73\" (UID: \"20b674ae-6b2a-4084-b746-b5b1c3638e73\") " Jul 13 09:37:02.410151 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.410087 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20b674ae-6b2a-4084-b746-b5b1c3638e73-credential-socket\") pod \"20b674ae-6b2a-4084-b746-b5b1c3638e73\" (UID: \"20b674ae-6b2a-4084-b746-b5b1c3638e73\") " Jul 13 09:37:02.410376 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.410253 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20b674ae-6b2a-4084-b746-b5b1c3638e73-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "20b674ae-6b2a-4084-b746-b5b1c3638e73" (UID: "20b674ae-6b2a-4084-b746-b5b1c3638e73"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 09:37:02.410458 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.410442 2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20b674ae-6b2a-4084-b746-b5b1c3638e73-istiod-ca-cert\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:37:02.412437 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.412416 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20b674ae-6b2a-4084-b746-b5b1c3638e73-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "20b674ae-6b2a-4084-b746-b5b1c3638e73" (UID: "20b674ae-6b2a-4084-b746-b5b1c3638e73"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:37:02.412916 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.412897 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20b674ae-6b2a-4084-b746-b5b1c3638e73-istio-data" (OuterVolumeSpecName: "istio-data") pod "20b674ae-6b2a-4084-b746-b5b1c3638e73" (UID: "20b674ae-6b2a-4084-b746-b5b1c3638e73"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:37:02.414487 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.414459 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20b674ae-6b2a-4084-b746-b5b1c3638e73-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "20b674ae-6b2a-4084-b746-b5b1c3638e73" (UID: "20b674ae-6b2a-4084-b746-b5b1c3638e73"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:37:02.414487 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.414475 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20b674ae-6b2a-4084-b746-b5b1c3638e73-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "20b674ae-6b2a-4084-b746-b5b1c3638e73" (UID: "20b674ae-6b2a-4084-b746-b5b1c3638e73"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:37:02.416137 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.416106 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20b674ae-6b2a-4084-b746-b5b1c3638e73-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "20b674ae-6b2a-4084-b746-b5b1c3638e73" (UID: "20b674ae-6b2a-4084-b746-b5b1c3638e73"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:37:02.416137 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.416110 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20b674ae-6b2a-4084-b746-b5b1c3638e73-istio-token" (OuterVolumeSpecName: "istio-token") pod "20b674ae-6b2a-4084-b746-b5b1c3638e73" (UID: "20b674ae-6b2a-4084-b746-b5b1c3638e73"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:37:02.416274 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.416130 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20b674ae-6b2a-4084-b746-b5b1c3638e73-kube-api-access-9m5lm" (OuterVolumeSpecName: "kube-api-access-9m5lm") pod "20b674ae-6b2a-4084-b746-b5b1c3638e73" (UID: "20b674ae-6b2a-4084-b746-b5b1c3638e73"). InnerVolumeSpecName "kube-api-access-9m5lm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:37:02.416274 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.416137 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/20b674ae-6b2a-4084-b746-b5b1c3638e73-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "20b674ae-6b2a-4084-b746-b5b1c3638e73" (UID: "20b674ae-6b2a-4084-b746-b5b1c3638e73"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 09:37:02.511611 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.511583 2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20b674ae-6b2a-4084-b746-b5b1c3638e73-workload-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:37:02.511722 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.511611 2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20b674ae-6b2a-4084-b746-b5b1c3638e73-credential-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:37:02.511722 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.511626 2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20b674ae-6b2a-4084-b746-b5b1c3638e73-workload-certs\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:37:02.511722 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.511638 2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20b674ae-6b2a-4084-b746-b5b1c3638e73-istio-envoy\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:37:02.511722 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.511650 2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20b674ae-6b2a-4084-b746-b5b1c3638e73-istio-data\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:37:02.511722 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.511661 2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20b674ae-6b2a-4084-b746-b5b1c3638e73-istio-token\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:37:02.511722 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.511672 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9m5lm\" (UniqueName: \"kubernetes.io/projected/20b674ae-6b2a-4084-b746-b5b1c3638e73-kube-api-access-9m5lm\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:37:02.511722 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.511686 2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20b674ae-6b2a-4084-b746-b5b1c3638e73-istio-podinfo\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:37:02.653817 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.653791 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx"] Jul 13 09:37:02.658169 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.658148 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-f7615a16-openshift-default-6f4b64b5bf-cg6bx"] Jul 13 09:37:02.667997 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:02.667971 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20b674ae-6b2a-4084-b746-b5b1c3638e73" path="/var/lib/kubelet/pods/20b674ae-6b2a-4084-b746-b5b1c3638e73/volumes" Jul 13 09:37:03.696612 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:03.696582 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:37:08.064331 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.064297 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:37:08.169944 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.169915 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:37:08.245416 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.245382 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc"] Jul 13 09:37:08.245952 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.245927 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20b674ae-6b2a-4084-b746-b5b1c3638e73" containerName="istio-proxy" Jul 13 09:37:08.245952 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.245953 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="20b674ae-6b2a-4084-b746-b5b1c3638e73" containerName="istio-proxy" Jul 13 09:37:08.246111 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.246025 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="20b674ae-6b2a-4084-b746-b5b1c3638e73" containerName="istio-proxy" Jul 13 09:37:08.249054 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.249036 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" Jul 13 09:37:08.252170 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.252154 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-591d4b92-openshift-default-dockercfg-pvnhz\"" Jul 13 09:37:08.263880 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.263858 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc"] Jul 13 09:37:08.279997 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.279972 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:37:08.358823 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.358788 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bebc0c2d-0ff8-479f-ab85-e69cee059894-workload-socket\") pod \"e2e-derive-591d4b92-openshift-default-768b57899-5fxrc\" (UID: \"bebc0c2d-0ff8-479f-ab85-e69cee059894\") " pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" Jul 13 09:37:08.358986 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.358837 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bebc0c2d-0ff8-479f-ab85-e69cee059894-istio-podinfo\") pod \"e2e-derive-591d4b92-openshift-default-768b57899-5fxrc\" (UID: \"bebc0c2d-0ff8-479f-ab85-e69cee059894\") " pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" Jul 13 09:37:08.358986 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.358859 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bebc0c2d-0ff8-479f-ab85-e69cee059894-credential-socket\") pod \"e2e-derive-591d4b92-openshift-default-768b57899-5fxrc\" (UID: \"bebc0c2d-0ff8-479f-ab85-e69cee059894\") " pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" Jul 13 09:37:08.358986 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.358926 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzvz4\" (UniqueName: \"kubernetes.io/projected/bebc0c2d-0ff8-479f-ab85-e69cee059894-kube-api-access-wzvz4\") pod \"e2e-derive-591d4b92-openshift-default-768b57899-5fxrc\" (UID: \"bebc0c2d-0ff8-479f-ab85-e69cee059894\") " pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" Jul 13 09:37:08.358986 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.358963 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bebc0c2d-0ff8-479f-ab85-e69cee059894-istio-data\") pod \"e2e-derive-591d4b92-openshift-default-768b57899-5fxrc\" (UID: \"bebc0c2d-0ff8-479f-ab85-e69cee059894\") " pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" Jul 13 09:37:08.359156 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.359000 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bebc0c2d-0ff8-479f-ab85-e69cee059894-istio-envoy\") pod \"e2e-derive-591d4b92-openshift-default-768b57899-5fxrc\" (UID: \"bebc0c2d-0ff8-479f-ab85-e69cee059894\") " pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" Jul 13 09:37:08.359156 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.359036 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bebc0c2d-0ff8-479f-ab85-e69cee059894-istio-token\") pod \"e2e-derive-591d4b92-openshift-default-768b57899-5fxrc\" (UID: \"bebc0c2d-0ff8-479f-ab85-e69cee059894\") " pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" Jul 13 09:37:08.359156 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.359053 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bebc0c2d-0ff8-479f-ab85-e69cee059894-workload-certs\") pod \"e2e-derive-591d4b92-openshift-default-768b57899-5fxrc\" (UID: \"bebc0c2d-0ff8-479f-ab85-e69cee059894\") " pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" Jul 13 09:37:08.359156 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.359101 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bebc0c2d-0ff8-479f-ab85-e69cee059894-istiod-ca-cert\") pod \"e2e-derive-591d4b92-openshift-default-768b57899-5fxrc\" (UID: \"bebc0c2d-0ff8-479f-ab85-e69cee059894\") " pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" Jul 13 09:37:08.380309 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.380284 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:37:08.460376 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.460347 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bebc0c2d-0ff8-479f-ab85-e69cee059894-workload-socket\") pod \"e2e-derive-591d4b92-openshift-default-768b57899-5fxrc\" (UID: \"bebc0c2d-0ff8-479f-ab85-e69cee059894\") " pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" Jul 13 09:37:08.460526 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.460394 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bebc0c2d-0ff8-479f-ab85-e69cee059894-istio-podinfo\") pod \"e2e-derive-591d4b92-openshift-default-768b57899-5fxrc\" (UID: \"bebc0c2d-0ff8-479f-ab85-e69cee059894\") " pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" Jul 13 09:37:08.460526 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.460438 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bebc0c2d-0ff8-479f-ab85-e69cee059894-credential-socket\") pod \"e2e-derive-591d4b92-openshift-default-768b57899-5fxrc\" (UID: \"bebc0c2d-0ff8-479f-ab85-e69cee059894\") " pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" Jul 13 09:37:08.460526 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.460460 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzvz4\" (UniqueName: \"kubernetes.io/projected/bebc0c2d-0ff8-479f-ab85-e69cee059894-kube-api-access-wzvz4\") pod \"e2e-derive-591d4b92-openshift-default-768b57899-5fxrc\" (UID: \"bebc0c2d-0ff8-479f-ab85-e69cee059894\") " pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" Jul 13 09:37:08.460526 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.460487 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bebc0c2d-0ff8-479f-ab85-e69cee059894-istio-data\") pod \"e2e-derive-591d4b92-openshift-default-768b57899-5fxrc\" (UID: \"bebc0c2d-0ff8-479f-ab85-e69cee059894\") " pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" Jul 13 09:37:08.460757 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.460534 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bebc0c2d-0ff8-479f-ab85-e69cee059894-istio-envoy\") pod \"e2e-derive-591d4b92-openshift-default-768b57899-5fxrc\" (UID: \"bebc0c2d-0ff8-479f-ab85-e69cee059894\") " pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" Jul 13 09:37:08.460757 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.460579 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bebc0c2d-0ff8-479f-ab85-e69cee059894-istio-token\") pod \"e2e-derive-591d4b92-openshift-default-768b57899-5fxrc\" (UID: \"bebc0c2d-0ff8-479f-ab85-e69cee059894\") " pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" Jul 13 09:37:08.460757 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.460603 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bebc0c2d-0ff8-479f-ab85-e69cee059894-workload-certs\") pod \"e2e-derive-591d4b92-openshift-default-768b57899-5fxrc\" (UID: \"bebc0c2d-0ff8-479f-ab85-e69cee059894\") " pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" Jul 13 09:37:08.460757 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.460654 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bebc0c2d-0ff8-479f-ab85-e69cee059894-istiod-ca-cert\") pod \"e2e-derive-591d4b92-openshift-default-768b57899-5fxrc\" (UID: \"bebc0c2d-0ff8-479f-ab85-e69cee059894\") " pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" Jul 13 09:37:08.460967 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.460847 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bebc0c2d-0ff8-479f-ab85-e69cee059894-workload-socket\") pod \"e2e-derive-591d4b92-openshift-default-768b57899-5fxrc\" (UID: \"bebc0c2d-0ff8-479f-ab85-e69cee059894\") " pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" Jul 13 09:37:08.460967 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.460877 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bebc0c2d-0ff8-479f-ab85-e69cee059894-credential-socket\") pod \"e2e-derive-591d4b92-openshift-default-768b57899-5fxrc\" (UID: \"bebc0c2d-0ff8-479f-ab85-e69cee059894\") " pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" Jul 13 09:37:08.461288 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.461262 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bebc0c2d-0ff8-479f-ab85-e69cee059894-istio-data\") pod \"e2e-derive-591d4b92-openshift-default-768b57899-5fxrc\" (UID: \"bebc0c2d-0ff8-479f-ab85-e69cee059894\") " pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" Jul 13 09:37:08.461410 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.461360 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bebc0c2d-0ff8-479f-ab85-e69cee059894-istiod-ca-cert\") pod \"e2e-derive-591d4b92-openshift-default-768b57899-5fxrc\" (UID: \"bebc0c2d-0ff8-479f-ab85-e69cee059894\") " pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" Jul 13 09:37:08.461497 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.461473 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bebc0c2d-0ff8-479f-ab85-e69cee059894-workload-certs\") pod \"e2e-derive-591d4b92-openshift-default-768b57899-5fxrc\" (UID: \"bebc0c2d-0ff8-479f-ab85-e69cee059894\") " pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" Jul 13 09:37:08.463086 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.463062 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bebc0c2d-0ff8-479f-ab85-e69cee059894-istio-envoy\") pod \"e2e-derive-591d4b92-openshift-default-768b57899-5fxrc\" (UID: \"bebc0c2d-0ff8-479f-ab85-e69cee059894\") " pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" Jul 13 09:37:08.463516 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.463497 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bebc0c2d-0ff8-479f-ab85-e69cee059894-istio-podinfo\") pod \"e2e-derive-591d4b92-openshift-default-768b57899-5fxrc\" (UID: \"bebc0c2d-0ff8-479f-ab85-e69cee059894\") " pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" Jul 13 09:37:08.469460 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.469440 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzvz4\" (UniqueName: \"kubernetes.io/projected/bebc0c2d-0ff8-479f-ab85-e69cee059894-kube-api-access-wzvz4\") pod \"e2e-derive-591d4b92-openshift-default-768b57899-5fxrc\" (UID: \"bebc0c2d-0ff8-479f-ab85-e69cee059894\") " pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" Jul 13 09:37:08.469561 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.469550 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bebc0c2d-0ff8-479f-ab85-e69cee059894-istio-token\") pod \"e2e-derive-591d4b92-openshift-default-768b57899-5fxrc\" (UID: \"bebc0c2d-0ff8-479f-ab85-e69cee059894\") " pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" Jul 13 09:37:08.560771 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.560747 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" Jul 13 09:37:08.705817 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.705791 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc"] Jul 13 09:37:08.706612 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:37:08.706582 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbebc0c2d_0ff8_479f_ab85_e69cee059894.slice/crio-b97375b0b9c7e59c36d4084dd9b2982c0d94beb973e0eb93c72c68bd6a7a4f24 WatchSource:0}: Error finding container b97375b0b9c7e59c36d4084dd9b2982c0d94beb973e0eb93c72c68bd6a7a4f24: Status 404 returned error can't find the container with id b97375b0b9c7e59c36d4084dd9b2982c0d94beb973e0eb93c72c68bd6a7a4f24 Jul 13 09:37:08.708577 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.708546 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:37:08.708653 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.708607 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:37:08.708653 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:08.708635 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:37:09.375076 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:09.375039 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" event={"ID":"bebc0c2d-0ff8-479f-ab85-e69cee059894","Type":"ContainerStarted","Data":"fc4e9001925c0529345d50558521fcbf6885ae761e5d832cbf46b1c2dbe4535f"} Jul 13 09:37:09.375553 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:09.375113 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" event={"ID":"bebc0c2d-0ff8-479f-ab85-e69cee059894","Type":"ContainerStarted","Data":"b97375b0b9c7e59c36d4084dd9b2982c0d94beb973e0eb93c72c68bd6a7a4f24"} Jul 13 09:37:09.418140 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:09.418061 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" podStartSLOduration=1.418034774 podStartE2EDuration="1.418034774s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:37:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:37:09.416356295 +0000 UTC m=+1713.296311590" watchObservedRunningTime="2026-07-13 09:37:09.418034774 +0000 UTC m=+1713.297990042" Jul 13 09:37:09.437826 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:09.437787 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:37:09.561685 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:09.561639 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" Jul 13 09:37:09.564447 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:09.564096 2552 patch_prober.go:28] interesting pod/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 13 09:37:09.564447 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:09.564151 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" podUID="bebc0c2d-0ff8-479f-ab85-e69cee059894" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 13 09:37:09.617184 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:09.617157 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:37:09.713523 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:09.713449 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:37:10.384666 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:10.384639 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:37:10.487255 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:10.487225 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:37:10.566058 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:10.566032 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" Jul 13 09:37:11.323176 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:11.323148 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:37:11.390620 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:11.390586 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" Jul 13 09:37:11.391618 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:11.391590 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" Jul 13 09:37:13.389719 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:13.389662 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:37:13.802501 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:13.802421 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:37:13.973478 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:13.973447 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc"] Jul 13 09:37:14.049131 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:14.049099 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:37:14.137904 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:14.137866 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:37:14.223930 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:14.223897 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:37:14.406625 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:14.406529 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" podUID="bebc0c2d-0ff8-479f-ab85-e69cee059894" containerName="istio-proxy" containerID="cri-o://fc4e9001925c0529345d50558521fcbf6885ae761e5d832cbf46b1c2dbe4535f" gracePeriod=30 Jul 13 09:37:15.401808 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:15.401781 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:37:19.664364 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:19.664303 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" Jul 13 09:37:19.760845 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:19.760821 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bebc0c2d-0ff8-479f-ab85-e69cee059894-istio-envoy\") pod \"bebc0c2d-0ff8-479f-ab85-e69cee059894\" (UID: \"bebc0c2d-0ff8-479f-ab85-e69cee059894\") " Jul 13 09:37:19.760994 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:19.760861 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bebc0c2d-0ff8-479f-ab85-e69cee059894-workload-certs\") pod \"bebc0c2d-0ff8-479f-ab85-e69cee059894\" (UID: \"bebc0c2d-0ff8-479f-ab85-e69cee059894\") " Jul 13 09:37:19.760994 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:19.760895 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bebc0c2d-0ff8-479f-ab85-e69cee059894-credential-socket\") pod \"bebc0c2d-0ff8-479f-ab85-e69cee059894\" (UID: \"bebc0c2d-0ff8-479f-ab85-e69cee059894\") " Jul 13 09:37:19.760994 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:19.760937 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzvz4\" (UniqueName: \"kubernetes.io/projected/bebc0c2d-0ff8-479f-ab85-e69cee059894-kube-api-access-wzvz4\") pod \"bebc0c2d-0ff8-479f-ab85-e69cee059894\" (UID: \"bebc0c2d-0ff8-479f-ab85-e69cee059894\") " Jul 13 09:37:19.760994 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:19.760968 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bebc0c2d-0ff8-479f-ab85-e69cee059894-istio-token\") pod \"bebc0c2d-0ff8-479f-ab85-e69cee059894\" (UID: \"bebc0c2d-0ff8-479f-ab85-e69cee059894\") " Jul 13 09:37:19.760994 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:19.760990 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bebc0c2d-0ff8-479f-ab85-e69cee059894-istiod-ca-cert\") pod \"bebc0c2d-0ff8-479f-ab85-e69cee059894\" (UID: \"bebc0c2d-0ff8-479f-ab85-e69cee059894\") " Jul 13 09:37:19.761302 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:19.761072 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bebc0c2d-0ff8-479f-ab85-e69cee059894-istio-podinfo\") pod \"bebc0c2d-0ff8-479f-ab85-e69cee059894\" (UID: \"bebc0c2d-0ff8-479f-ab85-e69cee059894\") " Jul 13 09:37:19.761302 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:19.761107 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bebc0c2d-0ff8-479f-ab85-e69cee059894-workload-socket\") pod \"bebc0c2d-0ff8-479f-ab85-e69cee059894\" (UID: \"bebc0c2d-0ff8-479f-ab85-e69cee059894\") " Jul 13 09:37:19.761302 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:19.761125 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bebc0c2d-0ff8-479f-ab85-e69cee059894-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bebc0c2d-0ff8-479f-ab85-e69cee059894" (UID: "bebc0c2d-0ff8-479f-ab85-e69cee059894"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:37:19.761302 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:19.761139 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bebc0c2d-0ff8-479f-ab85-e69cee059894-istio-data\") pod \"bebc0c2d-0ff8-479f-ab85-e69cee059894\" (UID: \"bebc0c2d-0ff8-479f-ab85-e69cee059894\") " Jul 13 09:37:19.761302 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:19.761231 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bebc0c2d-0ff8-479f-ab85-e69cee059894-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bebc0c2d-0ff8-479f-ab85-e69cee059894" (UID: "bebc0c2d-0ff8-479f-ab85-e69cee059894"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:37:19.761593 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:19.761518 2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bebc0c2d-0ff8-479f-ab85-e69cee059894-workload-certs\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:37:19.761593 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:19.761537 2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bebc0c2d-0ff8-479f-ab85-e69cee059894-credential-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:37:19.761872 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:19.761841 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bebc0c2d-0ff8-479f-ab85-e69cee059894-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bebc0c2d-0ff8-479f-ab85-e69cee059894" (UID: "bebc0c2d-0ff8-479f-ab85-e69cee059894"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 09:37:19.761950 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:19.761908 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bebc0c2d-0ff8-479f-ab85-e69cee059894-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bebc0c2d-0ff8-479f-ab85-e69cee059894" (UID: "bebc0c2d-0ff8-479f-ab85-e69cee059894"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:37:19.762440 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:19.762412 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bebc0c2d-0ff8-479f-ab85-e69cee059894-istio-data" (OuterVolumeSpecName: "istio-data") pod "bebc0c2d-0ff8-479f-ab85-e69cee059894" (UID: "bebc0c2d-0ff8-479f-ab85-e69cee059894"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:37:19.763410 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:19.763368 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bebc0c2d-0ff8-479f-ab85-e69cee059894-kube-api-access-wzvz4" (OuterVolumeSpecName: "kube-api-access-wzvz4") pod "bebc0c2d-0ff8-479f-ab85-e69cee059894" (UID: "bebc0c2d-0ff8-479f-ab85-e69cee059894"). InnerVolumeSpecName "kube-api-access-wzvz4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:37:19.763550 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:19.763441 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bebc0c2d-0ff8-479f-ab85-e69cee059894-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bebc0c2d-0ff8-479f-ab85-e69cee059894" (UID: "bebc0c2d-0ff8-479f-ab85-e69cee059894"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:37:19.763790 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:19.763756 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bebc0c2d-0ff8-479f-ab85-e69cee059894-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bebc0c2d-0ff8-479f-ab85-e69cee059894" (UID: "bebc0c2d-0ff8-479f-ab85-e69cee059894"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 09:37:19.763912 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:19.763897 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bebc0c2d-0ff8-479f-ab85-e69cee059894-istio-token" (OuterVolumeSpecName: "istio-token") pod "bebc0c2d-0ff8-479f-ab85-e69cee059894" (UID: "bebc0c2d-0ff8-479f-ab85-e69cee059894"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:37:19.862784 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:19.862750 2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bebc0c2d-0ff8-479f-ab85-e69cee059894-istio-envoy\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:37:19.862784 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:19.862774 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzvz4\" (UniqueName: \"kubernetes.io/projected/bebc0c2d-0ff8-479f-ab85-e69cee059894-kube-api-access-wzvz4\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:37:19.862784 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:19.862785 2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bebc0c2d-0ff8-479f-ab85-e69cee059894-istio-token\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:37:19.862986 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:19.862793 2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bebc0c2d-0ff8-479f-ab85-e69cee059894-istiod-ca-cert\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:37:19.862986 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:19.862803 2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bebc0c2d-0ff8-479f-ab85-e69cee059894-istio-podinfo\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:37:19.862986 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:19.862811 2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bebc0c2d-0ff8-479f-ab85-e69cee059894-workload-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:37:19.862986 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:19.862818 2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bebc0c2d-0ff8-479f-ab85-e69cee059894-istio-data\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:37:20.190518 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:20.190485 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:37:20.275536 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:20.275498 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:37:20.440182 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:20.440154 2552 generic.go:356] "Generic (PLEG): container finished" podID="bebc0c2d-0ff8-479f-ab85-e69cee059894" containerID="fc4e9001925c0529345d50558521fcbf6885ae761e5d832cbf46b1c2dbe4535f" exitCode=0 Jul 13 09:37:20.440380 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:20.440225 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" Jul 13 09:37:20.440380 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:20.440243 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" event={"ID":"bebc0c2d-0ff8-479f-ab85-e69cee059894","Type":"ContainerDied","Data":"fc4e9001925c0529345d50558521fcbf6885ae761e5d832cbf46b1c2dbe4535f"} Jul 13 09:37:20.440380 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:20.440295 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc" event={"ID":"bebc0c2d-0ff8-479f-ab85-e69cee059894","Type":"ContainerDied","Data":"b97375b0b9c7e59c36d4084dd9b2982c0d94beb973e0eb93c72c68bd6a7a4f24"} Jul 13 09:37:20.440380 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:20.440318 2552 scope.go:117] "RemoveContainer" containerID="fc4e9001925c0529345d50558521fcbf6885ae761e5d832cbf46b1c2dbe4535f" Jul 13 09:37:20.450817 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:20.450798 2552 scope.go:117] "RemoveContainer" containerID="fc4e9001925c0529345d50558521fcbf6885ae761e5d832cbf46b1c2dbe4535f" Jul 13 09:37:20.451068 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:37:20.451050 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc4e9001925c0529345d50558521fcbf6885ae761e5d832cbf46b1c2dbe4535f\": container with ID starting with fc4e9001925c0529345d50558521fcbf6885ae761e5d832cbf46b1c2dbe4535f not found: ID does not exist" containerID="fc4e9001925c0529345d50558521fcbf6885ae761e5d832cbf46b1c2dbe4535f" Jul 13 09:37:20.451111 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:20.451077 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc4e9001925c0529345d50558521fcbf6885ae761e5d832cbf46b1c2dbe4535f"} err="failed to get container status \"fc4e9001925c0529345d50558521fcbf6885ae761e5d832cbf46b1c2dbe4535f\": rpc error: code = NotFound desc = could not find container \"fc4e9001925c0529345d50558521fcbf6885ae761e5d832cbf46b1c2dbe4535f\": container with ID starting with fc4e9001925c0529345d50558521fcbf6885ae761e5d832cbf46b1c2dbe4535f not found: ID does not exist" Jul 13 09:37:20.473546 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:20.473522 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc"] Jul 13 09:37:20.491072 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:20.491051 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-591d4b92-openshift-default-768b57899-5fxrc"] Jul 13 09:37:20.666667 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:20.666635 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bebc0c2d-0ff8-479f-ab85-e69cee059894" path="/var/lib/kubelet/pods/bebc0c2d-0ff8-479f-ab85-e69cee059894/volumes" Jul 13 09:37:20.698910 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:20.698841 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:37:20.789796 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:20.789770 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:37:20.888774 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:20.888751 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:37:22.374995 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:22.374963 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:37:28.204656 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:28.204620 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:37:28.269812 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:28.269784 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:37:28.516685 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:28.516604 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:37:28.631275 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:28.631242 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:37:28.960923 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:28.960891 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:37:54.953977 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:37:54.953947 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:38:00.891701 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:00.891671 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:38:00.993530 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:00.993495 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:38:01.093845 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:01.093808 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:38:06.367014 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:06.366980 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:38:16.995298 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:16.995261 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:38:23.388778 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:23.388741 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:38:23.497025 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:23.496984 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:38:23.611227 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:23.611180 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:38:34.070778 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:34.070690 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:38:34.831401 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:34.831372 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:38:36.717837 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:36.717811 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gp6mv_47250d22-56b6-4503-af3f-36a9e79691b2/ovn-acl-logging/0.log" Jul 13 09:38:36.725690 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:36.725670 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gp6mv_47250d22-56b6-4503-af3f-36a9e79691b2/ovn-acl-logging/0.log" Jul 13 09:38:40.752792 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:40.752763 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:38:40.856517 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:40.856483 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:38:40.954298 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:40.954269 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:38:41.054369 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:41.054294 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:38:41.799268 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:41.799237 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:38:41.963662 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:41.963629 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:38:43.558714 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:43.558685 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:38:49.622219 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:49.622189 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:38:49.724827 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:49.724798 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:38:50.299401 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:50.299361 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:38:56.192465 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.192433 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:38:56.371485 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.371455 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:38:56.483767 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.483691 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:38:56.731081 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.731053 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr"] Jul 13 09:38:56.731454 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.731440 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bebc0c2d-0ff8-479f-ab85-e69cee059894" containerName="istio-proxy" Jul 13 09:38:56.731507 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.731478 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="bebc0c2d-0ff8-479f-ab85-e69cee059894" containerName="istio-proxy" Jul 13 09:38:56.731598 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.731588 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="bebc0c2d-0ff8-479f-ab85-e69cee059894" containerName="istio-proxy" Jul 13 09:38:56.734595 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.734555 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" Jul 13 09:38:56.737364 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.737340 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-9e060cc5-openshift-default-dockercfg-6t9zv\"" Jul 13 09:38:56.757262 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.757232 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:38:56.763474 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.763446 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr"] Jul 13 09:38:56.830257 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.829895 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a79093d-8976-45af-a289-64d588069501-istio-envoy\") pod \"e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr\" (UID: \"4a79093d-8976-45af-a289-64d588069501\") " pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" Jul 13 09:38:56.830257 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.829970 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ggtr\" (UniqueName: \"kubernetes.io/projected/4a79093d-8976-45af-a289-64d588069501-kube-api-access-8ggtr\") pod \"e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr\" (UID: \"4a79093d-8976-45af-a289-64d588069501\") " pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" Jul 13 09:38:56.830257 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.830004 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a79093d-8976-45af-a289-64d588069501-istiod-ca-cert\") pod \"e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr\" (UID: \"4a79093d-8976-45af-a289-64d588069501\") " pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" Jul 13 09:38:56.830257 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.830031 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a79093d-8976-45af-a289-64d588069501-workload-certs\") pod \"e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr\" (UID: \"4a79093d-8976-45af-a289-64d588069501\") " pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" Jul 13 09:38:56.830257 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.830088 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a79093d-8976-45af-a289-64d588069501-credential-socket\") pod \"e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr\" (UID: \"4a79093d-8976-45af-a289-64d588069501\") " pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" Jul 13 09:38:56.830257 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.830124 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a79093d-8976-45af-a289-64d588069501-istio-data\") pod \"e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr\" (UID: \"4a79093d-8976-45af-a289-64d588069501\") " pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" Jul 13 09:38:56.830257 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.830172 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a79093d-8976-45af-a289-64d588069501-workload-socket\") pod \"e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr\" (UID: \"4a79093d-8976-45af-a289-64d588069501\") " pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" Jul 13 09:38:56.830257 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.830233 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a79093d-8976-45af-a289-64d588069501-istio-podinfo\") pod \"e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr\" (UID: \"4a79093d-8976-45af-a289-64d588069501\") " pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" Jul 13 09:38:56.830257 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.830258 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a79093d-8976-45af-a289-64d588069501-istio-token\") pod \"e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr\" (UID: \"4a79093d-8976-45af-a289-64d588069501\") " pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" Jul 13 09:38:56.931523 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.931496 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a79093d-8976-45af-a289-64d588069501-workload-socket\") pod \"e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr\" (UID: \"4a79093d-8976-45af-a289-64d588069501\") " pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" Jul 13 09:38:56.931697 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.931536 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a79093d-8976-45af-a289-64d588069501-istio-podinfo\") pod \"e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr\" (UID: \"4a79093d-8976-45af-a289-64d588069501\") " pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" Jul 13 09:38:56.931697 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.931556 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a79093d-8976-45af-a289-64d588069501-istio-token\") pod \"e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr\" (UID: \"4a79093d-8976-45af-a289-64d588069501\") " pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" Jul 13 09:38:56.931697 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.931599 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a79093d-8976-45af-a289-64d588069501-istio-envoy\") pod \"e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr\" (UID: \"4a79093d-8976-45af-a289-64d588069501\") " pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" Jul 13 09:38:56.931697 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.931637 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ggtr\" (UniqueName: \"kubernetes.io/projected/4a79093d-8976-45af-a289-64d588069501-kube-api-access-8ggtr\") pod \"e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr\" (UID: \"4a79093d-8976-45af-a289-64d588069501\") " pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" Jul 13 09:38:56.931697 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.931656 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a79093d-8976-45af-a289-64d588069501-istiod-ca-cert\") pod \"e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr\" (UID: \"4a79093d-8976-45af-a289-64d588069501\") " pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" Jul 13 09:38:56.931697 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.931678 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a79093d-8976-45af-a289-64d588069501-workload-certs\") pod \"e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr\" (UID: \"4a79093d-8976-45af-a289-64d588069501\") " pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" Jul 13 09:38:56.932035 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.931719 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a79093d-8976-45af-a289-64d588069501-credential-socket\") pod \"e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr\" (UID: \"4a79093d-8976-45af-a289-64d588069501\") " pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" Jul 13 09:38:56.932035 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.931740 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a79093d-8976-45af-a289-64d588069501-istio-data\") pod \"e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr\" (UID: \"4a79093d-8976-45af-a289-64d588069501\") " pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" Jul 13 09:38:56.932035 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.931960 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a79093d-8976-45af-a289-64d588069501-workload-socket\") pod \"e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr\" (UID: \"4a79093d-8976-45af-a289-64d588069501\") " pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" Jul 13 09:38:56.932196 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.932095 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a79093d-8976-45af-a289-64d588069501-credential-socket\") pod \"e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr\" (UID: \"4a79093d-8976-45af-a289-64d588069501\") " pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" Jul 13 09:38:56.932196 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.932137 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a79093d-8976-45af-a289-64d588069501-istio-data\") pod \"e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr\" (UID: \"4a79093d-8976-45af-a289-64d588069501\") " pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" Jul 13 09:38:56.932196 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.932186 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a79093d-8976-45af-a289-64d588069501-workload-certs\") pod \"e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr\" (UID: \"4a79093d-8976-45af-a289-64d588069501\") " pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" Jul 13 09:38:56.932385 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.932348 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a79093d-8976-45af-a289-64d588069501-istiod-ca-cert\") pod \"e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr\" (UID: \"4a79093d-8976-45af-a289-64d588069501\") " pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" Jul 13 09:38:56.934010 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.933984 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a79093d-8976-45af-a289-64d588069501-istio-envoy\") pod \"e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr\" (UID: \"4a79093d-8976-45af-a289-64d588069501\") " pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" Jul 13 09:38:56.934231 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.934190 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a79093d-8976-45af-a289-64d588069501-istio-podinfo\") pod \"e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr\" (UID: \"4a79093d-8976-45af-a289-64d588069501\") " pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" Jul 13 09:38:56.940026 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.939998 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ggtr\" (UniqueName: \"kubernetes.io/projected/4a79093d-8976-45af-a289-64d588069501-kube-api-access-8ggtr\") pod \"e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr\" (UID: \"4a79093d-8976-45af-a289-64d588069501\") " pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" Jul 13 09:38:56.940124 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.940086 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a79093d-8976-45af-a289-64d588069501-istio-token\") pod \"e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr\" (UID: \"4a79093d-8976-45af-a289-64d588069501\") " pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" Jul 13 09:38:56.946254 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:56.946202 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:38:57.045029 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:57.044964 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" Jul 13 09:38:57.188012 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:57.187986 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr"] Jul 13 09:38:57.189429 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:38:57.189401 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a79093d_8976_45af_a289_64d588069501.slice/crio-324e3ffe2a1f1ab55a86da868d148e9019f8671d13f2407bc0803e60ab9922fb WatchSource:0}: Error finding container 324e3ffe2a1f1ab55a86da868d148e9019f8671d13f2407bc0803e60ab9922fb: Status 404 returned error can't find the container with id 324e3ffe2a1f1ab55a86da868d148e9019f8671d13f2407bc0803e60ab9922fb Jul 13 09:38:57.191561 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:57.191528 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:38:57.191661 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:57.191592 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:38:57.191661 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:57.191618 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:38:57.992096 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:57.992063 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" event={"ID":"4a79093d-8976-45af-a289-64d588069501","Type":"ContainerStarted","Data":"87dc96ed375187179ecddbb69e4932f9998173d138a1e0000d20558525be22c9"} Jul 13 09:38:57.992480 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:57.992123 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" event={"ID":"4a79093d-8976-45af-a289-64d588069501","Type":"ContainerStarted","Data":"324e3ffe2a1f1ab55a86da868d148e9019f8671d13f2407bc0803e60ab9922fb"} Jul 13 09:38:58.037645 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:58.037598 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" podStartSLOduration=2.037585018 podStartE2EDuration="2.037585018s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:38:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:38:58.036647117 +0000 UTC m=+1821.916602387" watchObservedRunningTime="2026-07-13 09:38:58.037585018 +0000 UTC m=+1821.917540355" Jul 13 09:38:58.045866 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:58.045845 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" Jul 13 09:38:58.051238 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:58.051204 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" Jul 13 09:38:58.998625 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:58.998595 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" Jul 13 09:38:58.999443 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:58.999420 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" Jul 13 09:38:59.956529 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:38:59.956496 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:39:00.056268 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:00.056230 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:39:00.234119 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:00.234034 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:39:00.497579 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:00.497498 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:39:00.802474 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:00.802401 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:39:00.905245 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:00.905187 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:39:04.513877 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:04.513846 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:39:05.144345 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:05.144311 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:39:05.242570 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:05.242529 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:39:05.615439 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:05.615412 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:39:11.584924 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:11.584891 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:39:11.649804 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:11.649766 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:39:12.073416 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:12.073313 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:39:12.418774 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:12.418734 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:39:12.519469 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:12.519435 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:39:14.000121 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:14.000086 2552 patch_prober.go:28] interesting pod/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 13 09:39:14.000561 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:14.000140 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" podUID="4a79093d-8976-45af-a289-64d588069501" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 13 09:39:14.087555 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:14.087525 2552 generic.go:356] "Generic (PLEG): container finished" podID="4a79093d-8976-45af-a289-64d588069501" containerID="87dc96ed375187179ecddbb69e4932f9998173d138a1e0000d20558525be22c9" exitCode=0 Jul 13 09:39:14.087714 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:14.087592 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" event={"ID":"4a79093d-8976-45af-a289-64d588069501","Type":"ContainerDied","Data":"87dc96ed375187179ecddbb69e4932f9998173d138a1e0000d20558525be22c9"} Jul 13 09:39:14.087962 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:14.087946 2552 scope.go:117] "RemoveContainer" containerID="87dc96ed375187179ecddbb69e4932f9998173d138a1e0000d20558525be22c9" Jul 13 09:39:14.088922 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:14.088892 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:39:14.089029 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:14.088975 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:39:14.089107 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:14.089048 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:39:15.096141 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:15.096107 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" event={"ID":"4a79093d-8976-45af-a289-64d588069501","Type":"ContainerStarted","Data":"ae5d82c6e4ade67bd153b4462ed2ed203da2ac616b49572dd722f96418760fb8"} Jul 13 09:39:16.045314 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:16.045284 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" Jul 13 09:39:16.045574 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:16.045553 2552 patch_prober.go:28] interesting pod/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 13 09:39:16.045638 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:16.045614 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" podUID="4a79093d-8976-45af-a289-64d588069501" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 13 09:39:17.045919 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:17.045881 2552 patch_prober.go:28] interesting pod/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 13 09:39:17.046428 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:17.045943 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" podUID="4a79093d-8976-45af-a289-64d588069501" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 13 09:39:18.045649 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:18.045612 2552 patch_prober.go:28] interesting pod/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 13 09:39:18.045829 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:18.045666 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" podUID="4a79093d-8976-45af-a289-64d588069501" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 13 09:39:18.412196 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:18.412155 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:39:18.656045 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:18.656012 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr"] Jul 13 09:39:18.656499 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:18.656467 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" podUID="4a79093d-8976-45af-a289-64d588069501" containerName="istio-proxy" containerID="cri-o://ae5d82c6e4ade67bd153b4462ed2ed203da2ac616b49572dd722f96418760fb8" gracePeriod=30 Jul 13 09:39:19.825404 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:19.825366 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:39:19.893934 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:19.893902 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:39:24.020306 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.020283 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" Jul 13 09:39:24.080386 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.080349 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a79093d-8976-45af-a289-64d588069501-istio-token\") pod \"4a79093d-8976-45af-a289-64d588069501\" (UID: \"4a79093d-8976-45af-a289-64d588069501\") " Jul 13 09:39:24.080386 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.080391 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a79093d-8976-45af-a289-64d588069501-istio-data\") pod \"4a79093d-8976-45af-a289-64d588069501\" (UID: \"4a79093d-8976-45af-a289-64d588069501\") " Jul 13 09:39:24.080611 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.080408 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a79093d-8976-45af-a289-64d588069501-istio-envoy\") pod \"4a79093d-8976-45af-a289-64d588069501\" (UID: \"4a79093d-8976-45af-a289-64d588069501\") " Jul 13 09:39:24.080611 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.080426 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a79093d-8976-45af-a289-64d588069501-workload-socket\") pod \"4a79093d-8976-45af-a289-64d588069501\" (UID: \"4a79093d-8976-45af-a289-64d588069501\") " Jul 13 09:39:24.080611 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.080446 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a79093d-8976-45af-a289-64d588069501-istiod-ca-cert\") pod \"4a79093d-8976-45af-a289-64d588069501\" (UID: \"4a79093d-8976-45af-a289-64d588069501\") " Jul 13 09:39:24.080611 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.080462 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a79093d-8976-45af-a289-64d588069501-workload-certs\") pod \"4a79093d-8976-45af-a289-64d588069501\" (UID: \"4a79093d-8976-45af-a289-64d588069501\") " Jul 13 09:39:24.080611 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.080486 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a79093d-8976-45af-a289-64d588069501-credential-socket\") pod \"4a79093d-8976-45af-a289-64d588069501\" (UID: \"4a79093d-8976-45af-a289-64d588069501\") " Jul 13 09:39:24.080611 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.080533 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8ggtr\" (UniqueName: \"kubernetes.io/projected/4a79093d-8976-45af-a289-64d588069501-kube-api-access-8ggtr\") pod \"4a79093d-8976-45af-a289-64d588069501\" (UID: \"4a79093d-8976-45af-a289-64d588069501\") " Jul 13 09:39:24.080987 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.080720 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a79093d-8976-45af-a289-64d588069501-istio-podinfo\") pod \"4a79093d-8976-45af-a289-64d588069501\" (UID: \"4a79093d-8976-45af-a289-64d588069501\") " Jul 13 09:39:24.081085 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.080715 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a79093d-8976-45af-a289-64d588069501-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4a79093d-8976-45af-a289-64d588069501" (UID: "4a79093d-8976-45af-a289-64d588069501"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:39:24.081166 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.080844 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a79093d-8976-45af-a289-64d588069501-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4a79093d-8976-45af-a289-64d588069501" (UID: "4a79093d-8976-45af-a289-64d588069501"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:39:24.081166 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.080926 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a79093d-8976-45af-a289-64d588069501-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4a79093d-8976-45af-a289-64d588069501" (UID: "4a79093d-8976-45af-a289-64d588069501"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 09:39:24.081166 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.081113 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a79093d-8976-45af-a289-64d588069501-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4a79093d-8976-45af-a289-64d588069501" (UID: "4a79093d-8976-45af-a289-64d588069501"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:39:24.081473 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.081447 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a79093d-8976-45af-a289-64d588069501-istio-data" (OuterVolumeSpecName: "istio-data") pod "4a79093d-8976-45af-a289-64d588069501" (UID: "4a79093d-8976-45af-a289-64d588069501"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:39:24.082676 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.082642 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a79093d-8976-45af-a289-64d588069501-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4a79093d-8976-45af-a289-64d588069501" (UID: "4a79093d-8976-45af-a289-64d588069501"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:39:24.082676 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.082666 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a79093d-8976-45af-a289-64d588069501-istio-token" (OuterVolumeSpecName: "istio-token") pod "4a79093d-8976-45af-a289-64d588069501" (UID: "4a79093d-8976-45af-a289-64d588069501"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:39:24.082813 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.082696 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a79093d-8976-45af-a289-64d588069501-kube-api-access-8ggtr" (OuterVolumeSpecName: "kube-api-access-8ggtr") pod "4a79093d-8976-45af-a289-64d588069501" (UID: "4a79093d-8976-45af-a289-64d588069501"). InnerVolumeSpecName "kube-api-access-8ggtr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:39:24.082813 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.082739 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4a79093d-8976-45af-a289-64d588069501-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4a79093d-8976-45af-a289-64d588069501" (UID: "4a79093d-8976-45af-a289-64d588069501"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 09:39:24.148930 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.148848 2552 generic.go:356] "Generic (PLEG): container finished" podID="4a79093d-8976-45af-a289-64d588069501" containerID="ae5d82c6e4ade67bd153b4462ed2ed203da2ac616b49572dd722f96418760fb8" exitCode=0 Jul 13 09:39:24.148930 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.148913 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" Jul 13 09:39:24.149103 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.148939 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" event={"ID":"4a79093d-8976-45af-a289-64d588069501","Type":"ContainerDied","Data":"ae5d82c6e4ade67bd153b4462ed2ed203da2ac616b49572dd722f96418760fb8"} Jul 13 09:39:24.149103 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.148993 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr" event={"ID":"4a79093d-8976-45af-a289-64d588069501","Type":"ContainerDied","Data":"324e3ffe2a1f1ab55a86da868d148e9019f8671d13f2407bc0803e60ab9922fb"} Jul 13 09:39:24.149103 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.149015 2552 scope.go:117] "RemoveContainer" containerID="ae5d82c6e4ade67bd153b4462ed2ed203da2ac616b49572dd722f96418760fb8" Jul 13 09:39:24.158236 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.158204 2552 scope.go:117] "RemoveContainer" containerID="87dc96ed375187179ecddbb69e4932f9998173d138a1e0000d20558525be22c9" Jul 13 09:39:24.166560 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.166541 2552 scope.go:117] "RemoveContainer" containerID="ae5d82c6e4ade67bd153b4462ed2ed203da2ac616b49572dd722f96418760fb8" Jul 13 09:39:24.166819 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:39:24.166800 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae5d82c6e4ade67bd153b4462ed2ed203da2ac616b49572dd722f96418760fb8\": container with ID starting with ae5d82c6e4ade67bd153b4462ed2ed203da2ac616b49572dd722f96418760fb8 not found: ID does not exist" containerID="ae5d82c6e4ade67bd153b4462ed2ed203da2ac616b49572dd722f96418760fb8" Jul 13 09:39:24.166901 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.166831 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae5d82c6e4ade67bd153b4462ed2ed203da2ac616b49572dd722f96418760fb8"} err="failed to get container status \"ae5d82c6e4ade67bd153b4462ed2ed203da2ac616b49572dd722f96418760fb8\": rpc error: code = NotFound desc = could not find container \"ae5d82c6e4ade67bd153b4462ed2ed203da2ac616b49572dd722f96418760fb8\": container with ID starting with ae5d82c6e4ade67bd153b4462ed2ed203da2ac616b49572dd722f96418760fb8 not found: ID does not exist" Jul 13 09:39:24.166901 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.166854 2552 scope.go:117] "RemoveContainer" containerID="87dc96ed375187179ecddbb69e4932f9998173d138a1e0000d20558525be22c9" Jul 13 09:39:24.167129 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:39:24.167112 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87dc96ed375187179ecddbb69e4932f9998173d138a1e0000d20558525be22c9\": container with ID starting with 87dc96ed375187179ecddbb69e4932f9998173d138a1e0000d20558525be22c9 not found: ID does not exist" containerID="87dc96ed375187179ecddbb69e4932f9998173d138a1e0000d20558525be22c9" Jul 13 09:39:24.167174 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.167135 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87dc96ed375187179ecddbb69e4932f9998173d138a1e0000d20558525be22c9"} err="failed to get container status \"87dc96ed375187179ecddbb69e4932f9998173d138a1e0000d20558525be22c9\": rpc error: code = NotFound desc = could not find container \"87dc96ed375187179ecddbb69e4932f9998173d138a1e0000d20558525be22c9\": container with ID starting with 87dc96ed375187179ecddbb69e4932f9998173d138a1e0000d20558525be22c9 not found: ID does not exist" Jul 13 09:39:24.174016 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.173982 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr"] Jul 13 09:39:24.176419 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.176398 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-9e060cc5-openshift-default-6fc4c6ff4c-6dcpr"] Jul 13 09:39:24.181662 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.181641 2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a79093d-8976-45af-a289-64d588069501-istiod-ca-cert\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:39:24.181662 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.181664 2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a79093d-8976-45af-a289-64d588069501-workload-certs\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:39:24.181773 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.181673 2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a79093d-8976-45af-a289-64d588069501-credential-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:39:24.181773 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.181683 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ggtr\" (UniqueName: \"kubernetes.io/projected/4a79093d-8976-45af-a289-64d588069501-kube-api-access-8ggtr\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:39:24.181773 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.181694 2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a79093d-8976-45af-a289-64d588069501-istio-podinfo\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:39:24.181773 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.181720 2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a79093d-8976-45af-a289-64d588069501-istio-token\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:39:24.181773 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.181730 2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a79093d-8976-45af-a289-64d588069501-istio-data\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:39:24.181773 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.181737 2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a79093d-8976-45af-a289-64d588069501-istio-envoy\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:39:24.181773 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.181744 2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a79093d-8976-45af-a289-64d588069501-workload-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:39:24.666867 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.666836 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a79093d-8976-45af-a289-64d588069501" path="/var/lib/kubelet/pods/4a79093d-8976-45af-a289-64d588069501/volumes" Jul 13 09:39:24.966831 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:24.966754 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:39:25.069651 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:25.069623 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:39:25.167236 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:25.167181 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:39:25.362011 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:25.361983 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:39:31.149632 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:31.149599 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:39:31.225710 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:31.225679 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:39:38.201628 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.201593 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:39:38.276108 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.276071 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx"] Jul 13 09:39:38.276572 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.276550 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a79093d-8976-45af-a289-64d588069501" containerName="istio-proxy" Jul 13 09:39:38.276710 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.276618 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a79093d-8976-45af-a289-64d588069501" containerName="istio-proxy" Jul 13 09:39:38.276710 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.276668 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a79093d-8976-45af-a289-64d588069501" containerName="istio-proxy" Jul 13 09:39:38.276710 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.276677 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a79093d-8976-45af-a289-64d588069501" containerName="istio-proxy" Jul 13 09:39:38.276880 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.276795 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a79093d-8976-45af-a289-64d588069501" containerName="istio-proxy" Jul 13 09:39:38.276880 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.276812 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a79093d-8976-45af-a289-64d588069501" containerName="istio-proxy" Jul 13 09:39:38.281578 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.281557 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:39:38.284550 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.284530 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-7694df9c-openshift-default-dockercfg-z995b\"" Jul 13 09:39:38.293032 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.293009 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx"] Jul 13 09:39:38.302970 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.302943 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:39:38.387981 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.387947 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0aa03a53-6fc2-4952-8455-74b73bf14dd9-workload-certs\") pod \"e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx\" (UID: \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\") " pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:39:38.388154 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.387998 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0aa03a53-6fc2-4952-8455-74b73bf14dd9-workload-socket\") pod \"e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx\" (UID: \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\") " pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:39:38.388154 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.388056 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0aa03a53-6fc2-4952-8455-74b73bf14dd9-credential-socket\") pod \"e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx\" (UID: \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\") " pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:39:38.388154 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.388107 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0aa03a53-6fc2-4952-8455-74b73bf14dd9-istio-data\") pod \"e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx\" (UID: \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\") " pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:39:38.388154 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.388130 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlgpz\" (UniqueName: \"kubernetes.io/projected/0aa03a53-6fc2-4952-8455-74b73bf14dd9-kube-api-access-nlgpz\") pod \"e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx\" (UID: \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\") " pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:39:38.388419 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.388187 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0aa03a53-6fc2-4952-8455-74b73bf14dd9-istio-token\") pod \"e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx\" (UID: \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\") " pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:39:38.388419 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.388242 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0aa03a53-6fc2-4952-8455-74b73bf14dd9-istio-envoy\") pod \"e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx\" (UID: \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\") " pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:39:38.388419 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.388279 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0aa03a53-6fc2-4952-8455-74b73bf14dd9-istio-podinfo\") pod \"e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx\" (UID: \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\") " pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:39:38.388419 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.388319 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0aa03a53-6fc2-4952-8455-74b73bf14dd9-istiod-ca-cert\") pod \"e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx\" (UID: \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\") " pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:39:38.488874 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.488808 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0aa03a53-6fc2-4952-8455-74b73bf14dd9-workload-certs\") pod \"e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx\" (UID: \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\") " pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:39:38.488874 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.488846 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0aa03a53-6fc2-4952-8455-74b73bf14dd9-workload-socket\") pod \"e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx\" (UID: \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\") " pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:39:38.488874 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.488862 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0aa03a53-6fc2-4952-8455-74b73bf14dd9-credential-socket\") pod \"e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx\" (UID: \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\") " pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:39:38.489124 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.488885 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0aa03a53-6fc2-4952-8455-74b73bf14dd9-istio-data\") pod \"e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx\" (UID: \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\") " pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:39:38.489124 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.488902 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlgpz\" (UniqueName: \"kubernetes.io/projected/0aa03a53-6fc2-4952-8455-74b73bf14dd9-kube-api-access-nlgpz\") pod \"e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx\" (UID: \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\") " pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:39:38.489124 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.488939 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0aa03a53-6fc2-4952-8455-74b73bf14dd9-istio-token\") pod \"e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx\" (UID: \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\") " pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:39:38.489124 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.488983 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0aa03a53-6fc2-4952-8455-74b73bf14dd9-istio-envoy\") pod \"e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx\" (UID: \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\") " pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:39:38.489124 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.489008 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0aa03a53-6fc2-4952-8455-74b73bf14dd9-istio-podinfo\") pod \"e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx\" (UID: \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\") " pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:39:38.489124 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.489038 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0aa03a53-6fc2-4952-8455-74b73bf14dd9-istiod-ca-cert\") pod \"e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx\" (UID: \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\") " pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:39:38.489482 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.489285 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0aa03a53-6fc2-4952-8455-74b73bf14dd9-workload-certs\") pod \"e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx\" (UID: \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\") " pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:39:38.489482 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.489315 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0aa03a53-6fc2-4952-8455-74b73bf14dd9-credential-socket\") pod \"e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx\" (UID: \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\") " pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:39:38.489482 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.489383 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0aa03a53-6fc2-4952-8455-74b73bf14dd9-workload-socket\") pod \"e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx\" (UID: \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\") " pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:39:38.489482 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.489421 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0aa03a53-6fc2-4952-8455-74b73bf14dd9-istio-data\") pod \"e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx\" (UID: \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\") " pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:39:38.489880 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.489857 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0aa03a53-6fc2-4952-8455-74b73bf14dd9-istiod-ca-cert\") pod \"e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx\" (UID: \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\") " pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:39:38.491403 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.491373 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0aa03a53-6fc2-4952-8455-74b73bf14dd9-istio-envoy\") pod \"e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx\" (UID: \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\") " pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:39:38.491649 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.491629 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0aa03a53-6fc2-4952-8455-74b73bf14dd9-istio-podinfo\") pod \"e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx\" (UID: \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\") " pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:39:38.497866 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.497841 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0aa03a53-6fc2-4952-8455-74b73bf14dd9-istio-token\") pod \"e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx\" (UID: \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\") " pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:39:38.498737 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.498715 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlgpz\" (UniqueName: \"kubernetes.io/projected/0aa03a53-6fc2-4952-8455-74b73bf14dd9-kube-api-access-nlgpz\") pod \"e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx\" (UID: \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\") " pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:39:38.592987 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.592960 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:39:38.617893 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.617870 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:39:38.762611 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.762583 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx"] Jul 13 09:39:38.763226 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:39:38.763189 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0aa03a53_6fc2_4952_8455_74b73bf14dd9.slice/crio-fa9d7ead5aca10045dae97e4f3115b7b3ff3a8b6ae96c3e1fb6ca80063172e5e WatchSource:0}: Error finding container fa9d7ead5aca10045dae97e4f3115b7b3ff3a8b6ae96c3e1fb6ca80063172e5e: Status 404 returned error can't find the container with id fa9d7ead5aca10045dae97e4f3115b7b3ff3a8b6ae96c3e1fb6ca80063172e5e Jul 13 09:39:38.765931 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.765900 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:39:38.766034 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.765979 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:39:38.766034 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:38.766025 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:39:39.245923 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:39.245891 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" event={"ID":"0aa03a53-6fc2-4952-8455-74b73bf14dd9","Type":"ContainerStarted","Data":"fe094424f26ec919766cca82478cfbf63b6d0b64322eddc7f67ac4b7a07782c9"} Jul 13 09:39:39.246328 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:39.245941 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" event={"ID":"0aa03a53-6fc2-4952-8455-74b73bf14dd9","Type":"ContainerStarted","Data":"fa9d7ead5aca10045dae97e4f3115b7b3ff3a8b6ae96c3e1fb6ca80063172e5e"} Jul 13 09:39:39.273849 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:39.273794 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" podStartSLOduration=1.2737750860000001 podStartE2EDuration="1.273775086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:39:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:39:39.270065854 +0000 UTC m=+1863.150021122" watchObservedRunningTime="2026-07-13 09:39:39.273775086 +0000 UTC m=+1863.153730354" Jul 13 09:39:39.593311 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:39.593286 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:39:39.597931 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:39.597905 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:39:40.251890 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:40.251858 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:39:40.252714 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:40.252693 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:39:41.163156 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:41.163115 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:39:41.254259 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:41.254205 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:39:41.332902 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:41.332869 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:39:41.461742 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:41.461666 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:39:41.724033 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:41.723948 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:39:41.959978 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:41.959944 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:39:45.661942 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:45.661904 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:39:51.894974 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:51.894941 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:39:55.252847 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:55.252773 2552 patch_prober.go:28] interesting pod/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 13 09:39:55.252847 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:55.252827 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" podUID="0aa03a53-6fc2-4952-8455-74b73bf14dd9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 13 09:39:55.347306 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:55.347280 2552 generic.go:356] "Generic (PLEG): container finished" podID="0aa03a53-6fc2-4952-8455-74b73bf14dd9" containerID="fe094424f26ec919766cca82478cfbf63b6d0b64322eddc7f67ac4b7a07782c9" exitCode=0 Jul 13 09:39:55.347449 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:55.347356 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" event={"ID":"0aa03a53-6fc2-4952-8455-74b73bf14dd9","Type":"ContainerDied","Data":"fe094424f26ec919766cca82478cfbf63b6d0b64322eddc7f67ac4b7a07782c9"} Jul 13 09:39:55.347754 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:55.347741 2552 scope.go:117] "RemoveContainer" containerID="fe094424f26ec919766cca82478cfbf63b6d0b64322eddc7f67ac4b7a07782c9" Jul 13 09:39:55.348817 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:55.348789 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:39:55.348886 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:55.348862 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:39:55.348923 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:55.348896 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:39:56.355292 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:56.355259 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" event={"ID":"0aa03a53-6fc2-4952-8455-74b73bf14dd9","Type":"ContainerStarted","Data":"f09c5d0d81bc3e9d914fbd229b3e801c2a77e518b190065e5a2d0d5d3bcac2e0"} Jul 13 09:39:56.593543 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:56.593501 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:39:56.593744 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:56.593722 2552 patch_prober.go:28] interesting pod/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 13 09:39:56.593793 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:56.593769 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" podUID="0aa03a53-6fc2-4952-8455-74b73bf14dd9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 13 09:39:57.593820 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:57.593783 2552 patch_prober.go:28] interesting pod/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 13 09:39:57.594200 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:57.593842 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" podUID="0aa03a53-6fc2-4952-8455-74b73bf14dd9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 13 09:39:58.594146 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:58.594106 2552 patch_prober.go:28] interesting pod/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 13 09:39:58.594556 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:58.594180 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" podUID="0aa03a53-6fc2-4952-8455-74b73bf14dd9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 13 09:39:59.593561 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:59.593525 2552 patch_prober.go:28] interesting pod/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 13 09:39:59.593735 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:39:59.593577 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" podUID="0aa03a53-6fc2-4952-8455-74b73bf14dd9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 13 09:40:00.596782 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:40:00.596755 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:40:00.597173 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:40:00.596976 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:40:00.597597 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:40:00.597579 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:40:06.628506 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:40:06.628470 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:40:06.725406 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:40:06.725377 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:40:18.195034 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:40:18.194997 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:40:18.642384 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:40:18.642351 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:40:18.743812 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:40:18.743787 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:40:18.903509 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:40:18.903431 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:40:24.503250 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:40:24.503201 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:40:29.579729 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:40:29.579688 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:41:05.658834 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:05.658796 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:41:06.643437 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:06.643409 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:41:12.474354 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:12.474289 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:41:18.457654 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:18.457620 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:41:18.558999 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:18.558940 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:41:18.708460 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:18.708378 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx"] Jul 13 09:41:18.708612 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:18.708579 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" podUID="0aa03a53-6fc2-4952-8455-74b73bf14dd9" containerName="istio-proxy" containerID="cri-o://f09c5d0d81bc3e9d914fbd229b3e801c2a77e518b190065e5a2d0d5d3bcac2e0" gracePeriod=30 Jul 13 09:41:18.728181 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:18.728150 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:41:19.977915 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:19.977882 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:41:20.075898 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:20.075848 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:41:20.178340 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:20.178299 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:41:20.895150 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:20.895094 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:41:20.997879 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:20.997846 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:41:21.094110 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:21.094079 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:41:22.441342 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:22.441311 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:41:23.856222 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:23.856181 2552 generic.go:356] "Generic (PLEG): container finished" podID="0aa03a53-6fc2-4952-8455-74b73bf14dd9" containerID="f09c5d0d81bc3e9d914fbd229b3e801c2a77e518b190065e5a2d0d5d3bcac2e0" exitCode=0 Jul 13 09:41:23.856635 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:23.856260 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" event={"ID":"0aa03a53-6fc2-4952-8455-74b73bf14dd9","Type":"ContainerDied","Data":"f09c5d0d81bc3e9d914fbd229b3e801c2a77e518b190065e5a2d0d5d3bcac2e0"} Jul 13 09:41:23.856635 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:23.856349 2552 scope.go:117] "RemoveContainer" containerID="fe094424f26ec919766cca82478cfbf63b6d0b64322eddc7f67ac4b7a07782c9" Jul 13 09:41:24.012326 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:24.012303 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:41:24.092963 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:24.092923 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nlgpz\" (UniqueName: \"kubernetes.io/projected/0aa03a53-6fc2-4952-8455-74b73bf14dd9-kube-api-access-nlgpz\") pod \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\" (UID: \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\") " Jul 13 09:41:24.092963 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:24.092958 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0aa03a53-6fc2-4952-8455-74b73bf14dd9-istio-podinfo\") pod \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\" (UID: \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\") " Jul 13 09:41:24.093278 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:24.093003 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0aa03a53-6fc2-4952-8455-74b73bf14dd9-istio-token\") pod \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\" (UID: \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\") " Jul 13 09:41:24.093278 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:24.093024 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0aa03a53-6fc2-4952-8455-74b73bf14dd9-workload-socket\") pod \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\" (UID: \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\") " Jul 13 09:41:24.093278 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:24.093051 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0aa03a53-6fc2-4952-8455-74b73bf14dd9-istio-data\") pod \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\" (UID: \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\") " Jul 13 09:41:24.093278 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:24.093069 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0aa03a53-6fc2-4952-8455-74b73bf14dd9-credential-socket\") pod \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\" (UID: \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\") " Jul 13 09:41:24.093278 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:24.093178 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0aa03a53-6fc2-4952-8455-74b73bf14dd9-workload-certs\") pod \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\" (UID: \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\") " Jul 13 09:41:24.093278 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:24.093200 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0aa03a53-6fc2-4952-8455-74b73bf14dd9-istio-envoy\") pod \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\" (UID: \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\") " Jul 13 09:41:24.093278 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:24.093253 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0aa03a53-6fc2-4952-8455-74b73bf14dd9-istiod-ca-cert\") pod \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\" (UID: \"0aa03a53-6fc2-4952-8455-74b73bf14dd9\") " Jul 13 09:41:24.093658 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:24.093329 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aa03a53-6fc2-4952-8455-74b73bf14dd9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0aa03a53-6fc2-4952-8455-74b73bf14dd9" (UID: "0aa03a53-6fc2-4952-8455-74b73bf14dd9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:41:24.093658 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:24.093440 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aa03a53-6fc2-4952-8455-74b73bf14dd9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0aa03a53-6fc2-4952-8455-74b73bf14dd9" (UID: "0aa03a53-6fc2-4952-8455-74b73bf14dd9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:41:24.093658 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:24.093576 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aa03a53-6fc2-4952-8455-74b73bf14dd9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0aa03a53-6fc2-4952-8455-74b73bf14dd9" (UID: "0aa03a53-6fc2-4952-8455-74b73bf14dd9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:41:24.093658 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:24.093610 2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0aa03a53-6fc2-4952-8455-74b73bf14dd9-workload-certs\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:41:24.093658 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:24.093642 2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0aa03a53-6fc2-4952-8455-74b73bf14dd9-workload-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:41:24.093658 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:24.093612 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0aa03a53-6fc2-4952-8455-74b73bf14dd9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0aa03a53-6fc2-4952-8455-74b73bf14dd9" (UID: "0aa03a53-6fc2-4952-8455-74b73bf14dd9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 09:41:24.094442 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:24.094399 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aa03a53-6fc2-4952-8455-74b73bf14dd9-istio-data" (OuterVolumeSpecName: "istio-data") pod "0aa03a53-6fc2-4952-8455-74b73bf14dd9" (UID: "0aa03a53-6fc2-4952-8455-74b73bf14dd9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:41:24.096256 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:24.096106 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aa03a53-6fc2-4952-8455-74b73bf14dd9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0aa03a53-6fc2-4952-8455-74b73bf14dd9" (UID: "0aa03a53-6fc2-4952-8455-74b73bf14dd9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:41:24.102233 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:24.097286 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0aa03a53-6fc2-4952-8455-74b73bf14dd9-kube-api-access-nlgpz" (OuterVolumeSpecName: "kube-api-access-nlgpz") pod "0aa03a53-6fc2-4952-8455-74b73bf14dd9" (UID: "0aa03a53-6fc2-4952-8455-74b73bf14dd9"). InnerVolumeSpecName "kube-api-access-nlgpz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:41:24.103350 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:24.103187 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0aa03a53-6fc2-4952-8455-74b73bf14dd9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0aa03a53-6fc2-4952-8455-74b73bf14dd9" (UID: "0aa03a53-6fc2-4952-8455-74b73bf14dd9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 09:41:24.104274 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:24.104250 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0aa03a53-6fc2-4952-8455-74b73bf14dd9-istio-token" (OuterVolumeSpecName: "istio-token") pod "0aa03a53-6fc2-4952-8455-74b73bf14dd9" (UID: "0aa03a53-6fc2-4952-8455-74b73bf14dd9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:41:24.194306 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:24.194277 2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0aa03a53-6fc2-4952-8455-74b73bf14dd9-istio-envoy\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:41:24.194306 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:24.194302 2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0aa03a53-6fc2-4952-8455-74b73bf14dd9-istiod-ca-cert\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:41:24.194467 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:24.194311 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nlgpz\" (UniqueName: \"kubernetes.io/projected/0aa03a53-6fc2-4952-8455-74b73bf14dd9-kube-api-access-nlgpz\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:41:24.194467 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:24.194322 2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0aa03a53-6fc2-4952-8455-74b73bf14dd9-istio-podinfo\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:41:24.194467 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:24.194331 2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0aa03a53-6fc2-4952-8455-74b73bf14dd9-istio-token\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:41:24.194467 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:24.194338 2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0aa03a53-6fc2-4952-8455-74b73bf14dd9-istio-data\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:41:24.194467 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:24.194346 2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0aa03a53-6fc2-4952-8455-74b73bf14dd9-credential-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:41:24.867018 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:24.865804 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" event={"ID":"0aa03a53-6fc2-4952-8455-74b73bf14dd9","Type":"ContainerDied","Data":"fa9d7ead5aca10045dae97e4f3115b7b3ff3a8b6ae96c3e1fb6ca80063172e5e"} Jul 13 09:41:24.867018 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:24.865870 2552 scope.go:117] "RemoveContainer" containerID="f09c5d0d81bc3e9d914fbd229b3e801c2a77e518b190065e5a2d0d5d3bcac2e0" Jul 13 09:41:24.867018 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:24.866025 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx" Jul 13 09:41:24.887687 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:24.887663 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx"] Jul 13 09:41:24.897472 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:24.897447 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-7694df9c-openshift-default-69457848b7-xhvcx"] Jul 13 09:41:26.667304 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:26.667273 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0aa03a53-6fc2-4952-8455-74b73bf14dd9" path="/var/lib/kubelet/pods/0aa03a53-6fc2-4952-8455-74b73bf14dd9/volumes" Jul 13 09:41:28.252856 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:28.252800 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:41:28.327424 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:28.327394 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:41:28.966534 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:28.966457 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:41:34.850652 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:34.850609 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:41:36.157995 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:36.157962 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:41:45.533296 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:45.533252 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:41:45.850542 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:41:45.850516 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:42:06.793928 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:06.793879 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:42:12.869655 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:12.869615 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:42:12.943381 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:12.943351 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:42:13.247823 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:13.247745 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:42:13.540780 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:13.540679 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh"] Jul 13 09:42:13.541289 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:13.541262 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0aa03a53-6fc2-4952-8455-74b73bf14dd9" containerName="istio-proxy" Jul 13 09:42:13.541289 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:13.541287 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="0aa03a53-6fc2-4952-8455-74b73bf14dd9" containerName="istio-proxy" Jul 13 09:42:13.541486 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:13.541415 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="0aa03a53-6fc2-4952-8455-74b73bf14dd9" containerName="istio-proxy" Jul 13 09:42:13.541486 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:13.541427 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="0aa03a53-6fc2-4952-8455-74b73bf14dd9" containerName="istio-proxy" Jul 13 09:42:13.544542 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:13.544525 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:42:13.548697 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:13.548676 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-c47dc5-openshift-default-dockercfg-4sdpt\"" Jul 13 09:42:13.574955 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:13.574934 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh"] Jul 13 09:42:13.682315 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:13.682285 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:42:13.709369 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:13.709341 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7573846-b279-4f0e-a424-ba8f57e0d7cf-istio-token\") pod \"e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh\" (UID: \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\") " pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:42:13.709518 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:13.709399 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7573846-b279-4f0e-a424-ba8f57e0d7cf-istiod-ca-cert\") pod \"e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh\" (UID: \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\") " pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:42:13.709518 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:13.709449 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xj9nh\" (UniqueName: \"kubernetes.io/projected/d7573846-b279-4f0e-a424-ba8f57e0d7cf-kube-api-access-xj9nh\") pod \"e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh\" (UID: \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\") " pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:42:13.709647 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:13.709514 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7573846-b279-4f0e-a424-ba8f57e0d7cf-workload-certs\") pod \"e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh\" (UID: \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\") " pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:42:13.709647 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:13.709561 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7573846-b279-4f0e-a424-ba8f57e0d7cf-istio-envoy\") pod \"e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh\" (UID: \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\") " pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:42:13.709647 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:13.709592 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7573846-b279-4f0e-a424-ba8f57e0d7cf-workload-socket\") pod \"e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh\" (UID: \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\") " pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:42:13.709647 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:13.709631 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7573846-b279-4f0e-a424-ba8f57e0d7cf-istio-data\") pod \"e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh\" (UID: \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\") " pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:42:13.709833 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:13.709692 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7573846-b279-4f0e-a424-ba8f57e0d7cf-credential-socket\") pod \"e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh\" (UID: \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\") " pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:42:13.709833 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:13.709721 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7573846-b279-4f0e-a424-ba8f57e0d7cf-istio-podinfo\") pod \"e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh\" (UID: \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\") " pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:42:13.810516 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:13.810448 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7573846-b279-4f0e-a424-ba8f57e0d7cf-istiod-ca-cert\") pod \"e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh\" (UID: \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\") " pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:42:13.810516 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:13.810488 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xj9nh\" (UniqueName: \"kubernetes.io/projected/d7573846-b279-4f0e-a424-ba8f57e0d7cf-kube-api-access-xj9nh\") pod \"e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh\" (UID: \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\") " pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:42:13.810695 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:13.810526 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7573846-b279-4f0e-a424-ba8f57e0d7cf-workload-certs\") pod \"e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh\" (UID: \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\") " pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:42:13.810695 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:13.810556 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7573846-b279-4f0e-a424-ba8f57e0d7cf-istio-envoy\") pod \"e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh\" (UID: \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\") " pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:42:13.810695 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:13.810584 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7573846-b279-4f0e-a424-ba8f57e0d7cf-workload-socket\") pod \"e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh\" (UID: \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\") " pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:42:13.810932 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:13.810899 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7573846-b279-4f0e-a424-ba8f57e0d7cf-workload-certs\") pod \"e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh\" (UID: \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\") " pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:42:13.811011 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:13.810955 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7573846-b279-4f0e-a424-ba8f57e0d7cf-workload-socket\") pod \"e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh\" (UID: \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\") " pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:42:13.811072 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:13.811059 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7573846-b279-4f0e-a424-ba8f57e0d7cf-istio-data\") pod \"e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh\" (UID: \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\") " pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:42:13.811142 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:13.811083 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7573846-b279-4f0e-a424-ba8f57e0d7cf-istiod-ca-cert\") pod \"e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh\" (UID: \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\") " pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:42:13.811203 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:13.811145 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7573846-b279-4f0e-a424-ba8f57e0d7cf-credential-socket\") pod \"e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh\" (UID: \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\") " pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:42:13.811203 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:13.811181 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7573846-b279-4f0e-a424-ba8f57e0d7cf-istio-podinfo\") pod \"e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh\" (UID: \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\") " pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:42:13.811335 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:13.811227 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7573846-b279-4f0e-a424-ba8f57e0d7cf-istio-token\") pod \"e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh\" (UID: \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\") " pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:42:13.811398 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:13.811337 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7573846-b279-4f0e-a424-ba8f57e0d7cf-istio-data\") pod \"e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh\" (UID: \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\") " pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:42:13.811490 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:13.811470 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7573846-b279-4f0e-a424-ba8f57e0d7cf-credential-socket\") pod \"e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh\" (UID: \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\") " pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:42:13.813342 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:13.813322 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7573846-b279-4f0e-a424-ba8f57e0d7cf-istio-envoy\") pod \"e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh\" (UID: \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\") " pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:42:13.813472 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:13.813457 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7573846-b279-4f0e-a424-ba8f57e0d7cf-istio-podinfo\") pod \"e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh\" (UID: \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\") " pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:42:13.822911 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:13.822875 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xj9nh\" (UniqueName: \"kubernetes.io/projected/d7573846-b279-4f0e-a424-ba8f57e0d7cf-kube-api-access-xj9nh\") pod \"e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh\" (UID: \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\") " pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:42:13.824347 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:13.824327 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7573846-b279-4f0e-a424-ba8f57e0d7cf-istio-token\") pod \"e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh\" (UID: \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\") " pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:42:13.855025 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:13.855002 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:42:14.006401 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:14.006370 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh"] Jul 13 09:42:14.007297 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:42:14.007266 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7573846_b279_4f0e_a424_ba8f57e0d7cf.slice/crio-ade7e70de6fa2beeac5e76b682e294a74b0fd3bd27dd2cc1d0d7141b8d62dafa WatchSource:0}: Error finding container ade7e70de6fa2beeac5e76b682e294a74b0fd3bd27dd2cc1d0d7141b8d62dafa: Status 404 returned error can't find the container with id ade7e70de6fa2beeac5e76b682e294a74b0fd3bd27dd2cc1d0d7141b8d62dafa Jul 13 09:42:14.009359 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:14.009329 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:42:14.009424 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:14.009391 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:42:14.009424 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:14.009419 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:42:14.137509 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:14.137471 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" event={"ID":"d7573846-b279-4f0e-a424-ba8f57e0d7cf","Type":"ContainerStarted","Data":"1c1938e2bcf88a009195d118dfba817ba853af8202f23c914124e3d36a2236bd"} Jul 13 09:42:14.137659 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:14.137570 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" event={"ID":"d7573846-b279-4f0e-a424-ba8f57e0d7cf","Type":"ContainerStarted","Data":"ade7e70de6fa2beeac5e76b682e294a74b0fd3bd27dd2cc1d0d7141b8d62dafa"} Jul 13 09:42:14.162098 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:14.162051 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" podStartSLOduration=1.162035385 podStartE2EDuration="1.162035385s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:42:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:42:14.159330724 +0000 UTC m=+2018.039286020" watchObservedRunningTime="2026-07-13 09:42:14.162035385 +0000 UTC m=+2018.041990651" Jul 13 09:42:14.182335 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:14.182304 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:42:14.856154 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:14.856126 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:42:15.860527 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:15.860495 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:42:16.149805 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:16.149716 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:42:16.150750 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:16.150730 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:42:16.213177 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:16.213149 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:42:16.307788 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:16.307753 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:42:16.403145 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:16.403045 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:42:16.871345 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:16.871317 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:42:16.969395 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:16.969368 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:42:20.825495 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:20.825464 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:42:21.244475 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.244443 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:42:21.453243 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.453196 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j"] Jul 13 09:42:21.454671 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.454645 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0aa03a53-6fc2-4952-8455-74b73bf14dd9" containerName="istio-proxy" Jul 13 09:42:21.454822 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.454811 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="0aa03a53-6fc2-4952-8455-74b73bf14dd9" containerName="istio-proxy" Jul 13 09:42:21.459656 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.459635 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:42:21.463300 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.463274 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-6f7934-openshift-default-dockercfg-c5xtj\"" Jul 13 09:42:21.466108 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.466081 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j"] Jul 13 09:42:21.488016 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.487995 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:42:21.580584 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.580559 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28982754-c105-4e29-972f-331557d904e4-istio-envoy\") pod \"e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j\" (UID: \"28982754-c105-4e29-972f-331557d904e4\") " pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:42:21.580750 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.580646 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28982754-c105-4e29-972f-331557d904e4-workload-socket\") pod \"e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j\" (UID: \"28982754-c105-4e29-972f-331557d904e4\") " pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:42:21.580750 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.580701 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28982754-c105-4e29-972f-331557d904e4-istio-token\") pod \"e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j\" (UID: \"28982754-c105-4e29-972f-331557d904e4\") " pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:42:21.580842 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.580793 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28982754-c105-4e29-972f-331557d904e4-workload-certs\") pod \"e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j\" (UID: \"28982754-c105-4e29-972f-331557d904e4\") " pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:42:21.580885 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.580831 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28982754-c105-4e29-972f-331557d904e4-istio-podinfo\") pod \"e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j\" (UID: \"28982754-c105-4e29-972f-331557d904e4\") " pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:42:21.580885 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.580878 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28982754-c105-4e29-972f-331557d904e4-istiod-ca-cert\") pod \"e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j\" (UID: \"28982754-c105-4e29-972f-331557d904e4\") " pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:42:21.580982 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.580913 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fx8l\" (UniqueName: \"kubernetes.io/projected/28982754-c105-4e29-972f-331557d904e4-kube-api-access-8fx8l\") pod \"e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j\" (UID: \"28982754-c105-4e29-972f-331557d904e4\") " pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:42:21.580982 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.580945 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28982754-c105-4e29-972f-331557d904e4-credential-socket\") pod \"e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j\" (UID: \"28982754-c105-4e29-972f-331557d904e4\") " pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:42:21.580982 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.580974 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28982754-c105-4e29-972f-331557d904e4-istio-data\") pod \"e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j\" (UID: \"28982754-c105-4e29-972f-331557d904e4\") " pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:42:21.681593 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.681556 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28982754-c105-4e29-972f-331557d904e4-workload-certs\") pod \"e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j\" (UID: \"28982754-c105-4e29-972f-331557d904e4\") " pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:42:21.681593 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.681594 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28982754-c105-4e29-972f-331557d904e4-istio-podinfo\") pod \"e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j\" (UID: \"28982754-c105-4e29-972f-331557d904e4\") " pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:42:21.681821 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.681616 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28982754-c105-4e29-972f-331557d904e4-istiod-ca-cert\") pod \"e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j\" (UID: \"28982754-c105-4e29-972f-331557d904e4\") " pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:42:21.681821 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.681666 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fx8l\" (UniqueName: \"kubernetes.io/projected/28982754-c105-4e29-972f-331557d904e4-kube-api-access-8fx8l\") pod \"e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j\" (UID: \"28982754-c105-4e29-972f-331557d904e4\") " pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:42:21.681821 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.681707 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28982754-c105-4e29-972f-331557d904e4-credential-socket\") pod \"e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j\" (UID: \"28982754-c105-4e29-972f-331557d904e4\") " pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:42:21.681821 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.681736 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28982754-c105-4e29-972f-331557d904e4-istio-data\") pod \"e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j\" (UID: \"28982754-c105-4e29-972f-331557d904e4\") " pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:42:21.681821 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.681778 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28982754-c105-4e29-972f-331557d904e4-istio-envoy\") pod \"e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j\" (UID: \"28982754-c105-4e29-972f-331557d904e4\") " pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:42:21.682091 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.681859 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28982754-c105-4e29-972f-331557d904e4-workload-socket\") pod \"e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j\" (UID: \"28982754-c105-4e29-972f-331557d904e4\") " pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:42:21.682091 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.681910 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28982754-c105-4e29-972f-331557d904e4-istio-token\") pod \"e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j\" (UID: \"28982754-c105-4e29-972f-331557d904e4\") " pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:42:21.682202 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.682175 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28982754-c105-4e29-972f-331557d904e4-credential-socket\") pod \"e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j\" (UID: \"28982754-c105-4e29-972f-331557d904e4\") " pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:42:21.682285 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.682199 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28982754-c105-4e29-972f-331557d904e4-workload-certs\") pod \"e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j\" (UID: \"28982754-c105-4e29-972f-331557d904e4\") " pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:42:21.682491 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.682471 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28982754-c105-4e29-972f-331557d904e4-workload-socket\") pod \"e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j\" (UID: \"28982754-c105-4e29-972f-331557d904e4\") " pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:42:21.682686 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.682661 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28982754-c105-4e29-972f-331557d904e4-istiod-ca-cert\") pod \"e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j\" (UID: \"28982754-c105-4e29-972f-331557d904e4\") " pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:42:21.682940 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.682916 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28982754-c105-4e29-972f-331557d904e4-istio-data\") pod \"e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j\" (UID: \"28982754-c105-4e29-972f-331557d904e4\") " pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:42:21.684168 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.684146 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28982754-c105-4e29-972f-331557d904e4-istio-envoy\") pod \"e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j\" (UID: \"28982754-c105-4e29-972f-331557d904e4\") " pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:42:21.684460 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.684441 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28982754-c105-4e29-972f-331557d904e4-istio-podinfo\") pod \"e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j\" (UID: \"28982754-c105-4e29-972f-331557d904e4\") " pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:42:21.697403 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.697375 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28982754-c105-4e29-972f-331557d904e4-istio-token\") pod \"e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j\" (UID: \"28982754-c105-4e29-972f-331557d904e4\") " pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:42:21.697600 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.697578 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fx8l\" (UniqueName: \"kubernetes.io/projected/28982754-c105-4e29-972f-331557d904e4-kube-api-access-8fx8l\") pod \"e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j\" (UID: \"28982754-c105-4e29-972f-331557d904e4\") " pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:42:21.774411 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.774385 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:42:21.931413 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.931385 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j"] Jul 13 09:42:21.933399 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:42:21.933357 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28982754_c105_4e29_972f_331557d904e4.slice/crio-07ebfcc998ba9823792703720c6af0f0a3b4a92961db279984f7655d537f94ec WatchSource:0}: Error finding container 07ebfcc998ba9823792703720c6af0f0a3b4a92961db279984f7655d537f94ec: Status 404 returned error can't find the container with id 07ebfcc998ba9823792703720c6af0f0a3b4a92961db279984f7655d537f94ec Jul 13 09:42:21.935508 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.935478 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:42:21.935587 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.935542 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:42:21.935587 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:21.935574 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:42:22.184914 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:22.184816 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" event={"ID":"28982754-c105-4e29-972f-331557d904e4","Type":"ContainerStarted","Data":"db95e4978ca9c859b782c04f56b0dadeb982477ea93d8f1d935b9275af750c9a"} Jul 13 09:42:22.184914 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:22.184875 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" event={"ID":"28982754-c105-4e29-972f-331557d904e4","Type":"ContainerStarted","Data":"07ebfcc998ba9823792703720c6af0f0a3b4a92961db279984f7655d537f94ec"} Jul 13 09:42:22.209981 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:22.209933 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" podStartSLOduration=1.209916974 podStartE2EDuration="1.209916974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:42:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:42:22.207341385 +0000 UTC m=+2026.087296652" watchObservedRunningTime="2026-07-13 09:42:22.209916974 +0000 UTC m=+2026.089872241" Jul 13 09:42:22.277322 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:22.277296 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:42:22.775116 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:22.775087 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:42:23.794658 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:23.794625 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:42:24.195724 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:24.195692 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:42:24.196755 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:24.196736 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:42:24.239827 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:24.239788 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:42:24.345288 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:24.345255 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:42:24.445008 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:24.444980 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:42:24.981143 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:24.981100 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:42:25.077662 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:25.077607 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:42:28.948031 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:28.947992 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:42:31.955382 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:31.954897 2552 patch_prober.go:28] interesting pod/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": read tcp 10.133.0.2:49224->10.133.0.63:15021: read: connection reset by peer" start-of-body= Jul 13 09:42:31.955382 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:31.954978 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" podUID="d7573846-b279-4f0e-a424-ba8f57e0d7cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": read tcp 10.133.0.2:49224->10.133.0.63:15021: read: connection reset by peer" Jul 13 09:42:32.250599 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:32.250504 2552 generic.go:356] "Generic (PLEG): container finished" podID="d7573846-b279-4f0e-a424-ba8f57e0d7cf" containerID="1c1938e2bcf88a009195d118dfba817ba853af8202f23c914124e3d36a2236bd" exitCode=0 Jul 13 09:42:32.250599 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:32.250572 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" event={"ID":"d7573846-b279-4f0e-a424-ba8f57e0d7cf","Type":"ContainerDied","Data":"1c1938e2bcf88a009195d118dfba817ba853af8202f23c914124e3d36a2236bd"} Jul 13 09:42:32.250969 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:32.250952 2552 scope.go:117] "RemoveContainer" containerID="1c1938e2bcf88a009195d118dfba817ba853af8202f23c914124e3d36a2236bd" Jul 13 09:42:32.251922 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:32.251894 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:42:32.252025 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:32.251954 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:42:32.252025 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:32.251992 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:42:33.263424 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:33.263384 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" event={"ID":"d7573846-b279-4f0e-a424-ba8f57e0d7cf","Type":"ContainerStarted","Data":"f1f65fc61d16952b6f0942ec4467d28a1791b2dec6c1542eac641ad50f54e28f"} Jul 13 09:42:33.855546 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:33.855511 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:42:33.855737 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:33.855715 2552 patch_prober.go:28] interesting pod/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 13 09:42:33.855790 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:33.855766 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" podUID="d7573846-b279-4f0e-a424-ba8f57e0d7cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 13 09:42:34.856064 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:34.856037 2552 patch_prober.go:28] interesting pod/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 13 09:42:34.856543 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:34.856517 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" podUID="d7573846-b279-4f0e-a424-ba8f57e0d7cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 13 09:42:35.855957 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:35.855930 2552 patch_prober.go:28] interesting pod/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 13 09:42:35.856194 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:35.856168 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" podUID="d7573846-b279-4f0e-a424-ba8f57e0d7cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 13 09:42:36.856223 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:36.856175 2552 patch_prober.go:28] interesting pod/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 13 09:42:36.856659 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:36.856272 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" podUID="d7573846-b279-4f0e-a424-ba8f57e0d7cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 13 09:42:37.808913 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:37.808869 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:42:37.860310 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:37.860275 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:42:37.860757 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:37.860702 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:42:37.861466 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:37.861444 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:42:37.920129 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:37.920093 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:42:39.904530 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:39.903748 2552 patch_prober.go:28] interesting pod/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": read tcp 10.133.0.2:35012->10.133.0.64:15021: read: connection reset by peer" start-of-body= Jul 13 09:42:39.904530 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:39.903832 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" podUID="28982754-c105-4e29-972f-331557d904e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": read tcp 10.133.0.2:35012->10.133.0.64:15021: read: connection reset by peer" Jul 13 09:42:40.319372 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:40.319284 2552 generic.go:356] "Generic (PLEG): container finished" podID="28982754-c105-4e29-972f-331557d904e4" containerID="db95e4978ca9c859b782c04f56b0dadeb982477ea93d8f1d935b9275af750c9a" exitCode=0 Jul 13 09:42:40.319513 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:40.319357 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" event={"ID":"28982754-c105-4e29-972f-331557d904e4","Type":"ContainerDied","Data":"db95e4978ca9c859b782c04f56b0dadeb982477ea93d8f1d935b9275af750c9a"} Jul 13 09:42:40.319758 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:40.319744 2552 scope.go:117] "RemoveContainer" containerID="db95e4978ca9c859b782c04f56b0dadeb982477ea93d8f1d935b9275af750c9a" Jul 13 09:42:40.320797 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:40.320771 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:42:40.320857 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:40.320840 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:42:40.320893 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:40.320873 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:42:40.396552 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:40.396526 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:42:41.327542 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:41.327506 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" event={"ID":"28982754-c105-4e29-972f-331557d904e4","Type":"ContainerStarted","Data":"a89aed0f7c5901eaf28161e85dcb8ea6525b9c2148177598909bc7ba791379bf"} Jul 13 09:42:41.774854 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:41.774765 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:42:41.775232 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:41.775113 2552 patch_prober.go:28] interesting pod/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Jul 13 09:42:41.775399 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:41.775376 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" podUID="28982754-c105-4e29-972f-331557d904e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Jul 13 09:42:42.775378 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:42.775345 2552 patch_prober.go:28] interesting pod/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Jul 13 09:42:42.775949 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:42.775915 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" podUID="28982754-c105-4e29-972f-331557d904e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Jul 13 09:42:43.775677 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:43.775260 2552 patch_prober.go:28] interesting pod/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Jul 13 09:42:43.775677 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:43.775319 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" podUID="28982754-c105-4e29-972f-331557d904e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Jul 13 09:42:44.775749 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:44.775706 2552 patch_prober.go:28] interesting pod/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Jul 13 09:42:44.776260 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:44.775771 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" podUID="28982754-c105-4e29-972f-331557d904e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Jul 13 09:42:45.697494 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:45.697455 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:42:45.778045 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:45.778017 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:42:45.778486 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:45.778257 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:42:45.778907 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:45.778892 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:42:45.794604 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:45.794582 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:42:59.495959 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:42:59.495885 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:43:17.522124 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:43:17.522085 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:43:18.394998 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:43:18.394966 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:43:18.580013 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:43:18.579976 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:43:19.024857 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:43:19.024827 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:43:19.124788 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:43:19.124759 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:43:19.225519 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:43:19.225489 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:43:19.342324 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:43:19.342294 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:43:24.084472 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:43:24.084443 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:43:36.754921 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:43:36.754893 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gp6mv_47250d22-56b6-4503-af3f-36a9e79691b2/ovn-acl-logging/0.log" Jul 13 09:43:36.763546 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:43:36.763522 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gp6mv_47250d22-56b6-4503-af3f-36a9e79691b2/ovn-acl-logging/0.log" Jul 13 09:44:00.984696 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:00.984647 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:01.236276 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:01.236177 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh"] Jul 13 09:44:01.236443 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:01.236391 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" podUID="d7573846-b279-4f0e-a424-ba8f57e0d7cf" containerName="istio-proxy" containerID="cri-o://f1f65fc61d16952b6f0942ec4467d28a1791b2dec6c1542eac641ad50f54e28f" gracePeriod=30 Jul 13 09:44:01.270562 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:01.270529 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:01.677293 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:01.677257 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:02.132496 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:02.132451 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:02.233863 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:02.233835 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:02.336045 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:02.336011 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:02.453392 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:02.453289 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:06.490495 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.490466 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:44:06.574268 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.574163 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7573846-b279-4f0e-a424-ba8f57e0d7cf-workload-socket\") pod \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\" (UID: \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\") " Jul 13 09:44:06.574268 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.574248 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7573846-b279-4f0e-a424-ba8f57e0d7cf-istio-envoy\") pod \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\" (UID: \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\") " Jul 13 09:44:06.574489 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.574275 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7573846-b279-4f0e-a424-ba8f57e0d7cf-credential-socket\") pod \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\" (UID: \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\") " Jul 13 09:44:06.574489 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.574309 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7573846-b279-4f0e-a424-ba8f57e0d7cf-workload-certs\") pod \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\" (UID: \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\") " Jul 13 09:44:06.574489 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.574337 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7573846-b279-4f0e-a424-ba8f57e0d7cf-istio-podinfo\") pod \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\" (UID: \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\") " Jul 13 09:44:06.574489 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.574363 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7573846-b279-4f0e-a424-ba8f57e0d7cf-istiod-ca-cert\") pod \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\" (UID: \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\") " Jul 13 09:44:06.574489 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.574407 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7573846-b279-4f0e-a424-ba8f57e0d7cf-istio-token\") pod \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\" (UID: \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\") " Jul 13 09:44:06.574489 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.574434 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xj9nh\" (UniqueName: \"kubernetes.io/projected/d7573846-b279-4f0e-a424-ba8f57e0d7cf-kube-api-access-xj9nh\") pod \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\" (UID: \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\") " Jul 13 09:44:06.574489 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.574444 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7573846-b279-4f0e-a424-ba8f57e0d7cf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d7573846-b279-4f0e-a424-ba8f57e0d7cf" (UID: "d7573846-b279-4f0e-a424-ba8f57e0d7cf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:44:06.574872 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.574554 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7573846-b279-4f0e-a424-ba8f57e0d7cf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d7573846-b279-4f0e-a424-ba8f57e0d7cf" (UID: "d7573846-b279-4f0e-a424-ba8f57e0d7cf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:44:06.574872 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.574572 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7573846-b279-4f0e-a424-ba8f57e0d7cf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d7573846-b279-4f0e-a424-ba8f57e0d7cf" (UID: "d7573846-b279-4f0e-a424-ba8f57e0d7cf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:44:06.574872 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.574620 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7573846-b279-4f0e-a424-ba8f57e0d7cf-istio-data\") pod \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\" (UID: \"d7573846-b279-4f0e-a424-ba8f57e0d7cf\") " Jul 13 09:44:06.574872 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.574806 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7573846-b279-4f0e-a424-ba8f57e0d7cf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d7573846-b279-4f0e-a424-ba8f57e0d7cf" (UID: "d7573846-b279-4f0e-a424-ba8f57e0d7cf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 09:44:06.575084 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.575027 2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7573846-b279-4f0e-a424-ba8f57e0d7cf-workload-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:44:06.575084 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.575045 2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7573846-b279-4f0e-a424-ba8f57e0d7cf-credential-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:44:06.575084 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.575059 2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7573846-b279-4f0e-a424-ba8f57e0d7cf-workload-certs\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:44:06.575084 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.575072 2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7573846-b279-4f0e-a424-ba8f57e0d7cf-istiod-ca-cert\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:44:06.575322 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.575298 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7573846-b279-4f0e-a424-ba8f57e0d7cf-istio-data" (OuterVolumeSpecName: "istio-data") pod "d7573846-b279-4f0e-a424-ba8f57e0d7cf" (UID: "d7573846-b279-4f0e-a424-ba8f57e0d7cf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:44:06.576616 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.576596 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7573846-b279-4f0e-a424-ba8f57e0d7cf-kube-api-access-xj9nh" (OuterVolumeSpecName: "kube-api-access-xj9nh") pod "d7573846-b279-4f0e-a424-ba8f57e0d7cf" (UID: "d7573846-b279-4f0e-a424-ba8f57e0d7cf"). InnerVolumeSpecName "kube-api-access-xj9nh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:44:06.576726 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.576622 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7573846-b279-4f0e-a424-ba8f57e0d7cf-istio-token" (OuterVolumeSpecName: "istio-token") pod "d7573846-b279-4f0e-a424-ba8f57e0d7cf" (UID: "d7573846-b279-4f0e-a424-ba8f57e0d7cf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:44:06.576864 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.576849 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d7573846-b279-4f0e-a424-ba8f57e0d7cf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d7573846-b279-4f0e-a424-ba8f57e0d7cf" (UID: "d7573846-b279-4f0e-a424-ba8f57e0d7cf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 09:44:06.576905 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.576874 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7573846-b279-4f0e-a424-ba8f57e0d7cf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d7573846-b279-4f0e-a424-ba8f57e0d7cf" (UID: "d7573846-b279-4f0e-a424-ba8f57e0d7cf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:44:06.675606 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.675577 2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7573846-b279-4f0e-a424-ba8f57e0d7cf-istio-envoy\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:44:06.675606 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.675602 2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7573846-b279-4f0e-a424-ba8f57e0d7cf-istio-podinfo\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:44:06.675606 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.675611 2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7573846-b279-4f0e-a424-ba8f57e0d7cf-istio-token\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:44:06.675788 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.675619 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xj9nh\" (UniqueName: \"kubernetes.io/projected/d7573846-b279-4f0e-a424-ba8f57e0d7cf-kube-api-access-xj9nh\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:44:06.675788 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.675628 2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7573846-b279-4f0e-a424-ba8f57e0d7cf-istio-data\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:44:06.816169 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.816140 2552 generic.go:356] "Generic (PLEG): container finished" podID="d7573846-b279-4f0e-a424-ba8f57e0d7cf" containerID="f1f65fc61d16952b6f0942ec4467d28a1791b2dec6c1542eac641ad50f54e28f" exitCode=0 Jul 13 09:44:06.816311 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.816185 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" event={"ID":"d7573846-b279-4f0e-a424-ba8f57e0d7cf","Type":"ContainerDied","Data":"f1f65fc61d16952b6f0942ec4467d28a1791b2dec6c1542eac641ad50f54e28f"} Jul 13 09:44:06.816311 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.816187 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" Jul 13 09:44:06.816311 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.816258 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh" event={"ID":"d7573846-b279-4f0e-a424-ba8f57e0d7cf","Type":"ContainerDied","Data":"ade7e70de6fa2beeac5e76b682e294a74b0fd3bd27dd2cc1d0d7141b8d62dafa"} Jul 13 09:44:06.816311 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.816281 2552 scope.go:117] "RemoveContainer" containerID="f1f65fc61d16952b6f0942ec4467d28a1791b2dec6c1542eac641ad50f54e28f" Jul 13 09:44:06.825523 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.825507 2552 scope.go:117] "RemoveContainer" containerID="1c1938e2bcf88a009195d118dfba817ba853af8202f23c914124e3d36a2236bd" Jul 13 09:44:06.833745 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.833726 2552 scope.go:117] "RemoveContainer" containerID="f1f65fc61d16952b6f0942ec4467d28a1791b2dec6c1542eac641ad50f54e28f" Jul 13 09:44:06.834007 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:44:06.833983 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1f65fc61d16952b6f0942ec4467d28a1791b2dec6c1542eac641ad50f54e28f\": container with ID starting with f1f65fc61d16952b6f0942ec4467d28a1791b2dec6c1542eac641ad50f54e28f not found: ID does not exist" containerID="f1f65fc61d16952b6f0942ec4467d28a1791b2dec6c1542eac641ad50f54e28f" Jul 13 09:44:06.834124 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.834015 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1f65fc61d16952b6f0942ec4467d28a1791b2dec6c1542eac641ad50f54e28f"} err="failed to get container status \"f1f65fc61d16952b6f0942ec4467d28a1791b2dec6c1542eac641ad50f54e28f\": rpc error: code = NotFound desc = could not find container \"f1f65fc61d16952b6f0942ec4467d28a1791b2dec6c1542eac641ad50f54e28f\": container with ID starting with f1f65fc61d16952b6f0942ec4467d28a1791b2dec6c1542eac641ad50f54e28f not found: ID does not exist" Jul 13 09:44:06.834124 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.834037 2552 scope.go:117] "RemoveContainer" containerID="1c1938e2bcf88a009195d118dfba817ba853af8202f23c914124e3d36a2236bd" Jul 13 09:44:06.834340 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:44:06.834322 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c1938e2bcf88a009195d118dfba817ba853af8202f23c914124e3d36a2236bd\": container with ID starting with 1c1938e2bcf88a009195d118dfba817ba853af8202f23c914124e3d36a2236bd not found: ID does not exist" containerID="1c1938e2bcf88a009195d118dfba817ba853af8202f23c914124e3d36a2236bd" Jul 13 09:44:06.834390 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.834347 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c1938e2bcf88a009195d118dfba817ba853af8202f23c914124e3d36a2236bd"} err="failed to get container status \"1c1938e2bcf88a009195d118dfba817ba853af8202f23c914124e3d36a2236bd\": rpc error: code = NotFound desc = could not find container \"1c1938e2bcf88a009195d118dfba817ba853af8202f23c914124e3d36a2236bd\": container with ID starting with 1c1938e2bcf88a009195d118dfba817ba853af8202f23c914124e3d36a2236bd not found: ID does not exist" Jul 13 09:44:06.835415 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.835399 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh"] Jul 13 09:44:06.839015 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:06.838993 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-c47dc5-openshift-default-565b8d4bc7-mpqhh"] Jul 13 09:44:07.844878 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:07.844840 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:07.911879 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:07.911849 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:08.092267 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:08.092236 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j"] Jul 13 09:44:08.092473 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:08.092448 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" podUID="28982754-c105-4e29-972f-331557d904e4" containerName="istio-proxy" containerID="cri-o://a89aed0f7c5901eaf28161e85dcb8ea6525b9c2148177598909bc7ba791379bf" gracePeriod=30 Jul 13 09:44:08.107603 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:08.107536 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:08.667538 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:08.667505 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7573846-b279-4f0e-a424-ba8f57e0d7cf" path="/var/lib/kubelet/pods/d7573846-b279-4f0e-a424-ba8f57e0d7cf/volumes" Jul 13 09:44:08.729464 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:08.729429 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:08.971618 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:08.971521 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2"] Jul 13 09:44:08.972140 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:08.972116 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7573846-b279-4f0e-a424-ba8f57e0d7cf" containerName="istio-proxy" Jul 13 09:44:08.972298 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:08.972228 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7573846-b279-4f0e-a424-ba8f57e0d7cf" containerName="istio-proxy" Jul 13 09:44:08.972366 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:08.972296 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7573846-b279-4f0e-a424-ba8f57e0d7cf" containerName="istio-proxy" Jul 13 09:44:08.972366 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:08.972308 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7573846-b279-4f0e-a424-ba8f57e0d7cf" containerName="istio-proxy" Jul 13 09:44:08.972469 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:08.972445 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7573846-b279-4f0e-a424-ba8f57e0d7cf" containerName="istio-proxy" Jul 13 09:44:08.972469 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:08.972460 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7573846-b279-4f0e-a424-ba8f57e0d7cf" containerName="istio-proxy" Jul 13 09:44:08.977017 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:08.976996 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:08.979665 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:08.979647 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-c45f95-openshift-default-dockercfg-hzcgk\"" Jul 13 09:44:08.988000 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:08.987980 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2"] Jul 13 09:44:08.995688 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:08.995665 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:09.098506 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:09.098474 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwd7w\" (UniqueName: \"kubernetes.io/projected/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-kube-api-access-zwd7w\") pod \"e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2\" (UID: \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\") " pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:09.098689 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:09.098510 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-istio-podinfo\") pod \"e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2\" (UID: \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\") " pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:09.098689 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:09.098531 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-istio-envoy\") pod \"e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2\" (UID: \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\") " pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:09.098689 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:09.098558 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-istio-data\") pod \"e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2\" (UID: \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\") " pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:09.098689 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:09.098643 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-istio-token\") pod \"e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2\" (UID: \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\") " pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:09.098689 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:09.098677 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-istiod-ca-cert\") pod \"e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2\" (UID: \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\") " pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:09.098934 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:09.098695 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-credential-socket\") pod \"e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2\" (UID: \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\") " pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:09.098934 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:09.098711 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-workload-certs\") pod \"e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2\" (UID: \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\") " pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:09.098934 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:09.098737 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-workload-socket\") pod \"e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2\" (UID: \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\") " pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:09.199234 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:09.199173 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-istio-podinfo\") pod \"e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2\" (UID: \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\") " pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:09.199234 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:09.199221 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-istio-envoy\") pod \"e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2\" (UID: \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\") " pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:09.199483 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:09.199247 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-istio-data\") pod \"e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2\" (UID: \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\") " pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:09.199483 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:09.199277 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-istio-token\") pod \"e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2\" (UID: \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\") " pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:09.199483 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:09.199297 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-istiod-ca-cert\") pod \"e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2\" (UID: \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\") " pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:09.199483 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:09.199313 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-credential-socket\") pod \"e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2\" (UID: \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\") " pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:09.199483 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:09.199327 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-workload-certs\") pod \"e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2\" (UID: \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\") " pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:09.199483 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:09.199357 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-workload-socket\") pod \"e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2\" (UID: \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\") " pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:09.199483 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:09.199462 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwd7w\" (UniqueName: \"kubernetes.io/projected/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-kube-api-access-zwd7w\") pod \"e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2\" (UID: \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\") " pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:09.199871 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:09.199712 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-istio-data\") pod \"e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2\" (UID: \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\") " pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:09.199871 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:09.199768 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-credential-socket\") pod \"e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2\" (UID: \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\") " pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:09.199966 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:09.199914 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-workload-certs\") pod \"e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2\" (UID: \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\") " pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:09.200016 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:09.199993 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-workload-socket\") pod \"e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2\" (UID: \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\") " pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:09.200321 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:09.200299 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-istiod-ca-cert\") pod \"e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2\" (UID: \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\") " pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:09.201573 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:09.201552 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-istio-envoy\") pod \"e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2\" (UID: \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\") " pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:09.201869 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:09.201847 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-istio-podinfo\") pod \"e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2\" (UID: \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\") " pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:09.208108 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:09.208082 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-istio-token\") pod \"e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2\" (UID: \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\") " pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:09.208409 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:09.208388 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwd7w\" (UniqueName: \"kubernetes.io/projected/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-kube-api-access-zwd7w\") pod \"e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2\" (UID: \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\") " pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:09.288954 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:09.288896 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:09.431377 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:09.431352 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2"] Jul 13 09:44:09.433206 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:44:09.433177 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad53ae97_49e4_4a27_bd1f_72c50f355ff7.slice/crio-f4dad6ebff06254f96ba14d0bfce5e8f22ca29a9ff8827c396addcf68c73b91d WatchSource:0}: Error finding container f4dad6ebff06254f96ba14d0bfce5e8f22ca29a9ff8827c396addcf68c73b91d: Status 404 returned error can't find the container with id f4dad6ebff06254f96ba14d0bfce5e8f22ca29a9ff8827c396addcf68c73b91d Jul 13 09:44:09.436124 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:09.436082 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:44:09.436243 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:09.436178 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:44:09.436295 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:09.436244 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:44:09.485978 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:09.485953 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:09.837459 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:09.837416 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" event={"ID":"ad53ae97-49e4-4a27-bd1f-72c50f355ff7","Type":"ContainerStarted","Data":"80e31ce37f2dc1bcaa4c7150943c5dbd0cbce0632905211dcef283e6a3eeaea4"} Jul 13 09:44:09.837652 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:09.837491 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" event={"ID":"ad53ae97-49e4-4a27-bd1f-72c50f355ff7","Type":"ContainerStarted","Data":"f4dad6ebff06254f96ba14d0bfce5e8f22ca29a9ff8827c396addcf68c73b91d"} Jul 13 09:44:09.863728 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:09.863678 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" podStartSLOduration=1.863662001 podStartE2EDuration="1.863662001s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:44:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:44:09.862233983 +0000 UTC m=+2133.742189250" watchObservedRunningTime="2026-07-13 09:44:09.863662001 +0000 UTC m=+2133.743617336" Jul 13 09:44:10.290124 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:10.290028 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:10.294755 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:10.294725 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:10.843369 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:10.843335 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:10.844188 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:10.844172 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:11.787524 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:11.787467 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:11.891972 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:11.891932 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:11.986409 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:11.986374 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:12.085274 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:12.085242 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:12.431281 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:12.431171 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:12.528056 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:12.528023 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:13.344968 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.344944 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:44:13.441500 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.441427 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28982754-c105-4e29-972f-331557d904e4-istio-token\") pod \"28982754-c105-4e29-972f-331557d904e4\" (UID: \"28982754-c105-4e29-972f-331557d904e4\") " Jul 13 09:44:13.441500 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.441465 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28982754-c105-4e29-972f-331557d904e4-istio-envoy\") pod \"28982754-c105-4e29-972f-331557d904e4\" (UID: \"28982754-c105-4e29-972f-331557d904e4\") " Jul 13 09:44:13.441689 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.441503 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28982754-c105-4e29-972f-331557d904e4-istio-podinfo\") pod \"28982754-c105-4e29-972f-331557d904e4\" (UID: \"28982754-c105-4e29-972f-331557d904e4\") " Jul 13 09:44:13.441689 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.441527 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28982754-c105-4e29-972f-331557d904e4-workload-certs\") pod \"28982754-c105-4e29-972f-331557d904e4\" (UID: \"28982754-c105-4e29-972f-331557d904e4\") " Jul 13 09:44:13.441689 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.441609 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28982754-c105-4e29-972f-331557d904e4-workload-socket\") pod \"28982754-c105-4e29-972f-331557d904e4\" (UID: \"28982754-c105-4e29-972f-331557d904e4\") " Jul 13 09:44:13.441689 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.441664 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28982754-c105-4e29-972f-331557d904e4-istio-data\") pod \"28982754-c105-4e29-972f-331557d904e4\" (UID: \"28982754-c105-4e29-972f-331557d904e4\") " Jul 13 09:44:13.441903 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.441697 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28982754-c105-4e29-972f-331557d904e4-credential-socket\") pod \"28982754-c105-4e29-972f-331557d904e4\" (UID: \"28982754-c105-4e29-972f-331557d904e4\") " Jul 13 09:44:13.441903 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.441726 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8fx8l\" (UniqueName: \"kubernetes.io/projected/28982754-c105-4e29-972f-331557d904e4-kube-api-access-8fx8l\") pod \"28982754-c105-4e29-972f-331557d904e4\" (UID: \"28982754-c105-4e29-972f-331557d904e4\") " Jul 13 09:44:13.441903 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.441762 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28982754-c105-4e29-972f-331557d904e4-istiod-ca-cert\") pod \"28982754-c105-4e29-972f-331557d904e4\" (UID: \"28982754-c105-4e29-972f-331557d904e4\") " Jul 13 09:44:13.442057 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.441911 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28982754-c105-4e29-972f-331557d904e4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "28982754-c105-4e29-972f-331557d904e4" (UID: "28982754-c105-4e29-972f-331557d904e4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:44:13.442057 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.441950 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28982754-c105-4e29-972f-331557d904e4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "28982754-c105-4e29-972f-331557d904e4" (UID: "28982754-c105-4e29-972f-331557d904e4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:44:13.442168 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.442074 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28982754-c105-4e29-972f-331557d904e4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "28982754-c105-4e29-972f-331557d904e4" (UID: "28982754-c105-4e29-972f-331557d904e4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:44:13.442168 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.442122 2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28982754-c105-4e29-972f-331557d904e4-workload-certs\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:44:13.442168 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.442142 2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28982754-c105-4e29-972f-331557d904e4-workload-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:44:13.442370 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.442279 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28982754-c105-4e29-972f-331557d904e4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "28982754-c105-4e29-972f-331557d904e4" (UID: "28982754-c105-4e29-972f-331557d904e4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 09:44:13.442855 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.442825 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28982754-c105-4e29-972f-331557d904e4-istio-data" (OuterVolumeSpecName: "istio-data") pod "28982754-c105-4e29-972f-331557d904e4" (UID: "28982754-c105-4e29-972f-331557d904e4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:44:13.443714 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.443691 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28982754-c105-4e29-972f-331557d904e4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "28982754-c105-4e29-972f-331557d904e4" (UID: "28982754-c105-4e29-972f-331557d904e4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:44:13.443803 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.443739 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28982754-c105-4e29-972f-331557d904e4-istio-token" (OuterVolumeSpecName: "istio-token") pod "28982754-c105-4e29-972f-331557d904e4" (UID: "28982754-c105-4e29-972f-331557d904e4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:44:13.443847 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.443818 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/28982754-c105-4e29-972f-331557d904e4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "28982754-c105-4e29-972f-331557d904e4" (UID: "28982754-c105-4e29-972f-331557d904e4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 09:44:13.443942 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.443927 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28982754-c105-4e29-972f-331557d904e4-kube-api-access-8fx8l" (OuterVolumeSpecName: "kube-api-access-8fx8l") pod "28982754-c105-4e29-972f-331557d904e4" (UID: "28982754-c105-4e29-972f-331557d904e4"). InnerVolumeSpecName "kube-api-access-8fx8l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:44:13.542779 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.542750 2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28982754-c105-4e29-972f-331557d904e4-istio-data\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:44:13.542779 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.542780 2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28982754-c105-4e29-972f-331557d904e4-credential-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:44:13.542943 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.542794 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fx8l\" (UniqueName: \"kubernetes.io/projected/28982754-c105-4e29-972f-331557d904e4-kube-api-access-8fx8l\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:44:13.542943 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.542806 2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28982754-c105-4e29-972f-331557d904e4-istiod-ca-cert\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:44:13.542943 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.542818 2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28982754-c105-4e29-972f-331557d904e4-istio-token\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:44:13.542943 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.542828 2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28982754-c105-4e29-972f-331557d904e4-istio-envoy\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:44:13.542943 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.542839 2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28982754-c105-4e29-972f-331557d904e4-istio-podinfo\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:44:13.862643 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.862612 2552 generic.go:356] "Generic (PLEG): container finished" podID="28982754-c105-4e29-972f-331557d904e4" containerID="a89aed0f7c5901eaf28161e85dcb8ea6525b9c2148177598909bc7ba791379bf" exitCode=0 Jul 13 09:44:13.862800 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.862682 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" event={"ID":"28982754-c105-4e29-972f-331557d904e4","Type":"ContainerDied","Data":"a89aed0f7c5901eaf28161e85dcb8ea6525b9c2148177598909bc7ba791379bf"} Jul 13 09:44:13.862800 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.862718 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" Jul 13 09:44:13.862800 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.862733 2552 scope.go:117] "RemoveContainer" containerID="a89aed0f7c5901eaf28161e85dcb8ea6525b9c2148177598909bc7ba791379bf" Jul 13 09:44:13.862904 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.862723 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j" event={"ID":"28982754-c105-4e29-972f-331557d904e4","Type":"ContainerDied","Data":"07ebfcc998ba9823792703720c6af0f0a3b4a92961db279984f7655d537f94ec"} Jul 13 09:44:13.873122 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.873105 2552 scope.go:117] "RemoveContainer" containerID="db95e4978ca9c859b782c04f56b0dadeb982477ea93d8f1d935b9275af750c9a" Jul 13 09:44:13.881718 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.881702 2552 scope.go:117] "RemoveContainer" containerID="a89aed0f7c5901eaf28161e85dcb8ea6525b9c2148177598909bc7ba791379bf" Jul 13 09:44:13.881976 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:44:13.881956 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a89aed0f7c5901eaf28161e85dcb8ea6525b9c2148177598909bc7ba791379bf\": container with ID starting with a89aed0f7c5901eaf28161e85dcb8ea6525b9c2148177598909bc7ba791379bf not found: ID does not exist" containerID="a89aed0f7c5901eaf28161e85dcb8ea6525b9c2148177598909bc7ba791379bf" Jul 13 09:44:13.882040 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.881984 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a89aed0f7c5901eaf28161e85dcb8ea6525b9c2148177598909bc7ba791379bf"} err="failed to get container status \"a89aed0f7c5901eaf28161e85dcb8ea6525b9c2148177598909bc7ba791379bf\": rpc error: code = NotFound desc = could not find container \"a89aed0f7c5901eaf28161e85dcb8ea6525b9c2148177598909bc7ba791379bf\": container with ID starting with a89aed0f7c5901eaf28161e85dcb8ea6525b9c2148177598909bc7ba791379bf not found: ID does not exist" Jul 13 09:44:13.882040 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.882002 2552 scope.go:117] "RemoveContainer" containerID="db95e4978ca9c859b782c04f56b0dadeb982477ea93d8f1d935b9275af750c9a" Jul 13 09:44:13.882419 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:44:13.882252 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db95e4978ca9c859b782c04f56b0dadeb982477ea93d8f1d935b9275af750c9a\": container with ID starting with db95e4978ca9c859b782c04f56b0dadeb982477ea93d8f1d935b9275af750c9a not found: ID does not exist" containerID="db95e4978ca9c859b782c04f56b0dadeb982477ea93d8f1d935b9275af750c9a" Jul 13 09:44:13.882419 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.882269 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db95e4978ca9c859b782c04f56b0dadeb982477ea93d8f1d935b9275af750c9a"} err="failed to get container status \"db95e4978ca9c859b782c04f56b0dadeb982477ea93d8f1d935b9275af750c9a\": rpc error: code = NotFound desc = could not find container \"db95e4978ca9c859b782c04f56b0dadeb982477ea93d8f1d935b9275af750c9a\": container with ID starting with db95e4978ca9c859b782c04f56b0dadeb982477ea93d8f1d935b9275af750c9a not found: ID does not exist" Jul 13 09:44:13.886637 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.886617 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j"] Jul 13 09:44:13.890580 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:13.890554 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-6f7934-openshift-default-6fddbcf9b9-5jm2j"] Jul 13 09:44:14.670098 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:14.670058 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28982754-c105-4e29-972f-331557d904e4" path="/var/lib/kubelet/pods/28982754-c105-4e29-972f-331557d904e4/volumes" Jul 13 09:44:16.244162 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:16.244119 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:16.685428 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:16.685395 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:17.122713 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.122683 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx"] Jul 13 09:44:17.123102 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.123089 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28982754-c105-4e29-972f-331557d904e4" containerName="istio-proxy" Jul 13 09:44:17.123102 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.123103 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="28982754-c105-4e29-972f-331557d904e4" containerName="istio-proxy" Jul 13 09:44:17.123193 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.123187 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="28982754-c105-4e29-972f-331557d904e4" containerName="istio-proxy" Jul 13 09:44:17.127960 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.127939 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:17.135474 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.135454 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-b0f0c0-openshift-default-dockercfg-gj8kk\"" Jul 13 09:44:17.160126 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.160088 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx"] Jul 13 09:44:17.220673 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.220642 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:17.276715 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.276682 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa610d95-d7d5-4caa-92e4-acee90bd6bab-workload-socket\") pod \"e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx\" (UID: \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\") " pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:17.277072 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.276726 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa610d95-d7d5-4caa-92e4-acee90bd6bab-istio-envoy\") pod \"e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx\" (UID: \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\") " pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:17.277072 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.276759 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa610d95-d7d5-4caa-92e4-acee90bd6bab-istiod-ca-cert\") pod \"e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx\" (UID: \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\") " pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:17.277072 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.276790 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa610d95-d7d5-4caa-92e4-acee90bd6bab-credential-socket\") pod \"e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx\" (UID: \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\") " pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:17.277072 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.276806 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa610d95-d7d5-4caa-92e4-acee90bd6bab-istio-data\") pod \"e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx\" (UID: \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\") " pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:17.277072 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.276854 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwmwz\" (UniqueName: \"kubernetes.io/projected/aa610d95-d7d5-4caa-92e4-acee90bd6bab-kube-api-access-cwmwz\") pod \"e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx\" (UID: \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\") " pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:17.277072 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.276886 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa610d95-d7d5-4caa-92e4-acee90bd6bab-istio-token\") pod \"e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx\" (UID: \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\") " pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:17.277072 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.276984 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa610d95-d7d5-4caa-92e4-acee90bd6bab-workload-certs\") pod \"e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx\" (UID: \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\") " pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:17.277072 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.277033 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa610d95-d7d5-4caa-92e4-acee90bd6bab-istio-podinfo\") pod \"e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx\" (UID: \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\") " pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:17.342614 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.342574 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:17.378020 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.377937 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwmwz\" (UniqueName: \"kubernetes.io/projected/aa610d95-d7d5-4caa-92e4-acee90bd6bab-kube-api-access-cwmwz\") pod \"e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx\" (UID: \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\") " pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:17.378020 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.377981 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa610d95-d7d5-4caa-92e4-acee90bd6bab-istio-token\") pod \"e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx\" (UID: \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\") " pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:17.378020 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.378018 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa610d95-d7d5-4caa-92e4-acee90bd6bab-workload-certs\") pod \"e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx\" (UID: \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\") " pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:17.378292 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.378048 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa610d95-d7d5-4caa-92e4-acee90bd6bab-istio-podinfo\") pod \"e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx\" (UID: \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\") " pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:17.378292 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.378116 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa610d95-d7d5-4caa-92e4-acee90bd6bab-workload-socket\") pod \"e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx\" (UID: \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\") " pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:17.378292 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.378158 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa610d95-d7d5-4caa-92e4-acee90bd6bab-istio-envoy\") pod \"e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx\" (UID: \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\") " pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:17.378450 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.378305 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa610d95-d7d5-4caa-92e4-acee90bd6bab-istiod-ca-cert\") pod \"e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx\" (UID: \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\") " pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:17.378450 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.378375 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa610d95-d7d5-4caa-92e4-acee90bd6bab-credential-socket\") pod \"e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx\" (UID: \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\") " pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:17.378450 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.378403 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa610d95-d7d5-4caa-92e4-acee90bd6bab-istio-data\") pod \"e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx\" (UID: \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\") " pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:17.378826 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.378771 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa610d95-d7d5-4caa-92e4-acee90bd6bab-istio-data\") pod \"e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx\" (UID: \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\") " pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:17.378826 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.378802 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa610d95-d7d5-4caa-92e4-acee90bd6bab-credential-socket\") pod \"e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx\" (UID: \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\") " pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:17.378993 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.378848 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa610d95-d7d5-4caa-92e4-acee90bd6bab-workload-certs\") pod \"e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx\" (UID: \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\") " pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:17.379074 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.378991 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa610d95-d7d5-4caa-92e4-acee90bd6bab-workload-socket\") pod \"e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx\" (UID: \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\") " pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:17.379074 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.379057 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa610d95-d7d5-4caa-92e4-acee90bd6bab-istiod-ca-cert\") pod \"e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx\" (UID: \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\") " pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:17.380629 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.380609 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa610d95-d7d5-4caa-92e4-acee90bd6bab-istio-envoy\") pod \"e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx\" (UID: \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\") " pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:17.381055 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.381035 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa610d95-d7d5-4caa-92e4-acee90bd6bab-istio-podinfo\") pod \"e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx\" (UID: \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\") " pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:17.387656 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.387625 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa610d95-d7d5-4caa-92e4-acee90bd6bab-istio-token\") pod \"e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx\" (UID: \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\") " pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:17.388202 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.388176 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwmwz\" (UniqueName: \"kubernetes.io/projected/aa610d95-d7d5-4caa-92e4-acee90bd6bab-kube-api-access-cwmwz\") pod \"e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx\" (UID: \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\") " pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:17.438416 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.438394 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:17.581608 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.581579 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx"] Jul 13 09:44:17.583442 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:44:17.583406 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa610d95_d7d5_4caa_92e4_acee90bd6bab.slice/crio-5d2d717d614aee982de36a610ec25894da98d8f03a3c9ec3f92bd6b0b77e9bdb WatchSource:0}: Error finding container 5d2d717d614aee982de36a610ec25894da98d8f03a3c9ec3f92bd6b0b77e9bdb: Status 404 returned error can't find the container with id 5d2d717d614aee982de36a610ec25894da98d8f03a3c9ec3f92bd6b0b77e9bdb Jul 13 09:44:17.586005 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.585965 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:44:17.586090 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.586051 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:44:17.586130 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.586095 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:44:17.888745 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.888647 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" event={"ID":"aa610d95-d7d5-4caa-92e4-acee90bd6bab","Type":"ContainerStarted","Data":"545b88771970d4ed5b147d05520d584c4baf446e243fd74d41ad554b9cc37a03"} Jul 13 09:44:17.888745 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.888706 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" event={"ID":"aa610d95-d7d5-4caa-92e4-acee90bd6bab","Type":"ContainerStarted","Data":"5d2d717d614aee982de36a610ec25894da98d8f03a3c9ec3f92bd6b0b77e9bdb"} Jul 13 09:44:17.912921 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:17.912867 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" podStartSLOduration=0.912850974 podStartE2EDuration="912.850974ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:44:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:44:17.910692719 +0000 UTC m=+2141.790647985" watchObservedRunningTime="2026-07-13 09:44:17.912850974 +0000 UTC m=+2141.792806232" Jul 13 09:44:18.439178 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:18.439142 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:18.444242 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:18.444200 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:18.894417 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:18.894384 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:18.895342 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:18.895320 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:19.824344 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:19.824306 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:19.915808 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:19.915771 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:20.016348 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:20.016308 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:20.560705 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:20.560669 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:20.660180 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:20.660146 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:24.276682 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:24.276644 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:26.885388 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:26.885350 2552 patch_prober.go:28] interesting pod/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 13 09:44:26.885741 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:26.885428 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" podUID="ad53ae97-49e4-4a27-bd1f-72c50f355ff7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 13 09:44:28.958246 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:28.958139 2552 generic.go:356] "Generic (PLEG): container finished" podID="ad53ae97-49e4-4a27-bd1f-72c50f355ff7" containerID="80e31ce37f2dc1bcaa4c7150943c5dbd0cbce0632905211dcef283e6a3eeaea4" exitCode=0 Jul 13 09:44:28.958753 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:28.958229 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" event={"ID":"ad53ae97-49e4-4a27-bd1f-72c50f355ff7","Type":"ContainerDied","Data":"80e31ce37f2dc1bcaa4c7150943c5dbd0cbce0632905211dcef283e6a3eeaea4"} Jul 13 09:44:28.970237 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:28.958823 2552 scope.go:117] "RemoveContainer" containerID="80e31ce37f2dc1bcaa4c7150943c5dbd0cbce0632905211dcef283e6a3eeaea4" Jul 13 09:44:28.970237 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:28.962672 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:44:28.970237 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:28.962767 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:44:28.970237 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:28.962816 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:44:29.263566 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:29.263423 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:29.336779 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:29.336738 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:29.402068 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:29.402030 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:29.970252 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:29.970187 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" event={"ID":"ad53ae97-49e4-4a27-bd1f-72c50f355ff7","Type":"ContainerStarted","Data":"6e79425c77bad3365fef9c571a90a738efebc06f40c9e2c2565990b862ac2e23"} Jul 13 09:44:30.290458 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:30.290346 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:30.290458 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:30.290388 2552 patch_prober.go:28] interesting pod/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Jul 13 09:44:30.290458 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:30.290440 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" podUID="ad53ae97-49e4-4a27-bd1f-72c50f355ff7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Jul 13 09:44:31.289514 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:31.289479 2552 patch_prober.go:28] interesting pod/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Jul 13 09:44:31.290170 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:31.290114 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" podUID="ad53ae97-49e4-4a27-bd1f-72c50f355ff7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Jul 13 09:44:32.291280 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:32.289653 2552 patch_prober.go:28] interesting pod/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Jul 13 09:44:32.291280 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:32.289719 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" podUID="ad53ae97-49e4-4a27-bd1f-72c50f355ff7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Jul 13 09:44:32.305263 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:32.305104 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:32.720257 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:32.720174 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:32.907957 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:32.907919 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:33.008640 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:33.008563 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:33.182236 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:33.179041 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:33.290267 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:33.290151 2552 patch_prober.go:28] interesting pod/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Jul 13 09:44:33.290267 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:33.290254 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" podUID="ad53ae97-49e4-4a27-bd1f-72c50f355ff7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Jul 13 09:44:34.294196 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:34.294161 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:34.294636 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:34.294558 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:34.295349 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:34.295330 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:34.936431 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:34.936387 2552 patch_prober.go:28] interesting pod/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 13 09:44:34.936630 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:34.936474 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" podUID="aa610d95-d7d5-4caa-92e4-acee90bd6bab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 13 09:44:37.023855 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:37.023821 2552 generic.go:356] "Generic (PLEG): container finished" podID="aa610d95-d7d5-4caa-92e4-acee90bd6bab" containerID="545b88771970d4ed5b147d05520d584c4baf446e243fd74d41ad554b9cc37a03" exitCode=0 Jul 13 09:44:37.024336 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:37.023893 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" event={"ID":"aa610d95-d7d5-4caa-92e4-acee90bd6bab","Type":"ContainerDied","Data":"545b88771970d4ed5b147d05520d584c4baf446e243fd74d41ad554b9cc37a03"} Jul 13 09:44:37.024475 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:37.024375 2552 scope.go:117] "RemoveContainer" containerID="545b88771970d4ed5b147d05520d584c4baf446e243fd74d41ad554b9cc37a03" Jul 13 09:44:37.025519 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:37.025487 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:44:37.025630 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:37.025556 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:44:37.025630 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:37.025586 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:44:38.032174 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:38.032139 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" event={"ID":"aa610d95-d7d5-4caa-92e4-acee90bd6bab","Type":"ContainerStarted","Data":"5004394e67f19b1c10cd7880095d3755df9c6efa7c44e434e2f41d08d7a91884"} Jul 13 09:44:38.439336 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:38.439294 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:38.439551 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:38.439527 2552 patch_prober.go:28] interesting pod/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 13 09:44:38.439604 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:38.439577 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" podUID="aa610d95-d7d5-4caa-92e4-acee90bd6bab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 13 09:44:38.574585 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:38.574542 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:38.674385 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:38.674351 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:38.806744 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:38.806650 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2"] Jul 13 09:44:38.806915 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:38.806879 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" podUID="ad53ae97-49e4-4a27-bd1f-72c50f355ff7" containerName="istio-proxy" containerID="cri-o://6e79425c77bad3365fef9c571a90a738efebc06f40c9e2c2565990b862ac2e23" gracePeriod=30 Jul 13 09:44:38.831893 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:38.831860 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:39.201853 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:39.201823 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:39.439281 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:39.439243 2552 patch_prober.go:28] interesting pod/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 13 09:44:39.439469 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:39.439301 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" podUID="aa610d95-d7d5-4caa-92e4-acee90bd6bab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 13 09:44:39.586117 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:39.586083 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:39.690758 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:39.690718 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:39.793411 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:39.793379 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:39.934344 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:39.934263 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:40.439744 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:40.439701 2552 patch_prober.go:28] interesting pod/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 13 09:44:40.440157 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:40.439766 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" podUID="aa610d95-d7d5-4caa-92e4-acee90bd6bab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 13 09:44:41.439707 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:41.439665 2552 patch_prober.go:28] interesting pod/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 13 09:44:41.439899 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:41.439742 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" podUID="aa610d95-d7d5-4caa-92e4-acee90bd6bab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 13 09:44:42.443106 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:42.443073 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:42.443523 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:42.443441 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:42.444118 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:42.444102 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:44.075076 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:44.074986 2552 generic.go:356] "Generic (PLEG): container finished" podID="ad53ae97-49e4-4a27-bd1f-72c50f355ff7" containerID="6e79425c77bad3365fef9c571a90a738efebc06f40c9e2c2565990b862ac2e23" exitCode=0 Jul 13 09:44:44.075076 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:44.075058 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" event={"ID":"ad53ae97-49e4-4a27-bd1f-72c50f355ff7","Type":"ContainerDied","Data":"6e79425c77bad3365fef9c571a90a738efebc06f40c9e2c2565990b862ac2e23"} Jul 13 09:44:44.075546 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:44.075121 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" event={"ID":"ad53ae97-49e4-4a27-bd1f-72c50f355ff7","Type":"ContainerDied","Data":"f4dad6ebff06254f96ba14d0bfce5e8f22ca29a9ff8827c396addcf68c73b91d"} Jul 13 09:44:44.075546 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:44.075146 2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f4dad6ebff06254f96ba14d0bfce5e8f22ca29a9ff8827c396addcf68c73b91d" Jul 13 09:44:44.075546 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:44.075161 2552 scope.go:117] "RemoveContainer" containerID="80e31ce37f2dc1bcaa4c7150943c5dbd0cbce0632905211dcef283e6a3eeaea4" Jul 13 09:44:44.078614 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:44.078592 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:44.145404 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:44.145363 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zwd7w\" (UniqueName: \"kubernetes.io/projected/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-kube-api-access-zwd7w\") pod \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\" (UID: \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\") " Jul 13 09:44:44.145596 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:44.145418 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-istio-data\") pod \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\" (UID: \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\") " Jul 13 09:44:44.145596 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:44.145439 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-istio-token\") pod \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\" (UID: \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\") " Jul 13 09:44:44.145596 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:44.145464 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-workload-socket\") pod \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\" (UID: \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\") " Jul 13 09:44:44.145596 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:44.145512 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-workload-certs\") pod \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\" (UID: \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\") " Jul 13 09:44:44.145596 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:44.145590 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-istio-podinfo\") pod \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\" (UID: \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\") " Jul 13 09:44:44.145875 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:44.145614 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-istio-envoy\") pod \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\" (UID: \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\") " Jul 13 09:44:44.145875 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:44.145682 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-istiod-ca-cert\") pod \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\" (UID: \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\") " Jul 13 09:44:44.145875 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:44.145710 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-credential-socket\") pod \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\" (UID: \"ad53ae97-49e4-4a27-bd1f-72c50f355ff7\") " Jul 13 09:44:44.145875 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:44.145780 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ad53ae97-49e4-4a27-bd1f-72c50f355ff7" (UID: "ad53ae97-49e4-4a27-bd1f-72c50f355ff7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:44:44.145875 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:44.145855 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ad53ae97-49e4-4a27-bd1f-72c50f355ff7" (UID: "ad53ae97-49e4-4a27-bd1f-72c50f355ff7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:44:44.146125 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:44.146039 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ad53ae97-49e4-4a27-bd1f-72c50f355ff7" (UID: "ad53ae97-49e4-4a27-bd1f-72c50f355ff7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:44:44.146382 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:44.146289 2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-workload-certs\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:44:44.146382 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:44.146314 2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-credential-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:44:44.146382 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:44.146356 2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-workload-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:44:44.146631 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:44.146405 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-istio-data" (OuterVolumeSpecName: "istio-data") pod "ad53ae97-49e4-4a27-bd1f-72c50f355ff7" (UID: "ad53ae97-49e4-4a27-bd1f-72c50f355ff7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:44:44.146631 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:44.146480 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ad53ae97-49e4-4a27-bd1f-72c50f355ff7" (UID: "ad53ae97-49e4-4a27-bd1f-72c50f355ff7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 09:44:44.147871 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:44.147844 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-istio-token" (OuterVolumeSpecName: "istio-token") pod "ad53ae97-49e4-4a27-bd1f-72c50f355ff7" (UID: "ad53ae97-49e4-4a27-bd1f-72c50f355ff7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:44:44.148194 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:44.148179 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ad53ae97-49e4-4a27-bd1f-72c50f355ff7" (UID: "ad53ae97-49e4-4a27-bd1f-72c50f355ff7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 09:44:44.148325 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:44.148311 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ad53ae97-49e4-4a27-bd1f-72c50f355ff7" (UID: "ad53ae97-49e4-4a27-bd1f-72c50f355ff7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:44:44.148380 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:44.148326 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-kube-api-access-zwd7w" (OuterVolumeSpecName: "kube-api-access-zwd7w") pod "ad53ae97-49e4-4a27-bd1f-72c50f355ff7" (UID: "ad53ae97-49e4-4a27-bd1f-72c50f355ff7"). InnerVolumeSpecName "kube-api-access-zwd7w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:44:44.247409 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:44.247362 2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-istio-podinfo\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:44:44.247409 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:44.247398 2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-istio-envoy\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:44:44.247409 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:44.247410 2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-istiod-ca-cert\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:44:44.247665 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:44.247421 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwd7w\" (UniqueName: \"kubernetes.io/projected/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-kube-api-access-zwd7w\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:44:44.247665 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:44.247434 2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-istio-data\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:44:44.247665 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:44.247446 2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad53ae97-49e4-4a27-bd1f-72c50f355ff7-istio-token\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:44:45.088639 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:45.088605 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2" Jul 13 09:44:45.127763 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:45.127727 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2"] Jul 13 09:44:45.132577 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:45.132547 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-c45f95-openshift-default-6684bf87ff-lhqz2"] Jul 13 09:44:45.519429 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:45.519326 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:45.590726 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:45.590692 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:45.758407 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:45.758367 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx"] Jul 13 09:44:45.758650 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:45.758619 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" podUID="aa610d95-d7d5-4caa-92e4-acee90bd6bab" containerName="istio-proxy" containerID="cri-o://5004394e67f19b1c10cd7880095d3755df9c6efa7c44e434e2f41d08d7a91884" gracePeriod=30 Jul 13 09:44:45.772605 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:45.772462 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:46.298412 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.298359 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:46.556814 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.556719 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb"] Jul 13 09:44:46.557406 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.557386 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad53ae97-49e4-4a27-bd1f-72c50f355ff7" containerName="istio-proxy" Jul 13 09:44:46.557457 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.557410 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad53ae97-49e4-4a27-bd1f-72c50f355ff7" containerName="istio-proxy" Jul 13 09:44:46.557457 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.557424 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28982754-c105-4e29-972f-331557d904e4" containerName="istio-proxy" Jul 13 09:44:46.557457 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.557432 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="28982754-c105-4e29-972f-331557d904e4" containerName="istio-proxy" Jul 13 09:44:46.557457 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.557443 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad53ae97-49e4-4a27-bd1f-72c50f355ff7" containerName="istio-proxy" Jul 13 09:44:46.557457 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.557451 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad53ae97-49e4-4a27-bd1f-72c50f355ff7" containerName="istio-proxy" Jul 13 09:44:46.557633 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.557550 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad53ae97-49e4-4a27-bd1f-72c50f355ff7" containerName="istio-proxy" Jul 13 09:44:46.557633 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.557562 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad53ae97-49e4-4a27-bd1f-72c50f355ff7" containerName="istio-proxy" Jul 13 09:44:46.557633 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.557571 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="28982754-c105-4e29-972f-331557d904e4" containerName="istio-proxy" Jul 13 09:44:46.562430 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.562404 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:44:46.565451 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.565422 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-f77299-openshift-default-dockercfg-pzv89\"" Jul 13 09:44:46.573740 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.573701 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb"] Jul 13 09:44:46.587827 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.587774 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:46.670489 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.670454 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad53ae97-49e4-4a27-bd1f-72c50f355ff7" path="/var/lib/kubelet/pods/ad53ae97-49e4-4a27-bd1f-72c50f355ff7/volumes" Jul 13 09:44:46.671638 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.671609 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2daaeacd-bce0-4663-885c-51bc852f3f12-istio-data\") pod \"e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb\" (UID: \"2daaeacd-bce0-4663-885c-51bc852f3f12\") " pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:44:46.671788 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.671658 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgtg7\" (UniqueName: \"kubernetes.io/projected/2daaeacd-bce0-4663-885c-51bc852f3f12-kube-api-access-wgtg7\") pod \"e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb\" (UID: \"2daaeacd-bce0-4663-885c-51bc852f3f12\") " pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:44:46.671788 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.671694 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2daaeacd-bce0-4663-885c-51bc852f3f12-istio-podinfo\") pod \"e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb\" (UID: \"2daaeacd-bce0-4663-885c-51bc852f3f12\") " pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:44:46.671788 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.671730 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2daaeacd-bce0-4663-885c-51bc852f3f12-workload-certs\") pod \"e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb\" (UID: \"2daaeacd-bce0-4663-885c-51bc852f3f12\") " pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:44:46.671949 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.671797 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2daaeacd-bce0-4663-885c-51bc852f3f12-credential-socket\") pod \"e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb\" (UID: \"2daaeacd-bce0-4663-885c-51bc852f3f12\") " pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:44:46.671949 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.671825 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2daaeacd-bce0-4663-885c-51bc852f3f12-workload-socket\") pod \"e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb\" (UID: \"2daaeacd-bce0-4663-885c-51bc852f3f12\") " pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:44:46.671949 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.671857 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2daaeacd-bce0-4663-885c-51bc852f3f12-istio-token\") pod \"e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb\" (UID: \"2daaeacd-bce0-4663-885c-51bc852f3f12\") " pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:44:46.671949 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.671917 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2daaeacd-bce0-4663-885c-51bc852f3f12-istio-envoy\") pod \"e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb\" (UID: \"2daaeacd-bce0-4663-885c-51bc852f3f12\") " pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:44:46.672146 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.671945 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2daaeacd-bce0-4663-885c-51bc852f3f12-istiod-ca-cert\") pod \"e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb\" (UID: \"2daaeacd-bce0-4663-885c-51bc852f3f12\") " pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:44:46.772507 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.772472 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2daaeacd-bce0-4663-885c-51bc852f3f12-workload-certs\") pod \"e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb\" (UID: \"2daaeacd-bce0-4663-885c-51bc852f3f12\") " pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:44:46.772716 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.772537 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2daaeacd-bce0-4663-885c-51bc852f3f12-credential-socket\") pod \"e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb\" (UID: \"2daaeacd-bce0-4663-885c-51bc852f3f12\") " pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:44:46.772716 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.772559 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2daaeacd-bce0-4663-885c-51bc852f3f12-workload-socket\") pod \"e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb\" (UID: \"2daaeacd-bce0-4663-885c-51bc852f3f12\") " pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:44:46.772716 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.772582 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2daaeacd-bce0-4663-885c-51bc852f3f12-istio-token\") pod \"e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb\" (UID: \"2daaeacd-bce0-4663-885c-51bc852f3f12\") " pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:44:46.772716 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.772620 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2daaeacd-bce0-4663-885c-51bc852f3f12-istio-envoy\") pod \"e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb\" (UID: \"2daaeacd-bce0-4663-885c-51bc852f3f12\") " pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:44:46.772943 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.772710 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2daaeacd-bce0-4663-885c-51bc852f3f12-istiod-ca-cert\") pod \"e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb\" (UID: \"2daaeacd-bce0-4663-885c-51bc852f3f12\") " pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:44:46.772943 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.772812 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2daaeacd-bce0-4663-885c-51bc852f3f12-istio-data\") pod \"e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb\" (UID: \"2daaeacd-bce0-4663-885c-51bc852f3f12\") " pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:44:46.772943 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.772861 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgtg7\" (UniqueName: \"kubernetes.io/projected/2daaeacd-bce0-4663-885c-51bc852f3f12-kube-api-access-wgtg7\") pod \"e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb\" (UID: \"2daaeacd-bce0-4663-885c-51bc852f3f12\") " pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:44:46.772943 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.772893 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2daaeacd-bce0-4663-885c-51bc852f3f12-istio-podinfo\") pod \"e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb\" (UID: \"2daaeacd-bce0-4663-885c-51bc852f3f12\") " pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:44:46.772943 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.772906 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2daaeacd-bce0-4663-885c-51bc852f3f12-workload-certs\") pod \"e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb\" (UID: \"2daaeacd-bce0-4663-885c-51bc852f3f12\") " pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:44:46.773169 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.772982 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2daaeacd-bce0-4663-885c-51bc852f3f12-credential-socket\") pod \"e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb\" (UID: \"2daaeacd-bce0-4663-885c-51bc852f3f12\") " pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:44:46.773515 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.773487 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2daaeacd-bce0-4663-885c-51bc852f3f12-workload-socket\") pod \"e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb\" (UID: \"2daaeacd-bce0-4663-885c-51bc852f3f12\") " pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:44:46.773620 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.773511 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2daaeacd-bce0-4663-885c-51bc852f3f12-istiod-ca-cert\") pod \"e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb\" (UID: \"2daaeacd-bce0-4663-885c-51bc852f3f12\") " pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:44:46.773747 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.773723 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2daaeacd-bce0-4663-885c-51bc852f3f12-istio-data\") pod \"e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb\" (UID: \"2daaeacd-bce0-4663-885c-51bc852f3f12\") " pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:44:46.775583 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.775556 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2daaeacd-bce0-4663-885c-51bc852f3f12-istio-envoy\") pod \"e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb\" (UID: \"2daaeacd-bce0-4663-885c-51bc852f3f12\") " pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:44:46.775809 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.775790 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2daaeacd-bce0-4663-885c-51bc852f3f12-istio-podinfo\") pod \"e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb\" (UID: \"2daaeacd-bce0-4663-885c-51bc852f3f12\") " pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:44:46.781717 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.781690 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2daaeacd-bce0-4663-885c-51bc852f3f12-istio-token\") pod \"e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb\" (UID: \"2daaeacd-bce0-4663-885c-51bc852f3f12\") " pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:44:46.781810 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.781778 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgtg7\" (UniqueName: \"kubernetes.io/projected/2daaeacd-bce0-4663-885c-51bc852f3f12-kube-api-access-wgtg7\") pod \"e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb\" (UID: \"2daaeacd-bce0-4663-885c-51bc852f3f12\") " pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:44:46.868840 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.868791 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:46.874915 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:46.874883 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:44:47.032494 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:47.032469 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb"] Jul 13 09:44:47.034991 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:44:47.034965 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2daaeacd_bce0_4663_885c_51bc852f3f12.slice/crio-82020d5ef17482f8f2629426207599bc5133527d9f6c2c613e0a357b40da15f6 WatchSource:0}: Error finding container 82020d5ef17482f8f2629426207599bc5133527d9f6c2c613e0a357b40da15f6: Status 404 returned error can't find the container with id 82020d5ef17482f8f2629426207599bc5133527d9f6c2c613e0a357b40da15f6 Jul 13 09:44:47.037573 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:47.037541 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:44:47.037649 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:47.037612 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:44:47.037649 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:47.037639 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:44:47.101968 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:47.101937 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" event={"ID":"2daaeacd-bce0-4663-885c-51bc852f3f12","Type":"ContainerStarted","Data":"82020d5ef17482f8f2629426207599bc5133527d9f6c2c613e0a357b40da15f6"} Jul 13 09:44:48.109380 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:48.109338 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" event={"ID":"2daaeacd-bce0-4663-885c-51bc852f3f12","Type":"ContainerStarted","Data":"9cf733cfd3401b85443ec54bafa3cdb85382152964a716e8f46d96de32f23a37"} Jul 13 09:44:48.133564 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:48.133508 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" podStartSLOduration=2.133491144 podStartE2EDuration="2.133491144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:44:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:44:48.131468351 +0000 UTC m=+2172.011423618" watchObservedRunningTime="2026-07-13 09:44:48.133491144 +0000 UTC m=+2172.013446411" Jul 13 09:44:48.875905 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:48.875868 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:44:48.880894 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:48.880864 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:44:49.115032 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:49.115000 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:44:49.115974 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:49.115955 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:44:50.015112 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:50.015066 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:50.101886 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:50.101850 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:50.171778 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:50.171734 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:50.318107 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:50.318015 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:50.814856 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:50.814823 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:50.917113 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:50.917083 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:51.031921 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.031852 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:51.128571 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.128536 2552 generic.go:356] "Generic (PLEG): container finished" podID="aa610d95-d7d5-4caa-92e4-acee90bd6bab" containerID="5004394e67f19b1c10cd7880095d3755df9c6efa7c44e434e2f41d08d7a91884" exitCode=0 Jul 13 09:44:51.128788 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.128595 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" Jul 13 09:44:51.128788 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.128633 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" event={"ID":"aa610d95-d7d5-4caa-92e4-acee90bd6bab","Type":"ContainerDied","Data":"5004394e67f19b1c10cd7880095d3755df9c6efa7c44e434e2f41d08d7a91884"} Jul 13 09:44:51.128788 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.128690 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx" event={"ID":"aa610d95-d7d5-4caa-92e4-acee90bd6bab","Type":"ContainerDied","Data":"5d2d717d614aee982de36a610ec25894da98d8f03a3c9ec3f92bd6b0b77e9bdb"} Jul 13 09:44:51.128788 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.128714 2552 scope.go:117] "RemoveContainer" containerID="5004394e67f19b1c10cd7880095d3755df9c6efa7c44e434e2f41d08d7a91884" Jul 13 09:44:51.138353 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.138325 2552 scope.go:117] "RemoveContainer" containerID="545b88771970d4ed5b147d05520d584c4baf446e243fd74d41ad554b9cc37a03" Jul 13 09:44:51.147414 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.147393 2552 scope.go:117] "RemoveContainer" containerID="5004394e67f19b1c10cd7880095d3755df9c6efa7c44e434e2f41d08d7a91884" Jul 13 09:44:51.147693 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:44:51.147674 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5004394e67f19b1c10cd7880095d3755df9c6efa7c44e434e2f41d08d7a91884\": container with ID starting with 5004394e67f19b1c10cd7880095d3755df9c6efa7c44e434e2f41d08d7a91884 not found: ID does not exist" containerID="5004394e67f19b1c10cd7880095d3755df9c6efa7c44e434e2f41d08d7a91884" Jul 13 09:44:51.147778 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.147708 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5004394e67f19b1c10cd7880095d3755df9c6efa7c44e434e2f41d08d7a91884"} err="failed to get container status \"5004394e67f19b1c10cd7880095d3755df9c6efa7c44e434e2f41d08d7a91884\": rpc error: code = NotFound desc = could not find container \"5004394e67f19b1c10cd7880095d3755df9c6efa7c44e434e2f41d08d7a91884\": container with ID starting with 5004394e67f19b1c10cd7880095d3755df9c6efa7c44e434e2f41d08d7a91884 not found: ID does not exist" Jul 13 09:44:51.147778 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.147735 2552 scope.go:117] "RemoveContainer" containerID="545b88771970d4ed5b147d05520d584c4baf446e243fd74d41ad554b9cc37a03" Jul 13 09:44:51.148087 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:44:51.148059 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"545b88771970d4ed5b147d05520d584c4baf446e243fd74d41ad554b9cc37a03\": container with ID starting with 545b88771970d4ed5b147d05520d584c4baf446e243fd74d41ad554b9cc37a03 not found: ID does not exist" containerID="545b88771970d4ed5b147d05520d584c4baf446e243fd74d41ad554b9cc37a03" Jul 13 09:44:51.148249 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.148093 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"545b88771970d4ed5b147d05520d584c4baf446e243fd74d41ad554b9cc37a03"} err="failed to get container status \"545b88771970d4ed5b147d05520d584c4baf446e243fd74d41ad554b9cc37a03\": rpc error: code = NotFound desc = could not find container \"545b88771970d4ed5b147d05520d584c4baf446e243fd74d41ad554b9cc37a03\": container with ID starting with 545b88771970d4ed5b147d05520d584c4baf446e243fd74d41ad554b9cc37a03 not found: ID does not exist" Jul 13 09:44:51.218777 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.218734 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa610d95-d7d5-4caa-92e4-acee90bd6bab-istio-envoy\") pod \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\" (UID: \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\") " Jul 13 09:44:51.218777 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.218779 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cwmwz\" (UniqueName: \"kubernetes.io/projected/aa610d95-d7d5-4caa-92e4-acee90bd6bab-kube-api-access-cwmwz\") pod \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\" (UID: \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\") " Jul 13 09:44:51.219391 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.218817 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa610d95-d7d5-4caa-92e4-acee90bd6bab-workload-socket\") pod \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\" (UID: \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\") " Jul 13 09:44:51.219391 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.218837 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa610d95-d7d5-4caa-92e4-acee90bd6bab-workload-certs\") pod \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\" (UID: \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\") " Jul 13 09:44:51.219391 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.218865 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa610d95-d7d5-4caa-92e4-acee90bd6bab-istio-podinfo\") pod \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\" (UID: \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\") " Jul 13 09:44:51.219391 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.218891 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa610d95-d7d5-4caa-92e4-acee90bd6bab-credential-socket\") pod \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\" (UID: \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\") " Jul 13 09:44:51.219391 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.218973 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa610d95-d7d5-4caa-92e4-acee90bd6bab-istio-token\") pod \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\" (UID: \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\") " Jul 13 09:44:51.219391 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.219012 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa610d95-d7d5-4caa-92e4-acee90bd6bab-istio-data\") pod \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\" (UID: \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\") " Jul 13 09:44:51.219391 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.219097 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa610d95-d7d5-4caa-92e4-acee90bd6bab-istiod-ca-cert\") pod \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\" (UID: \"aa610d95-d7d5-4caa-92e4-acee90bd6bab\") " Jul 13 09:44:51.219391 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.219183 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa610d95-d7d5-4caa-92e4-acee90bd6bab-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "aa610d95-d7d5-4caa-92e4-acee90bd6bab" (UID: "aa610d95-d7d5-4caa-92e4-acee90bd6bab"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:44:51.219836 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.219395 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa610d95-d7d5-4caa-92e4-acee90bd6bab-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "aa610d95-d7d5-4caa-92e4-acee90bd6bab" (UID: "aa610d95-d7d5-4caa-92e4-acee90bd6bab"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:44:51.219836 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.219526 2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa610d95-d7d5-4caa-92e4-acee90bd6bab-workload-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:44:51.219836 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.219548 2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa610d95-d7d5-4caa-92e4-acee90bd6bab-credential-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:44:51.219836 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.219696 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa610d95-d7d5-4caa-92e4-acee90bd6bab-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "aa610d95-d7d5-4caa-92e4-acee90bd6bab" (UID: "aa610d95-d7d5-4caa-92e4-acee90bd6bab"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:44:51.219836 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.219742 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa610d95-d7d5-4caa-92e4-acee90bd6bab-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "aa610d95-d7d5-4caa-92e4-acee90bd6bab" (UID: "aa610d95-d7d5-4caa-92e4-acee90bd6bab"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 09:44:51.220106 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.220041 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa610d95-d7d5-4caa-92e4-acee90bd6bab-istio-data" (OuterVolumeSpecName: "istio-data") pod "aa610d95-d7d5-4caa-92e4-acee90bd6bab" (UID: "aa610d95-d7d5-4caa-92e4-acee90bd6bab"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:44:51.221231 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.221190 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa610d95-d7d5-4caa-92e4-acee90bd6bab-kube-api-access-cwmwz" (OuterVolumeSpecName: "kube-api-access-cwmwz") pod "aa610d95-d7d5-4caa-92e4-acee90bd6bab" (UID: "aa610d95-d7d5-4caa-92e4-acee90bd6bab"). InnerVolumeSpecName "kube-api-access-cwmwz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:44:51.221351 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.221303 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa610d95-d7d5-4caa-92e4-acee90bd6bab-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "aa610d95-d7d5-4caa-92e4-acee90bd6bab" (UID: "aa610d95-d7d5-4caa-92e4-acee90bd6bab"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:44:51.221351 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.221311 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa610d95-d7d5-4caa-92e4-acee90bd6bab-istio-token" (OuterVolumeSpecName: "istio-token") pod "aa610d95-d7d5-4caa-92e4-acee90bd6bab" (UID: "aa610d95-d7d5-4caa-92e4-acee90bd6bab"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:44:51.221432 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.221350 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aa610d95-d7d5-4caa-92e4-acee90bd6bab-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "aa610d95-d7d5-4caa-92e4-acee90bd6bab" (UID: "aa610d95-d7d5-4caa-92e4-acee90bd6bab"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 09:44:51.320606 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.320502 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwmwz\" (UniqueName: \"kubernetes.io/projected/aa610d95-d7d5-4caa-92e4-acee90bd6bab-kube-api-access-cwmwz\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:44:51.320606 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.320537 2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa610d95-d7d5-4caa-92e4-acee90bd6bab-workload-certs\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:44:51.320606 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.320547 2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa610d95-d7d5-4caa-92e4-acee90bd6bab-istio-podinfo\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:44:51.320606 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.320557 2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa610d95-d7d5-4caa-92e4-acee90bd6bab-istio-token\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:44:51.320606 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.320567 2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa610d95-d7d5-4caa-92e4-acee90bd6bab-istio-data\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:44:51.320606 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.320578 2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa610d95-d7d5-4caa-92e4-acee90bd6bab-istiod-ca-cert\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:44:51.320606 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.320590 2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa610d95-d7d5-4caa-92e4-acee90bd6bab-istio-envoy\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:44:51.453108 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.453053 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx"] Jul 13 09:44:51.456143 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:51.456110 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-b0f0c0-openshift-default-56c8fdbb79-r4ctx"] Jul 13 09:44:52.667168 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:52.667127 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa610d95-d7d5-4caa-92e4-acee90bd6bab" path="/var/lib/kubelet/pods/aa610d95-d7d5-4caa-92e4-acee90bd6bab/volumes" Jul 13 09:44:54.484066 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:54.484002 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:54.796311 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:54.796190 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:55.058126 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.058040 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c"] Jul 13 09:44:55.058718 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.058694 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa610d95-d7d5-4caa-92e4-acee90bd6bab" containerName="istio-proxy" Jul 13 09:44:55.058718 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.058716 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa610d95-d7d5-4caa-92e4-acee90bd6bab" containerName="istio-proxy" Jul 13 09:44:55.058894 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.058837 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa610d95-d7d5-4caa-92e4-acee90bd6bab" containerName="istio-proxy" Jul 13 09:44:55.058894 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.058850 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa610d95-d7d5-4caa-92e4-acee90bd6bab" containerName="istio-proxy" Jul 13 09:44:55.064369 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.064343 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:44:55.067132 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.067106 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-7bf32e-openshift-default-dockercfg-pwfqz\"" Jul 13 09:44:55.075075 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.075051 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c"] Jul 13 09:44:55.088682 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.088641 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:55.155247 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.155190 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wchrh\" (UniqueName: \"kubernetes.io/projected/33adf1db-f9de-42b5-9438-d0dfcff1eaec-kube-api-access-wchrh\") pod \"e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c\" (UID: \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\") " pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:44:55.155548 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.155522 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33adf1db-f9de-42b5-9438-d0dfcff1eaec-istio-data\") pod \"e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c\" (UID: \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\") " pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:44:55.155684 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.155668 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33adf1db-f9de-42b5-9438-d0dfcff1eaec-istio-token\") pod \"e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c\" (UID: \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\") " pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:44:55.155792 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.155778 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33adf1db-f9de-42b5-9438-d0dfcff1eaec-istio-envoy\") pod \"e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c\" (UID: \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\") " pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:44:55.155896 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.155880 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33adf1db-f9de-42b5-9438-d0dfcff1eaec-credential-socket\") pod \"e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c\" (UID: \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\") " pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:44:55.156020 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.156003 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33adf1db-f9de-42b5-9438-d0dfcff1eaec-workload-socket\") pod \"e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c\" (UID: \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\") " pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:44:55.156162 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.156135 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33adf1db-f9de-42b5-9438-d0dfcff1eaec-istiod-ca-cert\") pod \"e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c\" (UID: \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\") " pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:44:55.156162 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.156174 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33adf1db-f9de-42b5-9438-d0dfcff1eaec-workload-certs\") pod \"e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c\" (UID: \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\") " pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:44:55.156392 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.156247 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33adf1db-f9de-42b5-9438-d0dfcff1eaec-istio-podinfo\") pod \"e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c\" (UID: \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\") " pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:44:55.256940 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.256898 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wchrh\" (UniqueName: \"kubernetes.io/projected/33adf1db-f9de-42b5-9438-d0dfcff1eaec-kube-api-access-wchrh\") pod \"e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c\" (UID: \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\") " pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:44:55.256940 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.256942 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33adf1db-f9de-42b5-9438-d0dfcff1eaec-istio-data\") pod \"e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c\" (UID: \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\") " pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:44:55.257184 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.257135 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33adf1db-f9de-42b5-9438-d0dfcff1eaec-istio-token\") pod \"e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c\" (UID: \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\") " pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:44:55.257184 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.257178 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33adf1db-f9de-42b5-9438-d0dfcff1eaec-istio-envoy\") pod \"e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c\" (UID: \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\") " pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:44:55.257306 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.257237 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33adf1db-f9de-42b5-9438-d0dfcff1eaec-credential-socket\") pod \"e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c\" (UID: \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\") " pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:44:55.257306 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.257272 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33adf1db-f9de-42b5-9438-d0dfcff1eaec-workload-socket\") pod \"e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c\" (UID: \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\") " pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:44:55.257414 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.257333 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33adf1db-f9de-42b5-9438-d0dfcff1eaec-istiod-ca-cert\") pod \"e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c\" (UID: \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\") " pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:44:55.257414 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.257363 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33adf1db-f9de-42b5-9438-d0dfcff1eaec-workload-certs\") pod \"e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c\" (UID: \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\") " pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:44:55.257521 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.257425 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33adf1db-f9de-42b5-9438-d0dfcff1eaec-istio-podinfo\") pod \"e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c\" (UID: \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\") " pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:44:55.257633 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.257609 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33adf1db-f9de-42b5-9438-d0dfcff1eaec-istio-data\") pod \"e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c\" (UID: \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\") " pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:44:55.257696 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.257644 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33adf1db-f9de-42b5-9438-d0dfcff1eaec-workload-socket\") pod \"e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c\" (UID: \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\") " pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:44:55.257746 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.257718 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33adf1db-f9de-42b5-9438-d0dfcff1eaec-workload-certs\") pod \"e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c\" (UID: \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\") " pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:44:55.257842 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.257823 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33adf1db-f9de-42b5-9438-d0dfcff1eaec-credential-socket\") pod \"e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c\" (UID: \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\") " pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:44:55.258152 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.258133 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33adf1db-f9de-42b5-9438-d0dfcff1eaec-istiod-ca-cert\") pod \"e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c\" (UID: \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\") " pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:44:55.259657 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.259634 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33adf1db-f9de-42b5-9438-d0dfcff1eaec-istio-envoy\") pod \"e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c\" (UID: \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\") " pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:44:55.259900 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.259880 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33adf1db-f9de-42b5-9438-d0dfcff1eaec-istio-podinfo\") pod \"e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c\" (UID: \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\") " pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:44:55.265447 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.265420 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33adf1db-f9de-42b5-9438-d0dfcff1eaec-istio-token\") pod \"e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c\" (UID: \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\") " pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:44:55.265620 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.265599 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wchrh\" (UniqueName: \"kubernetes.io/projected/33adf1db-f9de-42b5-9438-d0dfcff1eaec-kube-api-access-wchrh\") pod \"e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c\" (UID: \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\") " pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:44:55.378001 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.377960 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:44:55.456978 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.456937 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:55.542031 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.541996 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c"] Jul 13 09:44:55.544027 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:44:55.543998 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33adf1db_f9de_42b5_9438_d0dfcff1eaec.slice/crio-053d8fca26411fcbae5579d24f199b607954dce21e139483ccc942d761e49de9 WatchSource:0}: Error finding container 053d8fca26411fcbae5579d24f199b607954dce21e139483ccc942d761e49de9: Status 404 returned error can't find the container with id 053d8fca26411fcbae5579d24f199b607954dce21e139483ccc942d761e49de9 Jul 13 09:44:55.546438 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.546405 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:44:55.546508 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.546474 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:44:55.546508 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:55.546504 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:44:56.165690 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:56.165648 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" event={"ID":"33adf1db-f9de-42b5-9438-d0dfcff1eaec","Type":"ContainerStarted","Data":"7ef95645f71a68f1dc348abddcdb1d29c3d311341ba77b19bea21c8108cfc97a"} Jul 13 09:44:56.165914 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:56.165716 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" event={"ID":"33adf1db-f9de-42b5-9438-d0dfcff1eaec","Type":"ContainerStarted","Data":"053d8fca26411fcbae5579d24f199b607954dce21e139483ccc942d761e49de9"} Jul 13 09:44:56.192150 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:56.192094 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" podStartSLOduration=1.192076634 podStartE2EDuration="1.192076634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 09:44:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:44:56.188635816 +0000 UTC m=+2180.068591097" watchObservedRunningTime="2026-07-13 09:44:56.192076634 +0000 UTC m=+2180.072031900" Jul 13 09:44:56.378545 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:56.378507 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:44:56.383724 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:56.383696 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:44:57.172973 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:57.172947 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:44:57.174192 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:57.174174 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:44:58.048102 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:58.048063 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:58.151069 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:58.151039 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:58.293532 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:58.293500 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:58.462670 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:58.462635 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:58.803617 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:58.803511 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:44:58.901136 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:44:58.901088 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:45:02.536847 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:02.536813 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:45:05.156418 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:05.156370 2552 patch_prober.go:28] interesting pod/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 13 09:45:05.156863 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:05.156460 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" podUID="2daaeacd-bce0-4663-885c-51bc852f3f12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 13 09:45:05.447541 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:05.447460 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:45:06.231825 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:06.231725 2552 generic.go:356] "Generic (PLEG): container finished" podID="2daaeacd-bce0-4663-885c-51bc852f3f12" containerID="9cf733cfd3401b85443ec54bafa3cdb85382152964a716e8f46d96de32f23a37" exitCode=0 Jul 13 09:45:06.231825 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:06.231796 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" event={"ID":"2daaeacd-bce0-4663-885c-51bc852f3f12","Type":"ContainerDied","Data":"9cf733cfd3401b85443ec54bafa3cdb85382152964a716e8f46d96de32f23a37"} Jul 13 09:45:06.232290 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:06.232248 2552 scope.go:117] "RemoveContainer" containerID="9cf733cfd3401b85443ec54bafa3cdb85382152964a716e8f46d96de32f23a37" Jul 13 09:45:06.233373 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:06.233341 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:45:06.233490 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:06.233415 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:45:06.233490 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:06.233445 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:45:07.240030 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:07.239989 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" event={"ID":"2daaeacd-bce0-4663-885c-51bc852f3f12","Type":"ContainerStarted","Data":"f55c3d4ae9f651bbd68a85f827028f9fd7fce97edc41d4ef3ee8a44f5ebdfa51"} Jul 13 09:45:07.382080 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:07.382023 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:45:07.482197 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:07.482127 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:45:07.875820 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:07.875455 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:45:07.876056 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:07.875870 2552 patch_prober.go:28] interesting pod/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Jul 13 09:45:07.876056 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:07.875940 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" podUID="2daaeacd-bce0-4663-885c-51bc852f3f12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Jul 13 09:45:08.875535 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:08.875489 2552 patch_prober.go:28] interesting pod/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Jul 13 09:45:08.876109 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:08.875560 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" podUID="2daaeacd-bce0-4663-885c-51bc852f3f12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Jul 13 09:45:09.876031 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:09.875983 2552 patch_prober.go:28] interesting pod/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Jul 13 09:45:09.876595 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:09.876063 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" podUID="2daaeacd-bce0-4663-885c-51bc852f3f12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Jul 13 09:45:10.875869 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:10.875823 2552 patch_prober.go:28] interesting pod/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Jul 13 09:45:10.876253 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:10.876194 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" podUID="2daaeacd-bce0-4663-885c-51bc852f3f12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Jul 13 09:45:11.669396 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:11.669358 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:45:11.770415 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:11.770375 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:45:11.879069 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:11.879032 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:45:11.879526 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:11.879326 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:45:11.880114 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:11.880095 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:45:13.214423 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:13.214381 2552 patch_prober.go:28] interesting pod/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 13 09:45:13.214834 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:13.214446 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" podUID="33adf1db-f9de-42b5-9438-d0dfcff1eaec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 13 09:45:14.288092 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:14.288059 2552 generic.go:356] "Generic (PLEG): container finished" podID="33adf1db-f9de-42b5-9438-d0dfcff1eaec" containerID="7ef95645f71a68f1dc348abddcdb1d29c3d311341ba77b19bea21c8108cfc97a" exitCode=0 Jul 13 09:45:14.288596 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:14.288139 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" event={"ID":"33adf1db-f9de-42b5-9438-d0dfcff1eaec","Type":"ContainerDied","Data":"7ef95645f71a68f1dc348abddcdb1d29c3d311341ba77b19bea21c8108cfc97a"} Jul 13 09:45:14.288651 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:14.288629 2552 scope.go:117] "RemoveContainer" containerID="7ef95645f71a68f1dc348abddcdb1d29c3d311341ba77b19bea21c8108cfc97a" Jul 13 09:45:14.289777 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:14.289746 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:45:14.289849 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:14.289838 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:45:14.289888 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:14.289875 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 13 09:45:15.296645 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:15.296609 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" event={"ID":"33adf1db-f9de-42b5-9438-d0dfcff1eaec","Type":"ContainerStarted","Data":"b308a70b0b84443333dc317df688a2aca2249c61e971b055e93cdf11b4c577cf"} Jul 13 09:45:15.379788 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:15.379198 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:45:15.379788 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:15.379374 2552 patch_prober.go:28] interesting pod/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Jul 13 09:45:15.379788 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:15.379465 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" podUID="33adf1db-f9de-42b5-9438-d0dfcff1eaec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Jul 13 09:45:16.378679 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:16.378626 2552 patch_prober.go:28] interesting pod/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Jul 13 09:45:16.379232 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:16.378755 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" podUID="33adf1db-f9de-42b5-9438-d0dfcff1eaec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Jul 13 09:45:17.378787 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:17.378737 2552 patch_prober.go:28] interesting pod/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Jul 13 09:45:17.379325 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:17.378821 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" podUID="33adf1db-f9de-42b5-9438-d0dfcff1eaec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Jul 13 09:45:18.379344 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:18.379293 2552 patch_prober.go:28] interesting pod/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Jul 13 09:45:18.379897 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:18.379368 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" podUID="33adf1db-f9de-42b5-9438-d0dfcff1eaec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Jul 13 09:45:19.378792 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:19.378757 2552 patch_prober.go:28] interesting pod/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Jul 13 09:45:19.378996 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:19.378815 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" podUID="33adf1db-f9de-42b5-9438-d0dfcff1eaec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Jul 13 09:45:19.630320 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:19.630199 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:45:19.730443 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:19.730405 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:45:20.382784 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:20.382746 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:45:20.383069 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:20.383043 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:45:20.383708 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:45:20.383687 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:48:07.392195 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:07.392109 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:48:08.633074 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:08.633037 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:48:13.841323 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:13.841283 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:48:13.941477 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:13.941437 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:48:14.105689 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:14.105617 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb"] Jul 13 09:48:14.105833 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:14.105806 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" podUID="2daaeacd-bce0-4663-885c-51bc852f3f12" containerName="istio-proxy" containerID="cri-o://f55c3d4ae9f651bbd68a85f827028f9fd7fce97edc41d4ef3ee8a44f5ebdfa51" gracePeriod=30 Jul 13 09:48:14.128944 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:14.128913 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:48:14.548536 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:14.548449 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:48:14.941448 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:14.941417 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:48:15.041698 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:15.041661 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:48:15.143399 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:15.143367 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:48:15.367325 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:15.367272 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:48:19.370519 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.370456 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:48:19.411673 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.411641 2552 generic.go:356] "Generic (PLEG): container finished" podID="2daaeacd-bce0-4663-885c-51bc852f3f12" containerID="f55c3d4ae9f651bbd68a85f827028f9fd7fce97edc41d4ef3ee8a44f5ebdfa51" exitCode=0 Jul 13 09:48:19.411798 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.411682 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" Jul 13 09:48:19.411798 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.411726 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" event={"ID":"2daaeacd-bce0-4663-885c-51bc852f3f12","Type":"ContainerDied","Data":"f55c3d4ae9f651bbd68a85f827028f9fd7fce97edc41d4ef3ee8a44f5ebdfa51"} Jul 13 09:48:19.411798 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.411780 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb" event={"ID":"2daaeacd-bce0-4663-885c-51bc852f3f12","Type":"ContainerDied","Data":"82020d5ef17482f8f2629426207599bc5133527d9f6c2c613e0a357b40da15f6"} Jul 13 09:48:19.411915 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.411810 2552 scope.go:117] "RemoveContainer" containerID="f55c3d4ae9f651bbd68a85f827028f9fd7fce97edc41d4ef3ee8a44f5ebdfa51" Jul 13 09:48:19.421401 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.421371 2552 scope.go:117] "RemoveContainer" containerID="9cf733cfd3401b85443ec54bafa3cdb85382152964a716e8f46d96de32f23a37" Jul 13 09:48:19.429785 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.429766 2552 scope.go:117] "RemoveContainer" containerID="f55c3d4ae9f651bbd68a85f827028f9fd7fce97edc41d4ef3ee8a44f5ebdfa51" Jul 13 09:48:19.430059 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:48:19.430039 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f55c3d4ae9f651bbd68a85f827028f9fd7fce97edc41d4ef3ee8a44f5ebdfa51\": container with ID starting with f55c3d4ae9f651bbd68a85f827028f9fd7fce97edc41d4ef3ee8a44f5ebdfa51 not found: ID does not exist" containerID="f55c3d4ae9f651bbd68a85f827028f9fd7fce97edc41d4ef3ee8a44f5ebdfa51" Jul 13 09:48:19.430154 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.430064 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f55c3d4ae9f651bbd68a85f827028f9fd7fce97edc41d4ef3ee8a44f5ebdfa51"} err="failed to get container status \"f55c3d4ae9f651bbd68a85f827028f9fd7fce97edc41d4ef3ee8a44f5ebdfa51\": rpc error: code = NotFound desc = could not find container \"f55c3d4ae9f651bbd68a85f827028f9fd7fce97edc41d4ef3ee8a44f5ebdfa51\": container with ID starting with f55c3d4ae9f651bbd68a85f827028f9fd7fce97edc41d4ef3ee8a44f5ebdfa51 not found: ID does not exist" Jul 13 09:48:19.430154 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.430080 2552 scope.go:117] "RemoveContainer" containerID="9cf733cfd3401b85443ec54bafa3cdb85382152964a716e8f46d96de32f23a37" Jul 13 09:48:19.430345 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:48:19.430326 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cf733cfd3401b85443ec54bafa3cdb85382152964a716e8f46d96de32f23a37\": container with ID starting with 9cf733cfd3401b85443ec54bafa3cdb85382152964a716e8f46d96de32f23a37 not found: ID does not exist" containerID="9cf733cfd3401b85443ec54bafa3cdb85382152964a716e8f46d96de32f23a37" Jul 13 09:48:19.430409 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.430356 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cf733cfd3401b85443ec54bafa3cdb85382152964a716e8f46d96de32f23a37"} err="failed to get container status \"9cf733cfd3401b85443ec54bafa3cdb85382152964a716e8f46d96de32f23a37\": rpc error: code = NotFound desc = could not find container \"9cf733cfd3401b85443ec54bafa3cdb85382152964a716e8f46d96de32f23a37\": container with ID starting with 9cf733cfd3401b85443ec54bafa3cdb85382152964a716e8f46d96de32f23a37 not found: ID does not exist" Jul 13 09:48:19.458881 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.458856 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2daaeacd-bce0-4663-885c-51bc852f3f12-workload-socket\") pod \"2daaeacd-bce0-4663-885c-51bc852f3f12\" (UID: \"2daaeacd-bce0-4663-885c-51bc852f3f12\") " Jul 13 09:48:19.458988 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.458904 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2daaeacd-bce0-4663-885c-51bc852f3f12-istiod-ca-cert\") pod \"2daaeacd-bce0-4663-885c-51bc852f3f12\" (UID: \"2daaeacd-bce0-4663-885c-51bc852f3f12\") " Jul 13 09:48:19.458988 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.458943 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2daaeacd-bce0-4663-885c-51bc852f3f12-workload-certs\") pod \"2daaeacd-bce0-4663-885c-51bc852f3f12\" (UID: \"2daaeacd-bce0-4663-885c-51bc852f3f12\") " Jul 13 09:48:19.459110 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.459057 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2daaeacd-bce0-4663-885c-51bc852f3f12-istio-data\") pod \"2daaeacd-bce0-4663-885c-51bc852f3f12\" (UID: \"2daaeacd-bce0-4663-885c-51bc852f3f12\") " Jul 13 09:48:19.459167 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.459106 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2daaeacd-bce0-4663-885c-51bc852f3f12-istio-envoy\") pod \"2daaeacd-bce0-4663-885c-51bc852f3f12\" (UID: \"2daaeacd-bce0-4663-885c-51bc852f3f12\") " Jul 13 09:48:19.459167 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.459117 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2daaeacd-bce0-4663-885c-51bc852f3f12-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2daaeacd-bce0-4663-885c-51bc852f3f12" (UID: "2daaeacd-bce0-4663-885c-51bc852f3f12"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:48:19.459167 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.459139 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2daaeacd-bce0-4663-885c-51bc852f3f12-istio-token\") pod \"2daaeacd-bce0-4663-885c-51bc852f3f12\" (UID: \"2daaeacd-bce0-4663-885c-51bc852f3f12\") " Jul 13 09:48:19.459167 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.459164 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wgtg7\" (UniqueName: \"kubernetes.io/projected/2daaeacd-bce0-4663-885c-51bc852f3f12-kube-api-access-wgtg7\") pod \"2daaeacd-bce0-4663-885c-51bc852f3f12\" (UID: \"2daaeacd-bce0-4663-885c-51bc852f3f12\") " Jul 13 09:48:19.459419 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.459247 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2daaeacd-bce0-4663-885c-51bc852f3f12-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2daaeacd-bce0-4663-885c-51bc852f3f12" (UID: "2daaeacd-bce0-4663-885c-51bc852f3f12"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:48:19.459419 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.459262 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2daaeacd-bce0-4663-885c-51bc852f3f12-credential-socket\") pod \"2daaeacd-bce0-4663-885c-51bc852f3f12\" (UID: \"2daaeacd-bce0-4663-885c-51bc852f3f12\") " Jul 13 09:48:19.459419 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.459323 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2daaeacd-bce0-4663-885c-51bc852f3f12-istio-podinfo\") pod \"2daaeacd-bce0-4663-885c-51bc852f3f12\" (UID: \"2daaeacd-bce0-4663-885c-51bc852f3f12\") " Jul 13 09:48:19.459419 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.459333 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2daaeacd-bce0-4663-885c-51bc852f3f12-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2daaeacd-bce0-4663-885c-51bc852f3f12" (UID: "2daaeacd-bce0-4663-885c-51bc852f3f12"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 09:48:19.459635 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.459613 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2daaeacd-bce0-4663-885c-51bc852f3f12-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2daaeacd-bce0-4663-885c-51bc852f3f12" (UID: "2daaeacd-bce0-4663-885c-51bc852f3f12"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:48:19.459772 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.459704 2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2daaeacd-bce0-4663-885c-51bc852f3f12-credential-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:48:19.459772 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.459725 2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2daaeacd-bce0-4663-885c-51bc852f3f12-workload-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:48:19.459772 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.459738 2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2daaeacd-bce0-4663-885c-51bc852f3f12-istiod-ca-cert\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:48:19.459772 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.459752 2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2daaeacd-bce0-4663-885c-51bc852f3f12-workload-certs\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:48:19.460526 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.460499 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2daaeacd-bce0-4663-885c-51bc852f3f12-istio-data" (OuterVolumeSpecName: "istio-data") pod "2daaeacd-bce0-4663-885c-51bc852f3f12" (UID: "2daaeacd-bce0-4663-885c-51bc852f3f12"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:48:19.461509 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.461486 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2daaeacd-bce0-4663-885c-51bc852f3f12-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2daaeacd-bce0-4663-885c-51bc852f3f12" (UID: "2daaeacd-bce0-4663-885c-51bc852f3f12"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:48:19.461726 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.461703 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2daaeacd-bce0-4663-885c-51bc852f3f12-kube-api-access-wgtg7" (OuterVolumeSpecName: "kube-api-access-wgtg7") pod "2daaeacd-bce0-4663-885c-51bc852f3f12" (UID: "2daaeacd-bce0-4663-885c-51bc852f3f12"). InnerVolumeSpecName "kube-api-access-wgtg7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:48:19.461946 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.461922 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2daaeacd-bce0-4663-885c-51bc852f3f12-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2daaeacd-bce0-4663-885c-51bc852f3f12" (UID: "2daaeacd-bce0-4663-885c-51bc852f3f12"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 09:48:19.462048 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.461939 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2daaeacd-bce0-4663-885c-51bc852f3f12-istio-token" (OuterVolumeSpecName: "istio-token") pod "2daaeacd-bce0-4663-885c-51bc852f3f12" (UID: "2daaeacd-bce0-4663-885c-51bc852f3f12"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:48:19.561130 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.561101 2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2daaeacd-bce0-4663-885c-51bc852f3f12-istio-envoy\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:48:19.561130 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.561130 2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2daaeacd-bce0-4663-885c-51bc852f3f12-istio-token\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:48:19.561323 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.561139 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wgtg7\" (UniqueName: \"kubernetes.io/projected/2daaeacd-bce0-4663-885c-51bc852f3f12-kube-api-access-wgtg7\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:48:19.561323 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.561148 2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2daaeacd-bce0-4663-885c-51bc852f3f12-istio-podinfo\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:48:19.561323 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.561156 2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2daaeacd-bce0-4663-885c-51bc852f3f12-istio-data\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:48:19.736621 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.736593 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb"] Jul 13 09:48:19.740839 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:19.740809 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-f77299-openshift-default-66f8cd5ff-r7slb"] Jul 13 09:48:20.666781 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:20.666747 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2daaeacd-bce0-4663-885c-51bc852f3f12" path="/var/lib/kubelet/pods/2daaeacd-bce0-4663-885c-51bc852f3f12/volumes" Jul 13 09:48:20.839628 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:20.839594 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:48:20.913961 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:20.913930 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:48:21.071022 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:21.070949 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c"] Jul 13 09:48:21.071193 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:21.071168 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" podUID="33adf1db-f9de-42b5-9438-d0dfcff1eaec" containerName="istio-proxy" containerID="cri-o://b308a70b0b84443333dc317df688a2aca2249c61e971b055e93cdf11b4c577cf" gracePeriod=30 Jul 13 09:48:21.095679 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:21.095649 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:48:21.956836 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:21.956801 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-67757b8bf-58t5v_7c2323b5-0d28-4d00-a77a-b880a3920ffd/authorino/0.log" Jul 13 09:48:26.331030 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.330971 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:48:26.420117 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.420087 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33adf1db-f9de-42b5-9438-d0dfcff1eaec-istio-envoy\") pod \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\" (UID: \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\") " Jul 13 09:48:26.420290 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.420141 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33adf1db-f9de-42b5-9438-d0dfcff1eaec-workload-certs\") pod \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\" (UID: \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\") " Jul 13 09:48:26.420290 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.420172 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33adf1db-f9de-42b5-9438-d0dfcff1eaec-credential-socket\") pod \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\" (UID: \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\") " Jul 13 09:48:26.420290 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.420234 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33adf1db-f9de-42b5-9438-d0dfcff1eaec-istiod-ca-cert\") pod \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\" (UID: \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\") " Jul 13 09:48:26.420290 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.420259 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33adf1db-f9de-42b5-9438-d0dfcff1eaec-istio-data\") pod \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\" (UID: \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\") " Jul 13 09:48:26.420290 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.420290 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wchrh\" (UniqueName: \"kubernetes.io/projected/33adf1db-f9de-42b5-9438-d0dfcff1eaec-kube-api-access-wchrh\") pod \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\" (UID: \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\") " Jul 13 09:48:26.420857 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.420346 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33adf1db-f9de-42b5-9438-d0dfcff1eaec-istio-podinfo\") pod \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\" (UID: \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\") " Jul 13 09:48:26.420857 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.420379 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33adf1db-f9de-42b5-9438-d0dfcff1eaec-workload-socket\") pod \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\" (UID: \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\") " Jul 13 09:48:26.420857 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.420415 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33adf1db-f9de-42b5-9438-d0dfcff1eaec-istio-token\") pod \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\" (UID: \"33adf1db-f9de-42b5-9438-d0dfcff1eaec\") " Jul 13 09:48:26.420857 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.420454 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33adf1db-f9de-42b5-9438-d0dfcff1eaec-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "33adf1db-f9de-42b5-9438-d0dfcff1eaec" (UID: "33adf1db-f9de-42b5-9438-d0dfcff1eaec"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:48:26.420857 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.420501 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33adf1db-f9de-42b5-9438-d0dfcff1eaec-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "33adf1db-f9de-42b5-9438-d0dfcff1eaec" (UID: "33adf1db-f9de-42b5-9438-d0dfcff1eaec"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:48:26.420857 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.420677 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33adf1db-f9de-42b5-9438-d0dfcff1eaec-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "33adf1db-f9de-42b5-9438-d0dfcff1eaec" (UID: "33adf1db-f9de-42b5-9438-d0dfcff1eaec"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:48:26.420857 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.420713 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33adf1db-f9de-42b5-9438-d0dfcff1eaec-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "33adf1db-f9de-42b5-9438-d0dfcff1eaec" (UID: "33adf1db-f9de-42b5-9438-d0dfcff1eaec"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 09:48:26.420857 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.420769 2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33adf1db-f9de-42b5-9438-d0dfcff1eaec-workload-certs\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:48:26.420857 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.420791 2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33adf1db-f9de-42b5-9438-d0dfcff1eaec-credential-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:48:26.420857 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.420806 2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33adf1db-f9de-42b5-9438-d0dfcff1eaec-workload-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:48:26.421520 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.421491 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33adf1db-f9de-42b5-9438-d0dfcff1eaec-istio-data" (OuterVolumeSpecName: "istio-data") pod "33adf1db-f9de-42b5-9438-d0dfcff1eaec" (UID: "33adf1db-f9de-42b5-9438-d0dfcff1eaec"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:48:26.422697 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.422667 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33adf1db-f9de-42b5-9438-d0dfcff1eaec-istio-token" (OuterVolumeSpecName: "istio-token") pod "33adf1db-f9de-42b5-9438-d0dfcff1eaec" (UID: "33adf1db-f9de-42b5-9438-d0dfcff1eaec"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:48:26.422697 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.422674 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33adf1db-f9de-42b5-9438-d0dfcff1eaec-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "33adf1db-f9de-42b5-9438-d0dfcff1eaec" (UID: "33adf1db-f9de-42b5-9438-d0dfcff1eaec"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 09:48:26.422833 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.422760 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33adf1db-f9de-42b5-9438-d0dfcff1eaec-kube-api-access-wchrh" (OuterVolumeSpecName: "kube-api-access-wchrh") pod "33adf1db-f9de-42b5-9438-d0dfcff1eaec" (UID: "33adf1db-f9de-42b5-9438-d0dfcff1eaec"). InnerVolumeSpecName "kube-api-access-wchrh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 09:48:26.422833 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.422779 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/33adf1db-f9de-42b5-9438-d0dfcff1eaec-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "33adf1db-f9de-42b5-9438-d0dfcff1eaec" (UID: "33adf1db-f9de-42b5-9438-d0dfcff1eaec"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 09:48:26.453925 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.453902 2552 generic.go:356] "Generic (PLEG): container finished" podID="33adf1db-f9de-42b5-9438-d0dfcff1eaec" containerID="b308a70b0b84443333dc317df688a2aca2249c61e971b055e93cdf11b4c577cf" exitCode=0 Jul 13 09:48:26.454034 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.453955 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" Jul 13 09:48:26.454034 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.453960 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" event={"ID":"33adf1db-f9de-42b5-9438-d0dfcff1eaec","Type":"ContainerDied","Data":"b308a70b0b84443333dc317df688a2aca2249c61e971b055e93cdf11b4c577cf"} Jul 13 09:48:26.454034 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.454011 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c" event={"ID":"33adf1db-f9de-42b5-9438-d0dfcff1eaec","Type":"ContainerDied","Data":"053d8fca26411fcbae5579d24f199b607954dce21e139483ccc942d761e49de9"} Jul 13 09:48:26.454034 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.454034 2552 scope.go:117] "RemoveContainer" containerID="b308a70b0b84443333dc317df688a2aca2249c61e971b055e93cdf11b4c577cf" Jul 13 09:48:26.464752 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.464730 2552 scope.go:117] "RemoveContainer" containerID="7ef95645f71a68f1dc348abddcdb1d29c3d311341ba77b19bea21c8108cfc97a" Jul 13 09:48:26.472983 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.472965 2552 scope.go:117] "RemoveContainer" containerID="b308a70b0b84443333dc317df688a2aca2249c61e971b055e93cdf11b4c577cf" Jul 13 09:48:26.473253 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:48:26.473237 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b308a70b0b84443333dc317df688a2aca2249c61e971b055e93cdf11b4c577cf\": container with ID starting with b308a70b0b84443333dc317df688a2aca2249c61e971b055e93cdf11b4c577cf not found: ID does not exist" containerID="b308a70b0b84443333dc317df688a2aca2249c61e971b055e93cdf11b4c577cf" Jul 13 09:48:26.473306 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.473260 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b308a70b0b84443333dc317df688a2aca2249c61e971b055e93cdf11b4c577cf"} err="failed to get container status \"b308a70b0b84443333dc317df688a2aca2249c61e971b055e93cdf11b4c577cf\": rpc error: code = NotFound desc = could not find container \"b308a70b0b84443333dc317df688a2aca2249c61e971b055e93cdf11b4c577cf\": container with ID starting with b308a70b0b84443333dc317df688a2aca2249c61e971b055e93cdf11b4c577cf not found: ID does not exist" Jul 13 09:48:26.473306 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.473275 2552 scope.go:117] "RemoveContainer" containerID="7ef95645f71a68f1dc348abddcdb1d29c3d311341ba77b19bea21c8108cfc97a" Jul 13 09:48:26.473526 ip-10-0-129-99 kubenswrapper[2552]: E0713 09:48:26.473508 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ef95645f71a68f1dc348abddcdb1d29c3d311341ba77b19bea21c8108cfc97a\": container with ID starting with 7ef95645f71a68f1dc348abddcdb1d29c3d311341ba77b19bea21c8108cfc97a not found: ID does not exist" containerID="7ef95645f71a68f1dc348abddcdb1d29c3d311341ba77b19bea21c8108cfc97a" Jul 13 09:48:26.473579 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.473532 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ef95645f71a68f1dc348abddcdb1d29c3d311341ba77b19bea21c8108cfc97a"} err="failed to get container status \"7ef95645f71a68f1dc348abddcdb1d29c3d311341ba77b19bea21c8108cfc97a\": rpc error: code = NotFound desc = could not find container \"7ef95645f71a68f1dc348abddcdb1d29c3d311341ba77b19bea21c8108cfc97a\": container with ID starting with 7ef95645f71a68f1dc348abddcdb1d29c3d311341ba77b19bea21c8108cfc97a not found: ID does not exist" Jul 13 09:48:26.477706 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.477687 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c"] Jul 13 09:48:26.484194 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.484177 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-7bf32e-openshift-default-59b9d87475-g8t2c"] Jul 13 09:48:26.521965 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.521948 2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33adf1db-f9de-42b5-9438-d0dfcff1eaec-istiod-ca-cert\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:48:26.521965 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.521967 2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33adf1db-f9de-42b5-9438-d0dfcff1eaec-istio-data\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:48:26.522085 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.521976 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wchrh\" (UniqueName: \"kubernetes.io/projected/33adf1db-f9de-42b5-9438-d0dfcff1eaec-kube-api-access-wchrh\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:48:26.522085 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.521984 2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33adf1db-f9de-42b5-9438-d0dfcff1eaec-istio-podinfo\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:48:26.522085 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.521993 2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33adf1db-f9de-42b5-9438-d0dfcff1eaec-istio-token\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:48:26.522085 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.522000 2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33adf1db-f9de-42b5-9438-d0dfcff1eaec-istio-envoy\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\"" Jul 13 09:48:26.677194 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.677129 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33adf1db-f9de-42b5-9438-d0dfcff1eaec" path="/var/lib/kubelet/pods/33adf1db-f9de-42b5-9438-d0dfcff1eaec/volumes" Jul 13 09:48:26.749002 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:26.748980 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-xxjmb_0c3c450a-0391-4d1c-9db9-b9c1d3215a42/manager/0.log" Jul 13 09:48:27.125649 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:27.125623 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-lx9jr_70f4fd41-cd64-4ba4-9bc4-95a4c77971e0/server/0.log" Jul 13 09:48:27.248657 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:27.248626 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-gm9js_75c93628-a23e-4cb5-98eb-3134b0a39bec/manager/0.log" Jul 13 09:48:27.628001 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:27.627978 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4_c7e7373d-bf27-43fd-ace3-776f08e8c726/copy-manifests/0.log" Jul 13 09:48:27.635231 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:27.635191 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4_c7e7373d-bf27-43fd-ace3-776f08e8c726/manager/0.log" Jul 13 09:48:28.414856 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:28.414820 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gzc8d_cf0b4d43-1b14-4990-867d-7302291b8628/util/0.log" Jul 13 09:48:28.421032 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:28.421007 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gzc8d_cf0b4d43-1b14-4990-867d-7302291b8628/pull/0.log" Jul 13 09:48:28.427170 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:28.427146 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gzc8d_cf0b4d43-1b14-4990-867d-7302291b8628/extract/0.log" Jul 13 09:48:28.532380 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:28.532359 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g5qcn_f2d9fb3a-d67b-4422-a065-ad6c163aa1fb/util/0.log" Jul 13 09:48:28.539192 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:28.539173 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g5qcn_f2d9fb3a-d67b-4422-a065-ad6c163aa1fb/pull/0.log" Jul 13 09:48:28.546031 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:28.546013 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g5qcn_f2d9fb3a-d67b-4422-a065-ad6c163aa1fb/extract/0.log" Jul 13 09:48:28.663582 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:28.663560 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739v6hc_2e20a36e-85ef-4cf3-b242-318b75700abe/util/0.log" Jul 13 09:48:28.670188 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:28.670132 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739v6hc_2e20a36e-85ef-4cf3-b242-318b75700abe/pull/0.log" Jul 13 09:48:28.676330 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:28.676310 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739v6hc_2e20a36e-85ef-4cf3-b242-318b75700abe/extract/0.log" Jul 13 09:48:28.783067 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:28.783047 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bgqtp_1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f/util/0.log" Jul 13 09:48:28.788732 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:28.788712 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bgqtp_1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f/pull/0.log" Jul 13 09:48:28.793974 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:28.793955 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bgqtp_1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f/extract/0.log" Jul 13 09:48:28.922281 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:28.922198 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-67757b8bf-58t5v_7c2323b5-0d28-4d00-a77a-b880a3920ffd/authorino/0.log" Jul 13 09:48:29.044713 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:29.044692 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-h956b_4da2f93d-c905-4cc7-a2ef-509625c2a4f0/manager/0.log" Jul 13 09:48:29.159051 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:29.159028 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-9dsj7_99c56ba6-b6d4-42f1-a1cf-6212e111544b/manager/0.log" Jul 13 09:48:29.269982 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:29.269922 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-5jjbn_12a4f316-ca65-40dd-a193-106f0fc79e01/kuadrant-console-plugin/0.log" Jul 13 09:48:29.401245 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:29.401225 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-h6cpr_ed5c28f2-3236-47ce-9665-1c74fc79b889/registry-server/0.log" Jul 13 09:48:29.539636 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:29.539569 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-ms4nv_aaaf1412-55cc-4083-b4ea-d1b574a2a3b9/manager/0.log" Jul 13 09:48:29.815103 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:29.815024 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-cth6m_ba153729-3fe0-4ca7-a7f5-deec6c3586a0/manager/0.log" Jul 13 09:48:30.191583 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:30.191560 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4wzv25_5ccd1a61-cd7d-41c1-8236-76d04bc28b65/istio-proxy/0.log" Jul 13 09:48:30.445438 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:30.445365 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d64b4d488-d2lnz_d895d638-812a-437a-a2b0-5cb7a8eb4e0c/kube-auth-proxy/0.log" Jul 13 09:48:30.706812 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:30.706729 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-cmw8z_8863895d-7df7-4937-a6a5-06472e72737a/istio-proxy/1.log" Jul 13 09:48:31.079297 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:31.079261 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-77b8c6656d-wxkr8_b9a3e094-4012-4054-83f7-9e753c83e8ad/router/0.log" Jul 13 09:48:31.575020 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:31.574949 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-6kx2s_902feeb3-2172-433a-8d33-5a86d16d1513/main/0.log" Jul 13 09:48:32.063996 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:32.063970 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-nx2zr_7724876e-e287-4b92-b0bc-7d75618ea5ec/main/0.log" Jul 13 09:48:36.789423 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:36.789387 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:48:36.802955 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:36.802931 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gp6mv_47250d22-56b6-4503-af3f-36a9e79691b2/ovn-acl-logging/0.log" Jul 13 09:48:36.807827 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:36.807809 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gp6mv_47250d22-56b6-4503-af3f-36a9e79691b2/ovn-acl-logging/0.log" Jul 13 09:48:36.863294 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:36.863270 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:48:39.606399 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:39.606362 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wxkd7_708d0abc-11a7-403e-add0-08c1d93e33f8/global-pull-secret-syncer/0.log" Jul 13 09:48:39.659754 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:39.659733 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" Jul 13 09:48:39.761298 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:39.761267 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-99.ec2.internal_bbafb723d47a725ee02db2c015defda8/haproxy/0.log" Jul 13 09:48:44.199571 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:44.199533 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gzc8d_cf0b4d43-1b14-4990-867d-7302291b8628/extract/0.log" Jul 13 09:48:44.223772 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:44.223746 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gzc8d_cf0b4d43-1b14-4990-867d-7302291b8628/util/0.log" Jul 13 09:48:44.245307 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:44.245285 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gzc8d_cf0b4d43-1b14-4990-867d-7302291b8628/pull/0.log" Jul 13 09:48:44.272807 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:44.272785 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g5qcn_f2d9fb3a-d67b-4422-a065-ad6c163aa1fb/extract/0.log" Jul 13 09:48:44.300630 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:44.300609 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g5qcn_f2d9fb3a-d67b-4422-a065-ad6c163aa1fb/util/0.log" Jul 13 09:48:44.331099 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:44.331075 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g5qcn_f2d9fb3a-d67b-4422-a065-ad6c163aa1fb/pull/0.log" Jul 13 09:48:44.357562 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:44.357540 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739v6hc_2e20a36e-85ef-4cf3-b242-318b75700abe/extract/0.log" Jul 13 09:48:44.384068 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:44.384048 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739v6hc_2e20a36e-85ef-4cf3-b242-318b75700abe/util/0.log" Jul 13 09:48:44.408609 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:44.408590 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739v6hc_2e20a36e-85ef-4cf3-b242-318b75700abe/pull/0.log" Jul 13 09:48:44.445308 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:44.445287 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bgqtp_1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f/extract/0.log" Jul 13 09:48:44.468777 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:44.468718 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bgqtp_1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f/util/0.log" Jul 13 09:48:44.491402 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:44.491382 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bgqtp_1d6c1ccf-a6aa-483f-94aa-a5fa8a316b5f/pull/0.log" Jul 13 09:48:44.947529 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:44.947494 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-67757b8bf-58t5v_7c2323b5-0d28-4d00-a77a-b880a3920ffd/authorino/0.log" Jul 13 09:48:44.996953 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:44.996920 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-h956b_4da2f93d-c905-4cc7-a2ef-509625c2a4f0/manager/0.log" Jul 13 09:48:45.025220 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:45.025186 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-9dsj7_99c56ba6-b6d4-42f1-a1cf-6212e111544b/manager/0.log" Jul 13 09:48:45.071998 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:45.071979 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-5jjbn_12a4f316-ca65-40dd-a193-106f0fc79e01/kuadrant-console-plugin/0.log" Jul 13 09:48:45.119373 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:45.119344 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-h6cpr_ed5c28f2-3236-47ce-9665-1c74fc79b889/registry-server/0.log" Jul 13 09:48:45.207512 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:45.207437 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c6q7b_a57ce5c3-5f12-4858-9a2c-be4bcc36dd9d/konnectivity-agent/0.log" limit=1024 Jul 13 09:48:45.242336 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:45.242310 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-ms4nv_aaaf1412-55cc-4083-b4ea-d1b574a2a3b9/manager/0.log" Jul 13 09:48:45.348062 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:45.348035 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-cth6m_ba153729-3fe0-4ca7-a7f5-deec6c3586a0/manager/0.log" Jul 13 09:48:47.223678 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:47.223652 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-djfbm_88805853-a812-47a7-81a0-9f96bf4d16ae/kube-state-metrics/0.log" Jul 13 09:48:47.242866 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:47.242844 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-djfbm_88805853-a812-47a7-81a0-9f96bf4d16ae/kube-rbac-proxy-main/0.log" Jul 13 09:48:47.264508 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:47.264486 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-djfbm_88805853-a812-47a7-81a0-9f96bf4d16ae/kube-rbac-proxy-self/0.log" Jul 13 09:48:47.316010 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:47.315985 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-wjjsm_d8e5e06f-6b18-4e06-be34-bc4df8d4bf87/monitoring-plugin/0.log" Jul 13 09:48:47.545137 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:47.545051 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r5rh6_5def036b-13e7-4089-bfe0-3847346da725/node-exporter/0.log" Jul 13 09:48:47.567399 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:47.567372 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r5rh6_5def036b-13e7-4089-bfe0-3847346da725/kube-rbac-proxy/0.log" Jul 13 09:48:47.590886 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:47.590870 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r5rh6_5def036b-13e7-4089-bfe0-3847346da725/init-textfile/0.log" Jul 13 09:48:47.623311 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:47.623289 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-lmn8p_e21df076-b466-44be-afdc-ee35d2749a00/kube-rbac-proxy-main/0.log" Jul 13 09:48:47.643640 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:47.643619 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-lmn8p_e21df076-b466-44be-afdc-ee35d2749a00/kube-rbac-proxy-self/0.log" Jul 13 09:48:47.669299 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:47.669274 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-lmn8p_e21df076-b466-44be-afdc-ee35d2749a00/openshift-state-metrics/0.log" Jul 13 09:48:47.901703 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:47.901681 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-crsd7_86ebac58-f50b-4e17-8153-b9e3bfbed54e/prometheus-operator/0.log" Jul 13 09:48:47.922673 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:47.922651 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-crsd7_86ebac58-f50b-4e17-8153-b9e3bfbed54e/kube-rbac-proxy/0.log" Jul 13 09:48:47.945100 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:47.945082 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-bnqz9_c02bf867-897b-4144-bcf6-de76c08b030d/prometheus-operator-admission-webhook/0.log" Jul 13 09:48:49.488249 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:49.488202 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gb9tq/perf-node-gather-daemonset-8wm82"] Jul 13 09:48:49.488607 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:49.488571 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33adf1db-f9de-42b5-9438-d0dfcff1eaec" containerName="istio-proxy" Jul 13 09:48:49.488607 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:49.488582 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="33adf1db-f9de-42b5-9438-d0dfcff1eaec" containerName="istio-proxy" Jul 13 09:48:49.488607 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:49.488595 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa610d95-d7d5-4caa-92e4-acee90bd6bab" containerName="istio-proxy" Jul 13 09:48:49.488607 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:49.488601 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa610d95-d7d5-4caa-92e4-acee90bd6bab" containerName="istio-proxy" Jul 13 09:48:49.488754 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:49.488613 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2daaeacd-bce0-4663-885c-51bc852f3f12" containerName="istio-proxy" Jul 13 09:48:49.488754 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:49.488618 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="2daaeacd-bce0-4663-885c-51bc852f3f12" containerName="istio-proxy" Jul 13 09:48:49.488754 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:49.488625 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33adf1db-f9de-42b5-9438-d0dfcff1eaec" containerName="istio-proxy" Jul 13 09:48:49.488754 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:49.488630 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="33adf1db-f9de-42b5-9438-d0dfcff1eaec" containerName="istio-proxy" Jul 13 09:48:49.488754 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:49.488644 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2daaeacd-bce0-4663-885c-51bc852f3f12" containerName="istio-proxy" Jul 13 09:48:49.488754 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:49.488648 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="2daaeacd-bce0-4663-885c-51bc852f3f12" containerName="istio-proxy" Jul 13 09:48:49.488754 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:49.488711 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="2daaeacd-bce0-4663-885c-51bc852f3f12" containerName="istio-proxy" Jul 13 09:48:49.488754 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:49.488718 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="33adf1db-f9de-42b5-9438-d0dfcff1eaec" containerName="istio-proxy" Jul 13 09:48:49.488754 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:49.488725 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="33adf1db-f9de-42b5-9438-d0dfcff1eaec" containerName="istio-proxy" Jul 13 09:48:49.491900 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:49.491884 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gb9tq/perf-node-gather-daemonset-8wm82" Jul 13 09:48:49.496719 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:49.496700 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gb9tq\"/\"kube-root-ca.crt\"" Jul 13 09:48:49.497667 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:49.497646 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gb9tq\"/\"default-dockercfg-6th65\"" Jul 13 09:48:49.497780 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:49.497645 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gb9tq\"/\"openshift-service-ca.crt\"" Jul 13 09:48:49.510070 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:49.510049 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gb9tq/perf-node-gather-daemonset-8wm82"] Jul 13 09:48:49.608904 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:49.608880 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77b501fc-ef62-4552-ad41-c85b1e05199b-sys\") pod \"perf-node-gather-daemonset-8wm82\" (UID: \"77b501fc-ef62-4552-ad41-c85b1e05199b\") " pod="openshift-must-gather-gb9tq/perf-node-gather-daemonset-8wm82" Jul 13 09:48:49.609009 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:49.608909 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/77b501fc-ef62-4552-ad41-c85b1e05199b-lib-modules\") pod \"perf-node-gather-daemonset-8wm82\" (UID: \"77b501fc-ef62-4552-ad41-c85b1e05199b\") " pod="openshift-must-gather-gb9tq/perf-node-gather-daemonset-8wm82" Jul 13 09:48:49.609009 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:49.608924 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/77b501fc-ef62-4552-ad41-c85b1e05199b-proc\") pod \"perf-node-gather-daemonset-8wm82\" (UID: \"77b501fc-ef62-4552-ad41-c85b1e05199b\") " pod="openshift-must-gather-gb9tq/perf-node-gather-daemonset-8wm82" Jul 13 09:48:49.609083 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:49.609002 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/77b501fc-ef62-4552-ad41-c85b1e05199b-podres\") pod \"perf-node-gather-daemonset-8wm82\" (UID: \"77b501fc-ef62-4552-ad41-c85b1e05199b\") " pod="openshift-must-gather-gb9tq/perf-node-gather-daemonset-8wm82" Jul 13 09:48:49.609083 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:49.609065 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dw9nx\" (UniqueName: \"kubernetes.io/projected/77b501fc-ef62-4552-ad41-c85b1e05199b-kube-api-access-dw9nx\") pod \"perf-node-gather-daemonset-8wm82\" (UID: \"77b501fc-ef62-4552-ad41-c85b1e05199b\") " pod="openshift-must-gather-gb9tq/perf-node-gather-daemonset-8wm82" Jul 13 09:48:49.709558 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:49.709531 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77b501fc-ef62-4552-ad41-c85b1e05199b-sys\") pod \"perf-node-gather-daemonset-8wm82\" (UID: \"77b501fc-ef62-4552-ad41-c85b1e05199b\") " pod="openshift-must-gather-gb9tq/perf-node-gather-daemonset-8wm82" Jul 13 09:48:49.709673 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:49.709566 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/77b501fc-ef62-4552-ad41-c85b1e05199b-lib-modules\") pod \"perf-node-gather-daemonset-8wm82\" (UID: \"77b501fc-ef62-4552-ad41-c85b1e05199b\") " pod="openshift-must-gather-gb9tq/perf-node-gather-daemonset-8wm82" Jul 13 09:48:49.709673 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:49.709590 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/77b501fc-ef62-4552-ad41-c85b1e05199b-proc\") pod \"perf-node-gather-daemonset-8wm82\" (UID: \"77b501fc-ef62-4552-ad41-c85b1e05199b\") " pod="openshift-must-gather-gb9tq/perf-node-gather-daemonset-8wm82" Jul 13 09:48:49.709673 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:49.709640 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77b501fc-ef62-4552-ad41-c85b1e05199b-sys\") pod \"perf-node-gather-daemonset-8wm82\" (UID: \"77b501fc-ef62-4552-ad41-c85b1e05199b\") " pod="openshift-must-gather-gb9tq/perf-node-gather-daemonset-8wm82" Jul 13 09:48:49.709673 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:49.709643 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/77b501fc-ef62-4552-ad41-c85b1e05199b-podres\") pod \"perf-node-gather-daemonset-8wm82\" (UID: \"77b501fc-ef62-4552-ad41-c85b1e05199b\") " pod="openshift-must-gather-gb9tq/perf-node-gather-daemonset-8wm82" Jul 13 09:48:49.709843 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:49.709713 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dw9nx\" (UniqueName: \"kubernetes.io/projected/77b501fc-ef62-4552-ad41-c85b1e05199b-kube-api-access-dw9nx\") pod \"perf-node-gather-daemonset-8wm82\" (UID: \"77b501fc-ef62-4552-ad41-c85b1e05199b\") " pod="openshift-must-gather-gb9tq/perf-node-gather-daemonset-8wm82" Jul 13 09:48:49.709843 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:49.709721 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/77b501fc-ef62-4552-ad41-c85b1e05199b-proc\") pod \"perf-node-gather-daemonset-8wm82\" (UID: \"77b501fc-ef62-4552-ad41-c85b1e05199b\") " pod="openshift-must-gather-gb9tq/perf-node-gather-daemonset-8wm82" Jul 13 09:48:49.709843 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:49.709731 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/77b501fc-ef62-4552-ad41-c85b1e05199b-podres\") pod \"perf-node-gather-daemonset-8wm82\" (UID: \"77b501fc-ef62-4552-ad41-c85b1e05199b\") " pod="openshift-must-gather-gb9tq/perf-node-gather-daemonset-8wm82" Jul 13 09:48:49.709843 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:49.709721 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/77b501fc-ef62-4552-ad41-c85b1e05199b-lib-modules\") pod \"perf-node-gather-daemonset-8wm82\" (UID: \"77b501fc-ef62-4552-ad41-c85b1e05199b\") " pod="openshift-must-gather-gb9tq/perf-node-gather-daemonset-8wm82" Jul 13 09:48:49.719690 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:49.719662 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dw9nx\" (UniqueName: \"kubernetes.io/projected/77b501fc-ef62-4552-ad41-c85b1e05199b-kube-api-access-dw9nx\") pod \"perf-node-gather-daemonset-8wm82\" (UID: \"77b501fc-ef62-4552-ad41-c85b1e05199b\") " pod="openshift-must-gather-gb9tq/perf-node-gather-daemonset-8wm82" Jul 13 09:48:49.801007 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:49.800959 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gb9tq/perf-node-gather-daemonset-8wm82" Jul 13 09:48:49.940972 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:49.940946 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gb9tq/perf-node-gather-daemonset-8wm82"] Jul 13 09:48:49.942150 ip-10-0-129-99 kubenswrapper[2552]: W0713 09:48:49.942115 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod77b501fc_ef62_4552_ad41_c85b1e05199b.slice/crio-8951882559b7b1d2f2cca7e4a5786a5475a860f92d7f3b1067644cc80332646a WatchSource:0}: Error finding container 8951882559b7b1d2f2cca7e4a5786a5475a860f92d7f3b1067644cc80332646a: Status 404 returned error can't find the container with id 8951882559b7b1d2f2cca7e4a5786a5475a860f92d7f3b1067644cc80332646a Jul 13 09:48:50.558331 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:50.558304 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6f648df7b4-7n2bp_d3775975-f09b-4db7-8711-18775737dfd5/console/0.log" Jul 13 09:48:50.595254 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:50.595221 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gb9tq/perf-node-gather-daemonset-8wm82" event={"ID":"77b501fc-ef62-4552-ad41-c85b1e05199b","Type":"ContainerStarted","Data":"4187a9dcd80f4cd418de7edc1b77d2fc766766c661ee05f2b1aee370ebad29e4"} Jul 13 09:48:50.595415 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:50.595309 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gb9tq/perf-node-gather-daemonset-8wm82" Jul 13 09:48:50.595415 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:50.595334 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gb9tq/perf-node-gather-daemonset-8wm82" event={"ID":"77b501fc-ef62-4552-ad41-c85b1e05199b","Type":"ContainerStarted","Data":"8951882559b7b1d2f2cca7e4a5786a5475a860f92d7f3b1067644cc80332646a"} Jul 13 09:48:50.613184 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:50.613143 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gb9tq/perf-node-gather-daemonset-8wm82" podStartSLOduration=1.6131224579999999 podStartE2EDuration="1.613122458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 09:48:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 09:48:50.612479864 +0000 UTC m=+2414.492435131" watchObservedRunningTime="2026-07-13 09:48:50.613122458 +0000 UTC m=+2414.493077723" Jul 13 09:48:52.414574 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:52.414545 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-85jbg_08c6cfc3-a46b-447e-b470-a2f19a4ddcce/dns/0.log" Jul 13 09:48:52.439353 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:52.439334 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-85jbg_08c6cfc3-a46b-447e-b470-a2f19a4ddcce/kube-rbac-proxy/0.log" Jul 13 09:48:52.550972 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:52.550953 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-d9vsl_1a1d3bed-1a9d-42ce-b6b5-454d4aaac7da/dns-node-resolver/0.log" Jul 13 09:48:53.226943 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:53.226917 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-s8xqc_777385c6-f18f-4560-902e-67b4b1885fcd/node-ca/0.log" Jul 13 09:48:54.163228 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:54.163186 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4wzv25_5ccd1a61-cd7d-41c1-8236-76d04bc28b65/istio-proxy/0.log" Jul 13 09:48:54.349826 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:54.349793 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d64b4d488-d2lnz_d895d638-812a-437a-a2b0-5cb7a8eb4e0c/kube-auth-proxy/0.log" Jul 13 09:48:54.403751 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:54.403727 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-cmw8z_8863895d-7df7-4937-a6a5-06472e72737a/istio-proxy/0.log" Jul 13 09:48:54.455512 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:54.455453 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-cmw8z_8863895d-7df7-4937-a6a5-06472e72737a/istio-proxy/1.log" Jul 13 09:48:54.666176 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:54.666155 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-77b8c6656d-wxkr8_b9a3e094-4012-4054-83f7-9e753c83e8ad/router/0.log" Jul 13 09:48:55.434009 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:55.433976 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-m5k9t_1e384dd9-ffe8-45e5-81b1-8caf1f660fc5/serve-healthcheck-canary/0.log" Jul 13 09:48:56.015424 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:56.015399 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nllgk_c913a302-9335-4e64-bffc-22ce094e431b/kube-rbac-proxy/0.log" Jul 13 09:48:56.040972 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:56.040945 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nllgk_c913a302-9335-4e64-bffc-22ce094e431b/exporter/0.log" Jul 13 09:48:56.067042 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:56.067014 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nllgk_c913a302-9335-4e64-bffc-22ce094e431b/extractor/0.log" Jul 13 09:48:56.611071 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:56.611048 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gb9tq/perf-node-gather-daemonset-8wm82" Jul 13 09:48:58.407623 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:58.407569 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-xxjmb_0c3c450a-0391-4d1c-9db9-b9c1d3215a42/manager/0.log" Jul 13 09:48:58.732863 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:58.732796 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-lx9jr_70f4fd41-cd64-4ba4-9bc4-95a4c77971e0/server/0.log" Jul 13 09:48:58.831021 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:58.830996 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-gm9js_75c93628-a23e-4cb5-98eb-3134b0a39bec/manager/0.log" Jul 13 09:48:59.052392 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:59.052337 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4_c7e7373d-bf27-43fd-ace3-776f08e8c726/manager/0.log" Jul 13 09:48:59.076386 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:59.076371 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cd9bbfd9f-kr7p4_c7e7373d-bf27-43fd-ace3-776f08e8c726/copy-manifests/0.log" Jul 13 09:48:59.604140 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:59.604114 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4wzv25_5ccd1a61-cd7d-41c1-8236-76d04bc28b65/istio-proxy/0.log" Jul 13 09:48:59.772979 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:59.772951 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d64b4d488-d2lnz_d895d638-812a-437a-a2b0-5cb7a8eb4e0c/kube-auth-proxy/0.log" Jul 13 09:48:59.812586 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:59.812562 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-cmw8z_8863895d-7df7-4937-a6a5-06472e72737a/istio-proxy/0.log" Jul 13 09:48:59.867674 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:59.867615 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-cmw8z_8863895d-7df7-4937-a6a5-06472e72737a/istio-proxy/1.log" Jul 13 09:48:59.995628 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:48:59.995603 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-77b8c6656d-wxkr8_b9a3e094-4012-4054-83f7-9e753c83e8ad/router/0.log" Jul 13 09:49:01.226184 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:49:01.226150 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4wzv25_5ccd1a61-cd7d-41c1-8236-76d04bc28b65/istio-proxy/0.log" Jul 13 09:49:01.402228 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:49:01.402190 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d64b4d488-d2lnz_d895d638-812a-437a-a2b0-5cb7a8eb4e0c/kube-auth-proxy/0.log" Jul 13 09:49:01.453872 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:49:01.453846 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-cmw8z_8863895d-7df7-4937-a6a5-06472e72737a/istio-proxy/0.log" Jul 13 09:49:01.507895 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:49:01.507871 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-cmw8z_8863895d-7df7-4937-a6a5-06472e72737a/istio-proxy/1.log" Jul 13 09:49:01.648334 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:49:01.648306 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-77b8c6656d-wxkr8_b9a3e094-4012-4054-83f7-9e753c83e8ad/router/0.log" Jul 13 09:49:06.554101 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:49:06.554070 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bggk8_53caa5bd-dca1-4654-a407-0fc895185601/kube-multus-additional-cni-plugins/0.log" Jul 13 09:49:06.577245 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:49:06.577202 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bggk8_53caa5bd-dca1-4654-a407-0fc895185601/egress-router-binary-copy/0.log" Jul 13 09:49:06.597353 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:49:06.597332 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bggk8_53caa5bd-dca1-4654-a407-0fc895185601/cni-plugins/0.log" Jul 13 09:49:06.623697 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:49:06.623675 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bggk8_53caa5bd-dca1-4654-a407-0fc895185601/bond-cni-plugin/0.log" Jul 13 09:49:06.643430 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:49:06.643404 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bggk8_53caa5bd-dca1-4654-a407-0fc895185601/routeoverride-cni/0.log" Jul 13 09:49:06.663749 ip-10-0-129-99 kubenswrapper[2552]: I0713 09:49:06.663729 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bggk8_53caa5bd-dca1-4654-a407-0fc895185601/whereabouts-cni-bincopy/0.log"